The Ultimate Guide to Understanding Advanced Video Delivery with AIR for Mobile
Adobe AIR provides a very attractive abstraction layer for video playback on mobile devices, but a certain level of knowledge is required to take full advantage. Before we start, it is important to understand the difference between Adobe AIR and Flash Player on mobile devices. Flash Player on mobile runs in the browser, and allows to display Flash web content that otherwise wouldn’t be available. The challenge of this approach is iOS does not have Flash support, meaning for reach, mobile sites are not authored in Flash, but rather HTML. Adobe recognized this, and made appropriate adjustments late last year.
Adobe AIR in contrast is based on the application approach, and can combine Flash, HTML and native functionality – as result, it is possible to take advantage of the advanced video features that are widely deployed with Flash on the desktop, including adaptive streaming, content protection, ad monetization and beyond.
To reaffirm the commitment to Adobe AIR, Danny Winokur, Vice President & General Manager, Interactive Development at Adobe made the following the statement.
Our future work with Flash on mobile devices will be focused on enabling Flash developers to package native apps with Adobe AIR for all the major app stores.
The only platform that has not full support for the Adobe video stack is AIR for iOS due to iOS limitations, but it contains very legit workarounds to enable equivalent, fully App Store compliant, functionality within the same workflow, resulting in the same end user experience. Let’s look at the media capabilities of the different mobile platforms.
Protocols and codecs
Fully hardware accelerated H.264 video playback is supported on all AIR target mobile platforms including iOS. The supported protocols are slightly different on iOS though.
Other codecs are supported as well, but not HW accelerated.
Content protection is supported, but again with slight differences on iOS.
The figures above show that video features are mostly equivalent to the desktop, with minor differences on iOS. For a more in-depth overview how to use a unified workflow to include iOS video delivery, read the mobile strategy post, and the article comparing HDS with HLS.
In addition, here is a short video of HLS H.264 live streaming running in AIR for iOS, demonstrating how well it integrates.
The performance of AIR for Mobile video delivery depends on the target platform, but iOS and the Blackberry Playbook have very limited fragmentation, and the AIR video performance is equivalent to native. Thus the overall Apple encoding recommendations also apply to AIR for iOS, and the Playbook’s very well performing H.264 hardware decoder can consume most H.264 encoded content.
Android on the other side, with hundreds of different devices, is a lot more challenging, but also benefits most from video delivery with Adobe AIR. But before I dive into the specifics, let’s look at a crucial performance feature for mobile video delivery.
StageVideo, in contrast to the older video object, GPU accelerates 100% of the video pipeline, meaning best video quality, performance and battery life.
On Android the video object supports hardware decoding, but part of the video pipeline might use CPU. This is sufficient for smart phones, but not a good technique for HD quality video on tablets.
As you can see above, StageVideo has different characteristics on each platform, but the overall guidance is to detect StageVideo and use if available. The Open Source Media Framework automatically enables this functionality, but there is also a manual detection approach.
(Update: 09/08/12) While Android 2.2+ and 3.x require the use of the StageVideo API to take advantage of StageVideo, AIR on Android 4.x devices uses StageVideo automatic mode. This means all content will automatically use StageVideo, even if you use the regular video object, and not specifically the StageVideo API.
Android video fragmentation
Android as a video delivery platform is more challenging than a platform like iOS with very limited fragmentation. Android runs on hundreds of different devices, with different hardware, and GPU layers underneath it.
Adobe AIR provides an abstract layer on top of a complex and fragmented foundation, with a lot of continuous efforts invested in supporting different chipsets and drivers. As pointed out in a previous blog post, even the supported streaming protocols differ based on the Android version. Once built on top of AIR, the application will adopt to new devices with minimal efforts due to continuous AIR updates. That said, the Android team is working on decreasing the fragmentation. But to reach all devices today, a native approach without Adobe AIR in comparison is time consuming and often requires third party support to enhance e.g. the native streaming capabilities.
With such a diversity comes the need for a video encoding performance baseline to reach all devices with the sames encoding, which Adobe provides in detailed form, and with my short guide from 2010. In addition tablets with StageVideo and Android 3.1+ support 720p video playback, and the Kindle Fire offers high quality playback even on Android 2.3.
Note: In case you experience performance issues with 720p playback on Android 3.1+ devices with StageVideo enabled, ensure to download the latest AIR runtime from the Android Market.
If device specific video optimization is desired, it is possible to fairly accurately detect the device based on the method the AIR application Device Info uses, and deliver device optimized profiles.
As part of supporting the Flash Video ecosystem with Adobe AIR come all the monetization options, including video ads with Auditude, measurement with Omniture, and everything else that is part of the ecosystem.
To better understand how to convert your Flash video web experience into an Adobe AIR application, read the mobile strategy article.
Obviously there is more complexity to video delivery and the overall application approach, but this demonstrates the advantages of using a runtime with unified video features to reduce costs and increase scale. For successful examples, it’s worth checking out WatchESPN, Snagfilms or Psych Vision 2.
And as always, feedback and questions are welcome – this article is the result of reader comments from the recent survey.
Update 01/16/12: Added Source Code – 720p Video on iPad and Android Tablets with Adobe AIR.
Update 01/27/12: If you use the captive runtime approach, please update Flash Builder/Flash to the latest AIR version for 720p Android playback (AIR 220.127.116.1160 or higher)