1. What kinds of encoders are supported?

   The protocol specification does not limit the encoder selection. However, the current Apple implementation should interoperate with encoders that produce MPEG-2 Transport Streams containing H.264 video and AAC audio (HE-AAC or AAC-LC). Encoders that are capable of broadcasting the output stream over UDP should also be compatible with the current implementation of the Apple provided segmenter software.

   Apple has tested the current implementation with the following commercial encoders:

   • Inlet Technologies Spinnaker 7000
   • Envivio 4Caster C4
2. What are the specifics of the video and audio formats supported?

   Although the protocol specification does not limit the video and audio formats, the current Apple implementation supports the following formats:

   • Video: H.264 Baseline Level 3.0
   • Audio:
     • HE-AAC or AAC-LC up to 48 kHz, stereo audio
     • MP3 (MPEG-1 Audio Layer 3) 8 kHz to 48 kHz, stereo audio
     Note: iPad, iPhone 3G, and iPod touch (2nd generation and later) support H.264 Baseline 3.1. If your app runs on older versions of iPhone or iPod touch, however, you should use H.264 Baseline 3.0 for compatibility.
3. What duration should media files be?

   The main point to consider is that shorter segments result in more frequent refreshes of the index file, which might create unnecessary network overhead for the client. Longer segments will extend the inherent latency of the broadcast and initial startup time. A duration of 10 seconds of media per file seems to strike a reasonable balance for most broadcast content.

4. How many files should be in listed in the index file during a continuous, ongoing session?

   The specification requires at least 3 media files be listed in the index file, but the optimum number may be larger. The client identifies an ongoing session by the lack of an #EXT-X-ENDLIST tag in the index file. The client does not allow the user to seek into the last two files in the index for ongoing broadcasts. Periodically, the client requests a new copy of the index.

   The important point to consider when choosing the optimum number is that the number of files available during a live session constrains the client’s behavior when doing play/pause and seeking operations. The longer the list, the longer the client can be paused without losing its place in the broadcast, the further back in the broadcast a new client begins, and the wider the time range within which the client can seek. The trade-off is that a longer index file adds to network overhead—during live broadcasts, the clients are all refreshing the index file regularly, so it does add up, even though the index file is typically small.

   Another point to consider is that clients typically request new copies of the index file at higher rate when the index contains a shorter list of files.

5. What data rates are supported?

   The data rate that a content provider chooses for a stream is most influenced by the target client platform and the expected network topology. The streaming protocol itself places no limitations on the data rates that can be used. The current implementation has been tested using audio-video streams with data rates as low as 100 Kbps and as high as 1.6 Mbps to iPhone. Audio-only streams at 64 Kbps are recommended as alternates for delivery over slow cellular connections.

   Note: If the data rate exceeds the available bandwidth, there is more latency before startup and the client may have to pause to buffer more data periodically. During a broadcast using an index file that provides a moving window into the content, the client will eventually fall behind in such cases, causing one or more segments to be dropped. In the case of VOD, no segments are lost, but inadequate bandwidth does cause slower startup and periodic stalling while data buffers.
6. What is a .ts file?

   A .ts file contains an MPEG-2 Transport Stream. This is a file format that encapsulates a series of encoded media samples—typically audio and video. The file format supports a variety of compression formats, including MP3 audio, AAC audio, H.264 video, and so on. Not all compression formats are currently supported in the Apple HTTP Live Streaming implementation, however. (For a list of currently supported formats, see “Media Encoder.”)

7. What is an .M3U8 file?

   An .M3U8 file is a extensible playlist file format. It is an m3u playlist containing UTF-8 encoded text. The m3u file format is a de facto standard playlist format suitable for carrying lists of media file URLs. This is the format used as the index file for HTTP Live Streaming. For details, see IETF Internet-Draft of the HTTP Live Streaming specification.

8. How does the client software determine when to switch streams?

   The current implementation of the client observes the effective bandwidth while playing a stream. If a higher-quality stream is available and the bandwidth appears sufficient to support it, the client switches to a higher quality. If a lower-quality stream is available and the current bandwidth appears insufficient to support the current stream, the client switches to a lower quality.

   Note: For seamless transitions between alternate streams, the audio portion of the stream should be identical in all versions.
9. Where can I find a copy of the media stream segmenter from Apple?

   The media stream segmenter, file stream segmenter, and other tools are in the /usr/bin/ directory of Mac OS X computers, version 10.6 and later. These tools are frequently updated, so you should download the current version of the HTTP Live Streaming Tools from the Apple Developer website. See “Download the Tools” for details.

10. What settings are recommended for a typical HTTP stream, with alternates, for use with the media segmenter from Apple?

    Your encoder should produce MPEG-2 transport stream (.ts) files with the following characteristics for the Apple segmenter:

    • H.264 Baseline 3.0 video
    • Keyframes every 3 seconds
    • HE-AAC (version 1) stereo audio at 44.1 kHz
    • Four streams:
      • Cellular Fallback—Audio only or audio with still image, 64 Kbps
      • Low—96 Kbps video, 64 Kbps audio
      • Medium—256 Kbps video, 64 Kbps audio
      • High—800 Kbps video, 64 Kbps audio

    These settings are the current recommendations. There are also certain requirements. The current mediastreamsegmenter tool works only with MPEG-2 Transport Streams as defined in ISO/IEC 13818. The transport stream must contain H.264 (MPEG-4, part 10) video and AAC or MPEG audio. If AAC audio is used, it must have ADTS headers. H.264 video access units must use Access Unit Delimiter NALs, and must be in unique PES packets.

    The segmenter also has a number of user-configurable settings. You can obtain a list of the command line arguments and their meanings by typing man mediastreamsegmenter from the Terminal application. A target duration (length of the media segments) of 10 seconds is recommended, and is the default if no target duration is specified.

11. How can I specify what codecs or H.264 profile are required to play back my stream?

    Use the CODECS attribute of the EXT-X-STREAM-INF tag. When this attribute is present, it must include all codecs and profiles required to play back the stream. The following values are currently recognized:

    AAC-LC	“mp4a.40.2″
    HE-AAC	“mp4a.40.5″
    MP3	“mp4a.40.34″
    H.264 Baseline Profile level 3.0	“avc1.42001e” or “avc1.66.30” Note: Use “avc1.66.30” for compatibility with iPhone OS versions 3.0 to 3.12.
    H.264 Main Profile level 3.0	“avc1.4d001e” or “avc1.77.30” Note: Use “avc1.77.30” for compatibility with iPhone OS versions 3.0 to 3.12.

    The attribute value must be in quotes. If multiple values are specified, one set of quotes is used to contain all values, and the values are separated by commas. An example follows.

    #EXTM3U

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=500000

    mid_video_index.m38u

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=800000

    wifi_video_index.m38u

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=3000000, CODECS="avc1.4d001e, mp4a.40.5"

    h264main_heaac_index.m38u

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=64000, CODECS="mp4a.40.5"

    aacaudio_index.m38u

12. How can I create an audio-only stream from audio/video input?

    Add the -audio-only argument when invoking the stream or files segmenter.

13. How can I add a still image to an audio-only stream?

    Use the -meta-file argument when invoking the stream or file segmenter with -meta-type=picture to add an image to every segment. For example, this would add an image named poster.jpg to every segment of an audio stream created from the file track01.mp3:

    mediafilesegmenter -f /Dir/outputFile -a --meta-file=poster.jpg --meta-type=picture track01.mp3

    Remember that the image is typically resent every ten seconds, so it’s best to keep the file size small.

14. How can I specify an audio-only alternate to an audio-video stream?

    Use the CODECS and BANDWIDTH attributes of the EXT-X-STREAM-INF tag together.

    The BANDWIDTH attribute specifies the bandwidth required for each alternate stream. If the available bandwidth is enough for the audio alternate, but not enough for the lowest video alternate, the client switches to the audio stream.

    If the CODECS attribute is included, it must list all codecs required to play the stream. If only an audio codec is specified, the stream is identified as audio-only. Currently, it is not required to specify that a stream is audio-only, so use of the CODECS attribute is optional.

    The following is an example that specifies video streams at 500 Kbps for fast connections, 150 Kbps for slower connections, and an audio-only stream at 64 Kbps for very slow connections. All the streams should use the same 64 Kbps audio to allow transitions between streams without an audible disturbance.

    #EXTM3U

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=500000

    mid_video_index.m38u

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=150000

    3g_video_index.m38u

    #EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=64000, CODECS="mp4a.40.5"

    aacaudio_index.m38u

15. What are the hardware requirements or recommendations for servers?

    See question #1 for encoder hardware recommendations.

    The Apple stream segmenter is capable of running on any Intel-based Mac. We recommend using a Mac with two Ethernet network interfaces, such as a Mac Pro or an XServe. One network interface can be used to obtain the encoded stream from the local network, while the second network interface can provide access to a wider network.

16. Does the Apple implementation of HTTP Live Streaming support DRM?

    No. However, media can be encrypted and key access can be limited using HTTPS authentication.

17. What client platforms are supported?

    iPhone, iPad, and iPod touch (requires iPhone OS version 3.0 or later) or any device with QuickTime X or later installed.

18. Is the protocol specification available?

    Yes. The protocol specification is an IETF Internet-Draft, at http://tools.ietf.org/html/draft-pantos-http-live-streaming.

19. Does the client cache content?

    The index file can contain an instruction to the client that content should not be cached. Otherwise, the client may cache data for performance optimization when seeking within the media.

20. Is this a real-time delivery system?

    No. It has inherent latency corresponding to the size and duration of the media files containing stream segments. At least one segment must fully download before it can be viewed by the client, and two may be required to ensure seamless transitions between segments. In addition, the encoder and segmenter must create a file from the input; the duration of this file is the minimum latency before media is available for download. Typical latency with recommended settings is in the neighborhood of 30 seconds.

21. What is the latency?

    Approximately 30 seconds, with recommended settings. See question #15.

22. Do I need to use a hardware encoder?

    No. Using the protocol specification, it is possible to implement a software encoder.

23. What advantages does this approach have over RTP/RTSP?

    HTTP is less likely to be disallowed by routers, NAT, or firewall settings. No ports need to be opened that are commonly closed by default. Content is therefore more likely to get through to the client in more locations and without special settings. HTTP is also supported by more content-distribution networks, which can affect cost in large distribution models. In general, more available hardware and software works unmodified and as intended with HTTP than with RTP/RTSP. Expertise in customizing HTTP content delivery using tools such as PHP is also more widespread.

    Also, HTTP Live Streaming is supported in Safari and the media player framework on iPhone OS. RTSP streaming is not supported.

24. Why is my stream’s overall bit rate higher than the sum of the audio and video bitrates?

    MPEG-2 transport streams can include substantial overhead. They utilize fixed packet sizes that are padded when the packet contents are smaller than the default packet size. Encoder and multiplexer implementations vary in their efficiency at packing media data into these fixed packet sizes. The amount of padding can vary with frame rate, sample rate, and resolution.

25. How can I reduce the overhead and bring the bit rate down?

    Using a more efficient encoder can reduce the amount of overhead, as can tuning the encoder settings. Also, the -optimize argument can be passed to the Apple mediastreamsegmenter. This removes some unnecessary padding and can significantly reduce the overhead, particularly for low-bandwidth streams.

26. Do all media files have to be part of the same MPEG-2 Transport Stream?

    No. You can mix media files from different transport streams, as long as they are separated by EXT-X-DISCONTINUITY tags. See the protocol specification for more detail. For best results, however, all video media files should have the same height and width dimensions in pixels.

Hypertext transfer protocol, also known as HTTP, is universally supported, and is much more effective at bypassing firewalls and routers that may otherwise deny traffic that use MMS or RTSP protocols. Why is this? Because almost all internet traffic uses the HTTP protocol, so if a router or firewall rejected HTTP traffic, they would be blocking out most of the internet.

The shining benefit of HTTP streaming is that it allows content distributors to avoid the costly implementation of traditional streaming media servers, which is a huge benefit to small companies just getting started in streaming video. Although it works in a completely different way than RTSP, the effect is still the same. With on-demand video (as opposed to live video), the user can skip forward before the video is finished downloading, much like other types of on-demand streaming. And with live video, the viewer cannot skip forward at all, which reproduces the experience of watching a live broadcast on TV.

So, how do content producers utilize this technology? One option is to purchase a monthly iPhone & mobile streaming video hosting package with GravityLab Multimedia, which makes it easy to get up and running with HTTP streaming.

How our iPhone streaming video hosting works

1. You purchase an account
2. We send you a welcome e-mail with instructions on how to upload your files with any FTP program, and how to start a live broadcast

On-demand streaming:

1. You upload a video file, and it is available for viewing instantly
2. You link to the video via your your web site
   e.g. http://arecibo.gravlab.com:1935/iphone/mp4:your_account/your_video.mp4/playlist.m3u8

Live broadcasting on iPhone 3.0 and later:

1. You Download Quicktime Broadcaster, a free program for broadcasting video from any computer with OS X.
2. You configure the broadcaster according to your welcome e-mail, and then press the Broadcast button
3. You link viewers to http://arecibo.gravlab.com:1935/iphonelive/your_account.sdp/playlist.m3u8

File requirements

• Container format: .mp4, .m4v, .mov
• Dimensions: 480×320
• Codec: H.264 (Low-Complexity version of the H.264 Baseline Profile OR Baseline Profile up to Level 3.0)
• Audio: AAC-LC audio up to 160 Kbps, 48kHz, stereo audio
• Frame rate: up to 30 fps
• Bit rate: up to 1.5Mbps (200Kbps recommended)

More reading:

Apple’s official HTTP streaming documentation
RTSP/RTP/RTCP Streaming media overview
MMS Streaming media overview

Optimal settings for the BlackBerry Pearl 8100

Optimal settings for the BlackBerry 8800/8300/9000

WAP sites are generally one file that contains many sub-pages within the master WML page. Each page is defined with a special tag called “<card>”. This is why WML pages are known as “decks”, because they contain multiple cards. Each card can contain text, links, input fields, tasks, images, and more. You can provide navigation from card to card with links that work just like HTML anchors (with a # sign to indicate the id of the card.)

Example WAP code

Here is an example WAP site with two cards, each one containing a link to a different type of 3gp mobile video.

<?xml version="1.0"?>
<!DOCTYPE wml PUBLIC "-//WAPFORUM//DTD WML 1.1//EN"
"http://www.wapforum.org/DTD/wml_1.1.xml">
<wml>
<!-- Your site code goes in between the wml tags -->
<card id="card_one" title="Card 1">
<p> <a href="rtsp://streaming-server-address/sample-movie.3gp">3gpp movie </a> </p>
<a href="#card_two">Next Page</a>
</card>
<card id="card_two" title="Card 2">
<p> <a href="rtsp://streaming-server-address/sample-movie.sdp">Live 3gpp stream</a></p>
</card>
</wml>

Images

Though generally not encouraged for mobile sites, you can use images. To place images on your WAP site, you must use the .wbmp format. There are many free converters out there, such as WAP Tiger
For example:

<img  src="http://yoursite.com/logo.wbmp" align="middle" height="100" width="100"/>

Tables (not supported by all phones)

If you wish to use tables, there are a few differences between WML and HTML that you must note: First, the tbody tag does not exist in WML. Second, the attributes of the table tag are “title” (the title of the table), “align” (C=center, L=left and R=right) and “columns” (the number of the columns)

<table align="C" title="label" columns="3">
<tr>
<td>col 1</td>
<td>col 2</td>
<td>col 3</td>
</tr>
</table>

Finishing Up

Once you have uploaded your .wml file to your web server, you can direct mobile users to a URL such as “http://www.yoursite.com/your-wap-site.wml”

Quicktime Mov
http://helix.gravlab.com:554/sample_100kbit.mov
http://helix.gravlab.com:554/sample_300kbit.mov

Quicktime Mp4
http://helix.gravlab.com:554/sample_100kbit.mp4
http://helix.gravlab.com:554/sample_300kbit.mp4

Quicktime H.264
http://helix.gravlab.com:554/sample_h264_100kbit.mp4
http://helix.gravlab.com:554/sample_h264_1mbit.mp4

Mobile 3GP
http://helix.gravlab.com:554/sample.3gp
http://helix.gravlab.com:554/sample_50kbit.3gp

Windows Media
http://media.gravlab.com/powered.wmv
http://media.gravlab.com/racecar.wmv

“Microsoft^® Expression^® Encoder, a feature of Expression Media, offers encoding, enhancement, and publishing of rich media experiences to Microsoft Silverlight. ”

Features Include:

Better Encoding

VC-1 advanced authoring SDK integration: by adopting the new VC-1 authoring SDK, Expression Encoder 2 is able to author better quality video faster. We’ve exposed a set of presets for tuning the encoder as well as detailed settings with comprehensive tooltips for the advanced compressionist.

New profiles, and video codecs for streaming media: Encoder 2 supports a number of new modes and codecs including Quality VBR, WMA Pro (stereo), WMA Lossless, 24 bit audio and audio/video encoding stream selection.

• Improved MPEG source support: We now include an MPEG 2 decoder in the product to give a consistent story across platforms. The decoder we ship supports transport streams (so you are able to work with sources that originated from HDV cameras. Seeking performance and accuracy has been greatly improved.
• Better multi-core performance: we are not done with performance enhancements by any means but have made decent progress in Expression Encoder 2, particularly on multi-core systems.
• Improved aspect ratio handling: by default, the new Profile Adaptive mode is selected which preserves the width specified in the profile but varies the height according to the aspect ratio of the source. Translation: it just works. Additionally, when Source compression is used the aspect ratio mode as well as pixel aspect settings are fixed up to ensure source compression will just work.
• Pre-processing enhancements: the new Pre-processing palette contains a number of settings for controlling resizing (Bicubic, Lanczos, Supersampling in addition to nearest neighbor and bilinear), de-interlacing and audio options. As well as detecting the need to de-interlace from the header of the file, the user can manually override the defaults. In addition to the V1 capabilities, Expression Encoder 2 includes inverse telecine as well as a new pixel-adaptive de-interlacer. On the audio side, there is a volume leveling filter as well as a volume control.

Better Silverlight Interactive experiences

The second bucket of functionality covers improvements to Silverlight output.

• New templates: We’ve revisited both the functionality and the design of our core Silverlight Templates. As before these support core functionality such as Chapter points and close captioning and are user extensible.
• Template Parameters: We have had a lot of requests for more and easier control of how templates behave. Examples include not automatically playing (or buffering) the video, being able to control the background color of the HTML and more. In Expression Encoder 2 we’ve added parameters to templates. The parameter system is extensible enabling developers to easily expose custom things e.g. skin primary color etc.

Download Expression Encoder for Microsoft Silverlight

GravityLab offers comprehensive iPhone streaming video hosting packages to deliver your video to the iPhone and other mobile cell phone devices. Follow this link to review our iPhone video plans and pricing.

When the iPhone was unveiled, the mobile phone market was instantly changed. Consumers raced to buy them, and sales exceeded all expectation. The iPhone now dominates the mobile phone market. But what does this mean for mobile streaming?

Update 3/19/2008: The iPhone continues to garner market share across the mobile phone landscape, continuing to reward its loyal customers with a host of new features and updates. The latest software, iPhone 3.0, announced earlier this week, will now allow for HTTP streaming, which is compatible with GravityLab Multimedia’s video hosting plans.

While the current 2.0 version of the iPhone software doesn’t support true streaming, iPhone users can still watch video from your web site. You can easily provide a link to MP4, MOV, and M4V files encoded with the following settings:

1) Codec must be MPEG-4 or H.264
2) Resolution can be up to 720×480 (but its actual display resolution is 480×320)
3) Frame-rate: 25fps

All GravityLab Multimedia customers are able to use our iPhone video hosting to provide their viewers with crystal clear, high speed video, available anywhere, anytime.

This typical streaming encoded movie file can be delivered by any web server, using common protocols such as HTTP and FTP, just as if it were an HTML file or a JPEG image. It is necessary only to name the file correctly and associate the filename extension with the correct MIME type on the server. (The correct filename extension for QuickTime movies is .mov, and the correct MIME type is 'video/quicktime'.)

When a file is delivered over a network or downloaded over the Internet, the entire file is not available immediately, but a typical QuickTime movie can be played while it downloads. This is called progressive download, or Fast Start. It works because the movie atom is stored at the beginning of the file, so QuickTime knows how to interpret the movie sample data even before it arrives, and because the movie data is intelligently interleaved with respect to display time.

It is also possible to create a movie file with the sample data stored first, followed by the movie data structure. This is not usually desirable, because the entire file must download before QuickTime can interpret the sample data. You can correct this kind of data inversion simply by opening the movie file in QuickTime and saving it as a new, self-contained file. QuickTime stores the movie data structure at the beginning of the file by default.

A QuickTime movie file may contain only a movie data structure, pointing to sample data in other files or URLs. In most cases, this type of movie can also play as the movie data downloads, because, again, the movie data structure allows QuickTime to interpret the incoming data, and because the data source for each track is specified independently, causing the network to perform a kind of interleaving by delivering all of the media independently and simultaneously. Obviously, this kind of interleaving is less reliable than the deliberate interleaving QuickTime does when creating a self-contained movie file, so playback may not always be as smooth.

When the bandwidth of a connection meets or exceeds the data rate of the movie, a well-formed QuickTime movie file can play as it downloads. This kind of progressive download, or Fast Start movie, provides the same user experience as real-time streaming.

If the connection is not fast enough to play the movie in real time, you can either wait until the download completes or play as much of the movie as has downloaded at a given time. QuickTime can even estimate the required download time and begin playback when it calculates that enough data has arrived to play the movie smoothly (because the remaining data is expected to arrive by the time it is needed).

QuickTime movies can also be delivered using real-time protocols such as RTP and RTSP. This requires a streaming server, such as the QuickTime Streaming Server or Darwin Streaming Server. To stream movies in real time, the server requires information about how to packetize each track in the movie. This information is stored in special tracks in a QuickTime movie, known as hint tracks. There are functions in the API for adding hint tracks to existing movies, as well as flags that can be used to tell QuickTime to create hint tracks when saving a movie to disk.

Movies with hint tracks can also be delivered using HTTP or FTP protocols for progressive download, but additional bandwidth is needed to carry the hint tracks, which are used only for streaming. Consequently, it is best to determine how you will deliver a movie before saving it as hinted or nonhinted.

In addition to progressive download and real-time streaming of stored movie files, QuickTime supports broadcasting, the creation of one or more real-time streams from real-time sources, such as cameras or microphones. This involves capturing the incoming data, compressing it to the desired bandwidth, and generating streams of outgoing packets, all in real time. The QuickTime broadcast API is currently available for the Mac OS only; it is not available for Windows or Java.