
Adaptive bitrate streaming is a technique used in
streaming multimedia over
computer network
A computer network is a collection of communicating computers and other devices, such as printers and smart phones. In order to communicate, the computers and devices must be connected by wired media like copper cables, optical fibers, or b ...
s.
While in the past most video or audio streaming technologies utilized streaming protocols such as
RTP with
RTSP, today's adaptive streaming technologies are based almost exclusively on
HTTP
HTTP (Hypertext Transfer Protocol) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, wher ...
, and are designed to work efficiently over large distributed HTTP networks.
Adaptive bitrate streaming works by detecting a user's
bandwidth and
CPU capacity in real time, adjusting the quality of the media stream accordingly. It requires the use of an
encoder which encodes a single source media (video or audio) at multiple
bit rate
In telecommunications and computing, bit rate (bitrate or as a variable ''R'') is the number of bits that are conveyed or processed per unit of time.
The bit rate is expressed in the unit bit per second (symbol: bit/s), often in conjunction ...
s. The player client
[DASH at ITEC, VLC Plugin, DASHEncoder and Dataset](_blank)
by C. Mueller, S. Lederer, C. Timmerer switches between streaming the different encodings depending on available resources.
This results in providing very little
buffering, faster start times and a good experience for both high-end and low-end connections.
More specifically, adaptive bitrate streaming is a method of video streaming over HTTP where the source content is encoded at multiple bit rates. Each of the different bit rate streams are segmented into small multi-second parts.
The segment size can vary depending on the particular implementation, but they are typically between two and ten seconds.
First, the client downloads a
manifest file
In computer programming, a manifest file is a Data file, file containing metadata for a group of accompanying files that are part of a set or coherent unit. For example, the files of a computer program may have a manifest describing the name, Soft ...
that describes the available stream segments and their respective bit rates. During stream start-up, the client usually requests the segments from the lowest bit rate stream. If the client finds that the network throughput is greater than the bit rate of the downloaded segment, then it will request a higher bit rate segment. Later, if the client finds that the network throughput has deteriorated, it will request a lower bit rate segment. An adaptive bitrate (ABR) algorithm in the client performs the key function of deciding which bit rate segments to download, based on the current state of the network. Several types of ABR algorithms are in commercial use:
throughput
Network throughput (or just throughput, when in context) refers to the rate of message delivery over a communication channel in a communication network, such as Ethernet or packet radio. The data that these messages contain may be delivered ov ...
-based algorithms use the throughput achieved in recent prior downloads for decision-making (e.g., throughput rule i
dash.js, buffer-based algorithms use only the client's current buffer level (e.g., BOLA i
dash.js, and hybrid algorithms combine both types of information (e.g., DYNAMIC i
dash.js.
Current uses
Post-production
Post-production, also known simply as post, is part of the process of filmmaking, video production, audio production, and photography. Post-production includes all stages of production occurring after principal photography or recording indivi ...
houses,
content delivery network
A content delivery network (CDN) or content distribution network is a geographically distributed network of proxy servers and their data centers. The goal is to provide high availability and performance ("speed") by distributing the service spat ...
s and studios use adaptive bit rate technology in order to provide consumers with higher quality video using less manpower and fewer resources. The creation of multiple video outputs, particularly for adaptive bit rate streaming, adds great value to consumers. If the technology is working properly, the end user or consumer's content should play back without interruption and potentially go unnoticed. Media companies have been actively using adaptive bit rate technology for many years now and it has essentially become standard practice for high-end streaming providers; permitting little buffering when streaming high-resolution feeds (begins with low-resolution and climbs).
Benefits of adaptive bitrate streaming
Traditional server-driven adaptive bitrate streaming provides consumers of streaming media with the best-possible experience, since the media server automatically adapts to any changes in each user's network and playback conditions. The media and entertainment industry also benefit from adaptive bitrate streaming. As the video space grows, content delivery networks and video providers can provide customers with a superior viewing experience. Adaptive bitrate technology requires additional
encoding
In communications and Data processing, information processing, code is a system of rules to convert information—such as a letter (alphabet), letter, word, sound, image, or gesture—into another form, sometimes data compression, shortened or ...
, but simplifies the overall workflow and creates better results.
HTTP-based adaptive bitrate streaming technologies yield additional benefits over traditional server-driven adaptive bitrate streaming. First, since the streaming technology is built on top of
HTTP
HTTP (Hypertext Transfer Protocol) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, wher ...
, contrary to
RTP-based adaptive streaming, the packets have no difficulties traversing firewall and
NAT devices. Second, since HTTP streaming is purely client-driven, all adaptation logic resides at the client. This reduces the requirement of persistent connections between server and client application. Furthermore, the server is not required to maintain session state information on each client, increasing scalability. Finally, existing HTTP delivery infrastructure, such as HTTP caches and servers can be seamlessly adopted.
A scalable
CDN is used to deliver media streaming to an Internet audience. The CDN receives the stream from the source at its Origin server, then replicates it to many or all of its
Edge cache servers. The end-user requests the stream and is redirected to the "closest" Edge server. This can be tested using
libdash
libdash is a computer software library which provides an object-oriented interface to the Dynamic Adaptive Streaming over HTTP (DASH) standard. It is also the official reference implementation of the ISO/IEC MPEG-DASH standard, and maintained by t ...
[libdash: Open-source DASH client library](_blank)
by bitmovin and the Distributed DASH (D-DASH) dataset,
which has several mirrors across Europe, Asia and the US. The use of HTTP-based adaptive streaming allows the Edge server to run a simple HTTP server software, whose licence cost is cheap or free, reducing software licensing cost, compared to costly media server licences (e.g. Adobe Flash Media Streaming Server). The CDN cost for HTTP streaming media is then similar to HTTP web caching CDN cost.
History
Adaptive bit rate over HTTP was created by the DVD Forum at the WG1 Special Streaming group in October 2002. The group was co-chaired by
Toshiba
is a Japanese multinational electronics company headquartered in Minato, Tokyo. Its diversified products and services include power, industrial and social infrastructure systems, elevators and escalators, electronic components, semiconductors ...
and
Phoenix Technologies
Phoenix Technologies Ltd. is an American company that designs, develops and supports core system software for personal computers and other computing devices. The company's products commonly referred to as BIOS (Basic Input/Output System) or fir ...
, The expert group count with the collaboration of
Microsoft
Microsoft Corporation is an American multinational corporation and technology company, technology conglomerate headquartered in Redmond, Washington. Founded in 1975, the company became influential in the History of personal computers#The ear ...
,
Apple Computer
Apple Inc. is an American multinational corporation and technology company headquartered in Cupertino, California, in Silicon Valley. It is best known for its consumer electronics, software, and services. Founded in 1976 as Apple Computer Co ...
,
DTS Inc.,
Warner Brothers
Warner Bros. Entertainment Inc. (WBEI), commonly known as Warner Bros. (WB), is an American filmed entertainment studio headquartered at the Warner Bros. Studios complex in Burbank, California and the main namesake subsidiary of Warner Bro ...
,
20th Century Fox
20th Century Studios, Inc., formerly 20th Century Fox, is an American film studio, film production and Film distributor, distribution company owned by the Walt Disney Studios (division), Walt Disney Studios, the film studios division of the ...
,
Digital Deluxe,
Disney
The Walt Disney Company, commonly referred to as simply Disney, is an American multinational mass media and entertainment industry, entertainment conglomerate (company), conglomerate headquartered at the Walt Disney Studios (Burbank), Walt Di ...
,
Macromedia
Macromedia, Inc. was an American graphics, multimedia, and web development software company headquartered in San Francisco, California, that made products such as Adobe Flash, Flash and Adobe Dreamweaver, Dreamweaver. It was purchased by its riv ...
and
Akamai. The technology was originally called DVDoverIP and was an integral effort of the DVD ENAV book. The concept came from storing MPEG-1 and MPEG-2 DVD TS Sectors into small 2KB files, which will be served using an HTTP server to the player. The MPEG-1 segments provided the lower bandwidth stream, while the MPEG-2 provided a higher bit rate stream. The original XML schema provided a simple playlist of bit rates, languages and url servers. The first working prototype was presented to the DVD Forum by Phoenix Technologies at the
Harman Kardon Lab in Villingen Germany.
Implementations
Adaptive bit rate streaming was introduced by Move Networks in 2006 and is now being developed and utilized by
Adobe Systems
Adobe Inc. ( ), formerly Adobe Systems Incorporated, is an American software, computer software company based in San Jose, California. It offers a wide range of programs from web design tools, photo manipulation and vector creation, through to ...
,
Apple
An apple is a round, edible fruit produced by an apple tree (''Malus'' spp.). Fruit trees of the orchard or domestic apple (''Malus domestica''), the most widely grown in the genus, are agriculture, cultivated worldwide. The tree originated ...
,
Microsoft
Microsoft Corporation is an American multinational corporation and technology company, technology conglomerate headquartered in Redmond, Washington. Founded in 1975, the company became influential in the History of personal computers#The ear ...
and
Octoshape. In October 2010, Move Networks was awarded a patent for their adaptive bit rate streaming (US patent number 7818444).
Dynamic Adaptive Streaming over HTTP (DASH)
Dynamic Adaptive Streaming over HTTP (DASH), also known as MPEG-DASH, is the only adaptive bit-rate HTTP-based streaming solution that is an international standard
MPEG-DASH technology was developed under
MPEG
The Moving Picture Experts Group (MPEG) is an alliance of working groups established jointly by International Organization for Standardization, ISO and International Electrotechnical Commission, IEC that sets standards for media coding, includ ...
. Work on DASH started in 2010 and became a Draft International Standard in January 2011 and an International Standard in November 2011.
The MPEG-DASH standard was published a
ISO/IEC 23009-1:2012
in April, 2012.
MPEG-DASH is a technology related to Adobe Systems
Adobe Inc. ( ), formerly Adobe Systems Incorporated, is an American software, computer software company based in San Jose, California. It offers a wide range of programs from web design tools, photo manipulation and vector creation, through to ...
HTTP Dynamic Streaming, Apple Inc.
Apple Inc. is an American multinational corporation and technology company headquartered in Cupertino, California, in Silicon Valley. It is best known for its consumer electronics, software, and services. Founded in 1976 as Apple Comput ...
HTTP Live Streaming
HTTP Live Streaming (also known as HLS) is an HTTP-based adaptive bitrate streaming communications protocol developed by Apple Inc. and released in 2009. Support for the protocol is widespread in media players, web browsers, mobile devices, and ...
(HLS) and Microsoft
Microsoft Corporation is an American multinational corporation and technology company, technology conglomerate headquartered in Redmond, Washington. Founded in 1975, the company became influential in the History of personal computers#The ear ...
Smooth Streaming.[Updates on DASH](_blank)
– blog entry DASH is based on Adaptive HTTP streaming (AHS) in 3GPP
The 3rd Generation Partnership Project (3GPP) is an umbrella term for a number of standards organizations which develop protocols for mobile telecommunications. Its best known work is the development and maintenance of:
* GSM and related 2G and ...
Release 9 and on HTTP Adaptive Streaming (HAS) in Open IPTV Forum Release 2.[ETSI 3GP]
3GPP TS 26.247; Transparent end-to-end packet-switched streaming service (PSS); Progressive Download and Dynamic Adaptive Streaming over HTTP (3GP-DASH)
/ref>
As part of their collaboration with MPEG, 3GPP Release 10 has adopted DASH (with specific codecs and operating modes) for use over wireless networks.
The goal of standardizing an adaptive streaming solution is to assure the market that the solution can work universally, unlike other solutions that are more specific to certain vendors, such as Apple’s HLS, Microsoft’s Smooth Streaming, or Adobe’s HDS.
Available implementations are the HTML5-based ''bitdash'' MPEG-DASH player as well as the open source C++-based DASH client access library libdash
libdash is a computer software library which provides an object-oriented interface to the Dynamic Adaptive Streaming over HTTP (DASH) standard. It is also the official reference implementation of the ISO/IEC MPEG-DASH standard, and maintained by t ...
of bitmovin GmbH, the DASH tools of the Institute of Information Technology (ITEC) at Alpen-Adria University Klagenfurt, the multimedia framework of the GPAC group at Telecom ParisTech, and the dash.js player of the DASH-IF.
Apple HTTP Live Streaming (HLS)
HTTP Live Streaming (HLS) is an HTTP-based media streaming communications protocol implemented by Apple Inc.
Apple Inc. is an American multinational corporation and technology company headquartered in Cupertino, California, in Silicon Valley. It is best known for its consumer electronics, software, and services. Founded in 1976 as Apple Comput ...
as part of QuickTime X and iOS
Ios, Io or Nio (, ; ; locally Nios, Νιός) is a Greek island in the Cyclades group in the Aegean Sea. Ios is a hilly island with cliffs down to the sea on most sides. It is situated halfway between Naxos and Santorini. It is about long an ...
. HLS supports both live and Video on demand
Video on demand (VOD) is a media distribution system that allows users to access videos, television shows and films Digital distribution, digitally on request. These multimedia are accessed without a traditional video playback device and a typica ...
content. It works by breaking down media streams or files into short pieces (media segments) which are stored as MPEG-TS or fragmented MP4 files. This is typically done at multiple bitrates using a stream or file segmenter application, also known as a packager. One such segmenter implementation is provided by Apple. Additional packagers are available, including free / open source offerings like Google's Shaka Packager and various commercial tools as well - such as Unified Streaming. The segmenter is also responsible for producing a set of playlist files in the M3U8 format which describe the media chunks. Each playlist is specific to a given bitrate, and contains the relative or absolute URLs to the chunks for that bitrate. The client is then responsible for requesting the appropriate playlist depending on available bandwidth.
HTTP Live Streaming is a standard feature in the iPhone 3.0 and newer versions.
Apple has submitted its solution to the IETF
The Internet Engineering Task Force (IETF) is a standards organization for the Internet standard, Internet and is responsible for the technical standards that make up the Internet protocol suite (TCP/IP). It has no formal membership roster ...
for consideration as an Informational Request for Comments
A Request for Comments (RFC) is a publication in a series from the principal technical development and standards-setting bodies for the Internet, most prominently the Internet Engineering Task Force (IETF). An RFC is authored by individuals or ...
. This was officially accepted as A number of proprietary and open source solutions exist for both the server implementation (segmenter) and the client player.
HLS streams can be identified by the playlist URL format extension of or MIME type of application/vnd.apple.mpegurl. These adaptive streams can be made available in many different bitrates and the client device interacts with the server to obtain the best available bitrate which can reliably be delivered.
Playback of HLS is supported on many platforms including Safari and native apps on macOS / iOS, Microsoft Edge on Windows 10, ExoPlayer on Android, and the Roku platform. Many Smart TVs also have native support for HLS. Playing HLS on other platforms like Chrome / Firefox is typically achieved via a browser / JavaScript player implementation. Many open source and commercial players are available including hls.js, video.js http-streaming, BitMovin, JWPlayer, THEOplayer, etc.
Adobe HTTP Dynamic Streaming (HDS)
"HTTP Dynamic streaming is the process of efficiently delivering streaming video to users by dynamically switching among different streams of varying quality and size during playback. This provides users with the best possible viewing experience their bandwidth and local computer hardware ( CPU) can support. Another major goal of dynamic streaming is to make this process smooth and seamless to users, so that if up-scaling or down-scaling the quality of the stream is necessary, it is a smooth and nearly unnoticeable switch without disrupting the continuous playback."
The latest versions of Flash Player and Flash Media Server support adaptive bit-rate streaming over the traditional RTMP protocol, as well as HTTP
HTTP (Hypertext Transfer Protocol) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, wher ...
, similar to the HTTP-based solutions from Apple and Microsoft, HTTP dynamic streaming being supported in Flash Player 10.1 and later. HTTP-based streaming has the advantage of not requiring any firewall ports being opened outside of the normal ports used by web browsers. HTTP-based streaming also allows video fragments to be cached by browsers, proxies, and CDNs, drastically reducing the load on the source server.
Microsoft Smooth Streaming (MSS)
Smooth Streaming is an IIS Media Services extension that enables adaptive streaming of media to clients over HTTP. The format specification is based on the ISO base media file format and standardized by Microsoft as the Protected Interoperable File Format. Microsoft is actively involved with 3GPP
The 3rd Generation Partnership Project (3GPP) is an umbrella term for a number of standards organizations which develop protocols for mobile telecommunications. Its best known work is the development and maintenance of:
* GSM and related 2G and ...
, MPEG
The Moving Picture Experts Group (MPEG) is an alliance of working groups established jointly by International Organization for Standardization, ISO and International Electrotechnical Commission, IEC that sets standards for media coding, includ ...
and DECE organizations' efforts to standardize adaptive bit-rate HTTP streaming. Microsoft provides Smooth Streaming Client software development kits for Silverlight and Windows Phone 7, as well as a Smooth Streaming Porting Kit that can be used for other client operating systems, such as Apple iOS, Android, and Linux. IIS Media Services 4.0, released in November 2010, introduced a feature which enables Live Smooth Streaming H.264/AAC videos to be dynamically repackaged into the Apple HTTP Adaptive Streaming format and delivered to iOS devices without the need for re-encoding.
Microsoft has successfully demonstrated delivery of both live and on-demand 1080p HD video with Smooth Streaming to Silverlight clients. In 2010, Microsoft also partnered with NVIDIA to demonstrate live streaming of 1080p stereoscopic 3D video to PCs equipped with NVIDIA 3D Vision technology.
Common Media Application Format (CMAF)
CMAF is a presentation container format used for the delivery of both HLS and MPEG-DASH. Hence it is intended to simplify delivery of HTTP-based streaming media. It was proposed in 2016 by Apple and Microsoft and officially published in 2018.
QuavStreams Adaptive Streaming over HTTP
QuavStreams Adaptive Streaming is a multimedia streaming technology developed by Quavlive. The streaming server is an HTTP server that has multiple versions of each video, encoded at different bitrates and resolutions. The server delivers the encoded video/audio frames switching from one level to another, according to the current available bandwidth. The control is entirely server-based, so the client does not need special additional features. The streaming control employs feedback control theory. Currently, QuavStreams supports H.264/MP3 codecs muxed into the FLV container and VP8/Vorbis codecs muxed into the WEBM container.
Uplynk
Uplynk delivers HD adaptive bitrate streaming to multiple platforms, including iOS, Android, Windows Mac, Linux, and Roku, across various browser combinations, by encoding video in the cloud using a single non-proprietary adaptive streaming format. Rather than streaming and storing multiple formats for different platforms and devices, Uplynk stores and streams only one. The first studio to use this technology for delivery was Disney–ABC Television Group
Disney General Entertainment Content (DGEC), formerly ABC Group, Disney–ABC Television Group and the second incarnation of Walt Disney Television, is a division of the Disney Entertainment business segment of the Walt Disney Company that ove ...
, using it for video encoding for web, mobile and tablet streaming apps on the ABC Player, ABC Family and Watch Disney apps, as well as the live Watch Disney Channel, Watch Disney Junior, and Watch Disney XD.
Self-learning clients
In recent years, the benefits of self-learning algorithms in adaptive bitrate streaming have been investigated in academia. While most of the initial self-learning approaches are implemented at the server-side (e.g. performing admission control using reinforcement learning
Reinforcement learning (RL) is an interdisciplinary area of machine learning and optimal control concerned with how an intelligent agent should take actions in a dynamic environment in order to maximize a reward signal. Reinforcement learnin ...
or artificial neural network
In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks.
A neural network consists of connected ...
s), more recent research is focusing on the development of self-learning HTTP Adaptive Streaming clients. Multiple approaches have been presented in literature using the SARSA or Q-learning algorithm. In all of these approaches, the client state is modeled using, among others, information about the current perceived network throughput and buffer filling level. Based on this information, the self-learning client autonomously decides which quality level to select for the next video segment. The learning process is steered using feedback information, representing the Quality of Experience (QoE) (e.g. based on the quality level, the number of switches and the number of video freezes). Furthermore, it was shown that multi-agent Q-learning can be applied to improve QoE fairness among multiple adaptive streaming clients.
Criticisms
HTTP-based adaptive bit rate technologies are significantly more operationally complex than traditional streaming technologies. Some of the documented considerations are things such as additional storage and encoding costs, and challenges with maintaining quality globally. There have also been some interesting dynamics found around the interactions between complex adaptive bit rate logic competing with complex TCP flow control logic.
However, these criticisms have been outweighed in practice by the economics and scalability of HTTP delivery: whereas non-HTTP streaming solutions require massive deployment of specialized streaming server infrastructure, HTTP-based adaptive bit-rate streaming can leverage the same HTTP web servers used to deliver all other content over the Internet.
With no single clearly defined or open standard for the digital rights management
Digital rights management (DRM) is the management of legal access to digital content. Various tools or technological protection measures, such as access control technologies, can restrict the use of proprietary hardware and copyrighted works. DRM ...
used in the above methods, there is no 100% compatible way of delivering restricted or time-sensitive content to any device or player. This also proves to be a problem with digital rights management being employed by any streaming protocol.
The method of segmenting files into smaller files used by some implementations (as used by HTTP Live Streaming
HTTP Live Streaming (also known as HLS) is an HTTP-based adaptive bitrate streaming communications protocol developed by Apple Inc. and released in 2009. Support for the protocol is widespread in media players, web browsers, mobile devices, and ...
) could be deemed unnecessary due to the ability of HTTP clients to request byte ranges from a single video asset file that could have multiple video tracks at differing bit rates with the manifest file only indicating track number and bit rate. However, this approach allows for serving of chunks by any simple HTTP server and so therefore guarantees CDN compatibility. Implementations using byte ranges such as Microsoft Smooth Streaming require a dedicated HTTP server such as IIS to respond to the requests for video asset chunks.
See also
* Multiple description coding
*Hierarchical modulation
Hierarchical modulation, also called layered modulation, is one of the signal processing techniques for multiplexing and modulating multiple data streams into one single symbol stream, where base-layer symbols and enhancement-layer symbols are ...
– alternative with reduced storage and authoring demands
References
Further reading
The Next Big Thing in Video: Adaptive Bitrate Streaming
{{DEFAULTSORT:Adaptive Bit Rate
Streaming media systems
Internet broadcasting