GStreamer is a
pipeline-based
multimedia framework that links together a wide variety of media processing systems to complete complex workflows. For instance, GStreamer can be used to build a system that reads files in one format, processes them, and exports them in another. The formats and processes can be changed in a plug and play fashion.
GStreamer supports a wide variety of media-handling components, including simple
audio
Audio most commonly refers to sound, as it is transmitted in signal form. It may also refer to:
Sound
*Audio signal, an electrical representation of sound
*Audio frequency, a frequency in the audio spectrum
* Digital audio, representation of sou ...
playback, audio and video playback,
recording,
streaming
Streaming media is multimedia that is delivered and consumed in a continuous manner from a source, with little or no intermediate storage in network elements. ''Streaming'' refers to the delivery method of content, rather than the content i ...
and editing. The pipeline design serves as a base to create many types of
multimedia
Multimedia is a form of communication that uses a combination of different content forms such as text, audio, images, animations, or video into a single interactive presentation, in contrast to tradit ...
applications such as
video editors,
transcoders, streaming media broadcasters and
media players.
It is designed to work on a variety of operating systems, e.g. the
BSDs,
OpenSolaris
OpenSolaris () is a discontinued open-source computer operating system based on Solaris and created by Sun Microsystems. It was also, perhaps confusingly, the name of a project initiated by Sun to build a developer and user community around t ...
,
Android,
macOS
macOS (; previously OS X and originally Mac OS X) is a Unix operating system developed and marketed by Apple Inc. since 2001. It is the primary operating system for Apple's Mac computers. Within the market of desktop and la ...
,
iOS,
Windows
Windows is a group of several proprietary graphical operating system families developed and marketed by Microsoft. Each family caters to a certain sector of the computing industry. For example, Windows NT for consumers, Windows Server for se ...
,
OS/400
IBM i (the ''i'' standing for ''integrated'') is an operating system developed by IBM for IBM Power Systems. It was originally released in 1988 as OS/400, as the sole operating system of the IBM AS/400 line of systems. It was renamed to i5/OS ...
.
GStreamer is
free and open-source software
Free and open-source software (FOSS) is a term used to refer to groups of software consisting of both free software and open-source software where anyone is freely licensed to use, copy, study, and change the software in any way, and the source ...
subject to the terms of the
LGPL-2.1-or-later and is being hosted at
freedesktop.org.
Distribution and adoption
The
GNOME desktop environment, a heavy user of GStreamer, has included GStreamer since GNOME version 2.2 and encourages GNOME and
GTK applications to use it. Other projects also use or support it, such as the
Phonon
In physics, a phonon is a collective excitation in a periodic, elastic arrangement of atoms or molecules in condensed matter, specifically in solids and some liquids. A type of quasiparticle, a phonon is an excited state in the quantum mechani ...
media framework and the
Songbird
A songbird is a bird belonging to the suborder Passeri of the perching birds (Passeriformes). Another name that is sometimes seen as the scientific or vernacular name is Oscines, from Latin ''oscen'', "songbird". The Passeriformes contains 5000 ...
media player. It is also used in the
Webkit
WebKit is a browser engine developed by Apple and primarily used in its Safari web browser, as well as on the iOS and iPadOS version of any web browser. WebKit is also used by the BlackBerry Browser, PlayStation consoles beginning from the ...
browser engine.
GStreamer also operates in embedded devices like the
Jolla Phone, the
Palm Pre
The Palm Pre , styled as palm prē, is a multitasking smartphone that was designed and marketed by Palm with a multi-touch screen and a sliding keyboard. The smartphone was the first to use Palm's Linux-based mobile operating system, webOS. ...
,
Tizen
Tizen () is a Linux-based mobile operating system backed by the Linux Foundation, mainly developed and used primarily by Samsung Electronics.
The project was originally conceived as an HTML5-based platform for mobile devices to succeed MeeGo. ...
and the
Nokia 770,
N800
The Nokia N800 Internet tablet is a wireless Internet appliance from Nokia, originally announced at the Las Vegas CES 2007 Summit in January 2007. N800 allows the user to browse the Internet and communicate using Wi-Fi networks or with m ...
,
N810,
N900 and
N9 Internet Tablets running the
Maemo operating system.
In addition to source code releases, the GStreamer project provides binary builds for Android, iOS, OSX and Windows.
The
LIGO Laboratory make use of GStreamer to simulate and analyze gravitational wave data. The GStreamer interface is called GstLAL.
Software architecture
GStreamer is written in the
C programming language
''The C Programming Language'' (sometimes termed ''K&R'', after its authors' initials) is a computer programming book written by Brian Kernighan and Dennis Ritchie, the latter of whom originally designed and implemented the language, as well a ...
with the
type system
In computer programming, a type system is a logical system comprising a set of rules that assigns a property called a type to every "term" (a word, phrase, or other set of symbols). Usually the terms are various constructs of a computer progra ...
based on
GObject
The GLib Object System, or GObject, is a free software library providing a portable object system and transparent cross-language interoperability. GObject is designed for use both directly in C programs to provide object-oriented C-based APIs ...
and the GLib 2.0 object model.
Language bindings
A library written in one programming language may be used in another language if
bindings are written; GStreamer has a range of bindings for various languages such as
Go,
Python,
Rust
Rust is an iron oxide, a usually reddish-brown oxide formed by the reaction of iron and oxygen in the catalytic presence of water or air moisture. Rust consists of hydrous iron(III) oxides (Fe2O3·nH2O) and iron(III) oxide-hydroxide (FeO( ...
,
Vala,
C++,
Perl
Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offic ...
,
GNU Guile,
C# and
Ruby
A ruby is a pinkish red to blood-red colored gemstone, a variety of the mineral corundum ( aluminium oxide). Ruby is one of the most popular traditional jewelry gems and is very durable. Other varieties of gem-quality corundum are called ...
.
Overview
GStreamer processes media by connecting a number of processing ''elements'' into a ''pipeline''. Each element is provided by a
plug-in. Elements can be grouped into ''bins'', which can be further aggregated, thus forming a hierarchical graph. This is an example of a
filter graph.
Elements communicate by means of ''pads''. A ''source pad'' on one element can be connected to a ''sink pad'' on another. When the pipeline is in the ''playing'' state, data ''buffers'' flow from the source pad to the sink pad. Pads negotiate the kind of data that will be sent using ''capabilities''.
The diagram to the right could exemplify playing an
MP3 file using GStreamer. The file source reads an MP3 file from a computer's hard-drive and sends it to the MP3 decoder. The decoder decodes the file data and converts it into
PCM
Pulse-code modulation (PCM) is a method used to digitally represent sampled analog signals. It is the standard form of digital audio in computers, compact discs, digital telephony and other digital audio applications. In a PCM stream, the am ...
samples which then pass to the sound-driver. The sound-driver sends the PCM sound samples to the computer's speakers.
Plug-ins
GStreamer uses a
plug-in architecture which makes the most of GStreamer's functionality implemented as
shared libraries
In computer science, a library is a collection of non-volatile resources used by computer programs, often for software development. These may include configuration data, documentation, help data, message templates, pre-written code and ...
. GStreamer's base functionality contains functions for registering and loading plug-ins and for providing the fundamentals of all classes in the form of
base classes. Plug-in libraries get dynamically loaded to support a wide spectrum of
codecs,
container format
A container format (informally, sometimes called a wrapper) or metafile is a file format that allows multiple data streams to be embedded into a single file, usually along with metadata for identifying and further detailing those streams. No ...
s,
input/output drivers and effects.
Plug-ins can be installed semi-automatically when they are first needed. For that purpose distributions can register a backend that resolves feature-descriptions to package-names.
Since version 0.9, the plug-ins come grouped into three sets (named after the film ''
The Good, the Bad and the Ugly
''The Good, the Bad and the Ugly'' ( it, Il buono, il brutto, il cattivo, literally "The good, the ugly, the bad") is a 1966 Italian epic spaghetti Western film directed by Sergio Leone and starring Clint Eastwood as "the Good", Lee Van Clee ...
'').
Individual distributions may further sub-classify these plug-ins: for example
Ubuntu
Ubuntu ( ) is a Linux distribution based on Debian and composed mostly of free and open-source software. Ubuntu is officially released in three editions: '' Desktop'', ''Server'', and ''Core'' for Internet of things devices and robots. All ...
groups the "bad" and "ugly" sets into the "Universe" or the
"Multiverse" components.
In addition, there is a GStreamer FFmpeg plug-in (called ''gst-libav'' for historic reasons) that extends the number of supported media formats.
Video acceleration
There are various
SIP blocks that can do the computations to decode certain video codecs, such as
PureVideo,
UVD
Unified Video Decoder (UVD, previously called Universal Video Decoder) is the name given to AMD's dedicated video decoding ASIC. There are multiple versions implementing a multitude of video codecs, such as H.264 and VC-1.
UVD was introduced with ...
,
QuickSync Video,
TI Ducati and more. Such needs to be supported by the
device driver
In computing, a device driver is a computer program that operates or controls a particular type of device that is attached to a computer or automaton. A driver provides a software interface to hardware devices, enabling operating systems and o ...
, which in turn provides one or multiple
interfaces
Interface or interfacing may refer to:
Academic journals
* ''Interface'' (journal), by the Electrochemical Society
* '' Interface, Journal of Applied Linguistics'', now merged with ''ITL International Journal of Applied Linguistics''
* '' Int ...
, like
VDPAU,
VAAPI,
Distributed Codec Engine
Distributed Codec Engine (DCE) is an API and its implementation as software library ("''libdce''") by Texas Instruments. The library was released under the Revised BSD License and some additional terms.
It enables and provides remote access to h ...
or
DXVA DirectX Video Acceleration (DXVA) is a Microsoft API specification for the Microsoft Windows and Xbox 360 platforms that allows video decoding to be hardware-accelerated. The pipeline allows certain CPU-intensive operations such as iDCT, mo ...
to end-user software like MPlayer to access this hardware and offload computation to it.
* It is possible to use
Video Coding Engine with GStreamer through the
OpenMAX IL
OpenMAX (Open Media Acceleration), often shortened as "OMX", is a non-proprietary and royalty-free cross-platform set of C-language programming interfaces. It provides abstractions for routines that are especially useful for processing of audio, ...
wrapper plugin . This is for example possible on the
Raspberry Pi.
* The
SIP core present on some
Texas Instruments
Texas Instruments Incorporated (TI) is an American technology company headquartered in Dallas, Texas, that designs and manufactures semiconductors and various integrated circuits, which it sells to electronics designers and manufacturers globa ...
SoCs is also accessible through GStreamer: , , .
*
VDPAU and
VAAPI are supported with
GNOME Videos >= 2.28.0 and GStreamer >= 0.10.26 since 2010
*
Broadcom Crystal HD Crystal HD is the Broadcom's hardware SIP core that performs video decoding.
Product description
Crystal HD has been available as single chip high-definition advanced media processors BCM70012 (codenamed Link) and BCM70015 (codenamed Flea); these ...
is supported
Media formats
The Good, Bad and Ugly GStreamer plugins mentioned earlier provide, alongside processing elements/filters of all kinds, support for a wide variety of file formats, protocols and multimedia codecs. In addition to those, support for more than a hundred compression formats (including
MPEG-1
MPEG-1 is a standard for lossy compression of video and audio. It is designed to compress VHS-quality raw digital video and CD audio down to about 1.5 Mbit/s (26:1 and 6:1 compression ratios respectively) without excessive quality loss, making ...
,
MPEG-2
MPEG-2 (a.k.a. H.222/H.262 as was defined by the ITU) is a standard for "the generic coding of moving pictures and associated audio information". It describes a combination of lossy video compression and lossy audio data compression methods, ...
,
MPEG-4
MPEG-4 is a group of international standards for the compression of digital audio and visual data, multimedia systems, and file storage formats. It was originally introduced in late 1998 as a group of audio and video coding formats and related t ...
,
H.261,
H.263
H.263 is a video compression standard originally designed as a low-bit-rate compressed format for videotelephony. It was standardized by the ITU-T Video Coding Experts Group (VCEG) in a project ending in 1995/1996. It is a member of the H.26x fam ...
,
H.264,
RealVideo
RealVideo, or also spelled as Real Video, is a suite of proprietary video compression formats developed by RealNetworks – the specific format changes with the version. It was first released in 1997 and was at version 10. RealVideo is supported ...
,
MP3,
WMV
Windows Media Video (WMV) is a series of video codecs and their corresponding video coding formats developed by Microsoft. It is part of the Windows Media framework. WMV consists of three distinct codecs: The original video compression technology ...
, etc.) is transparently provided through the ''gst-libav'' plug-in.
History and development
Early days
Erik Walthinsen founded the GStreamer project in 1999. Many of its core design ideas came from a research project at the
Oregon Graduate Institute
The Oregon Graduate Center was a unique, private, postgraduate-only research university in Washington County, Oregon, on the west side of Portland, from 1963 to 2001. The center was renamed the Oregon Graduate Institute in 1989. The Institute me ...
.
Wim Taymans
Wim Odilia Georges Taymans is a Belgian software developer based in Malaga, Spain. Taymans started his career in multimedia development on the Commodore 64 writing various games and demos. He was known in the Commodore 64 coding community under ...
joined the project soon thereafter and greatly expanded on many aspects of the system. Many other software developers have contributed since then.
The first major release was 0.1.0 which was announced on 11 January 2001.
Not long after, GStreamer picked up its first commercial backer. Towards the end of January 2001, they hired Erik Walthinsen to develop methods for embedding GStreamer in smaller (
cell phone
A mobile phone, cellular phone, cell phone, cellphone, handphone, hand phone or pocket phone, sometimes shortened to simply mobile, cell, or just phone, is a portable telephone that can make and receive telephone call, calls over a radio freq ...
-class) devices. Another RidgeRun employee, Brock A. Frazier, designed the GStreamer logo. RidgeRun later struggled financially and had to lay off its staff, including Erik Walthinsen. GStreamer progress was mostly unaffected.
The project released a series of major releases with 0.2.0 coming out in July 2001, 0.4.0 in September 2002, and 0.8.0 in March 2004. During that period the project also changed its versioning strategy and while the first releases were simply new versions, later on the middle number started signifying release series. This meant the project did release a string of 0.6.x and 0.8.x releases which was meant to stay
binary compatible within those release series. Erik Walthinsen more or less left GStreamer development behind during this time, focusing on other ventures.
All release series, the project face difficulties. Every series is not very popular in the Linux community mostly because of stability issues and a serious lack of features compared to competing projects like
Xine,
MPlayer, and
VLC. The project also suffers a lack of leadership as Wim Taymans, the project lead since Erik Walthinsen had left, had largely stopped participating.
The 0.10 series
In 2004, a new company was founded,
Fluendo, which wanted to use GStreamer to write a streaming server
Flumotion and also provide multimedia solutions for GStreamer. During this time, Fluendo hired most of the core developers including Wim Taymans and attracted the support of companies such as
Nokia
Nokia Corporation (natively Nokia Oyj, referred to as Nokia) is a Finnish multinational telecommunications, information technology, and consumer electronics corporation, established in 1865. Nokia's main headquarters are in Espoo, Finland, i ...
and
Intel
Intel Corporation is an American multinational corporation and technology company headquartered in Santa Clara, California. It is the world's largest semiconductor chip manufacturer by revenue, and is one of the developers of the x86 ser ...
to bring GStreamer to a professional level and drive community adoption.
With Wim Taymans back at the helm, the core of GStreamer was redesigned and became what is the current 0.10.x series, which had its first release (0.10.0) in December 2005.
It has maintained
API and
ABI compatibility since.
With a new stable core in place, GStreamer gained in popularity in 2006, being used by media players including
Totem
A totem (from oj, ᑑᑌᒼ, italics=no or '' doodem'') is a spirit being, sacred object, or symbol that serves as an emblem of a group of people, such as a family, clan, lineage, or tribe, such as in the Anishinaabe clan system.
While ''the ...
,
Rhythmbox and
Banshee
A banshee ( ; Modern Irish , from sga, ben síde , "woman of the fairy mound" or "fairy woman") is a female spirit in Irish folklore who heralds the death of a family member, usually by screaming, wailing, shrieking, or keening. Her name i ...
with many more to follow. It was also adopted by corporations such as
Nokia
Nokia Corporation (natively Nokia Oyj, referred to as Nokia) is a Finnish multinational telecommunications, information technology, and consumer electronics corporation, established in 1865. Nokia's main headquarters are in Espoo, Finland, i ...
,
Motorola
Motorola, Inc. () was an American multinational telecommunications company based in Schaumburg, Illinois, United States. After having lost $4.3 billion from 2007 to 2009, the company split into two independent public companies, Motorola ...
,
Texas Instruments
Texas Instruments Incorporated (TI) is an American technology company headquartered in Dallas, Texas, that designs and manufactures semiconductors and various integrated circuits, which it sells to electronics designers and manufacturers globa ...
,
Freescale,
Tandberg, and
Intel
Intel Corporation is an American multinational corporation and technology company headquartered in Santa Clara, California. It is the world's largest semiconductor chip manufacturer by revenue, and is one of the developers of the x86 ser ...
.
In 2007, most of the core GStreamer developers left Fluendo, including GStreamer maintainer Wim Taymans who went on to co-found
Collabora Multimedia together with other GStreamer veterans, while others joined
Sun Microsystems
Sun Microsystems, Inc. (Sun for short) was an American technology company that sold computers, computer components, software, and information technology services and created the Java programming language, the Solaris operating system, ZFS, t ...
,
Oblong Industries
An oblong is a non-square rectangle.
Oblong may also refer to:
Places
* Oblong, Illinois, a village in the United States
* Oblong Township, Crawford County, Illinois, United States
* A strip of land on the New York-Connecticut border in the Uni ...
, and
Songbird
A songbird is a bird belonging to the suborder Passeri of the perching birds (Passeriformes). Another name that is sometimes seen as the scientific or vernacular name is Oscines, from Latin ''oscen'', "songbird". The Passeriformes contains 5000 ...
.
Between June 2012 and August 2014, GStreamer 0.10 was also distributed by
Collabora
Collabora is a global private company headquartered in Cambridge, United Kingdom, with offices in Cambridge and Montreal. It provides open-source consultancy, training and products to companies.
Collabora's initial focus was instant messaging ...
and
Fluendo as a multiplatform
SDK,
on the third-part
gstreamer.comwebsite (rather tha
gstreamer.freedesktop.orgfor the upstream community project). The goal was to provide application developers with a SDK that would be functionally identical on Windows,
Mac OS X
macOS (; previously OS X and originally Mac OS X) is a Unix operating system developed and marketed by Apple Inc. since 2001. It is the primary operating system for Apple's Mac computers. Within the market of desktop and lap ...
, iOS, and Android. The SDK initiative aimed to facilitate the commercial adoption of the GStreamer project, as it provided a standardized entry point to developing multimedia applications with GStreamer, without needing to build the entire platform by oneself. Users of the SDK also benefited fro
documentation, tutorials and instructions specific to that SDK.
The 1.x series
GStreamer 1.0 was released on September 24, 2012. The 1.x series is parallel installable to GStreamer 0.10 to ease the transition, and provides many architectural advantages over the 0.10 series. Generally speaking, GStreamer 1.0 brought significant improvements for:
* Embedded processors support, lower power consumption, offloading work to specialized hardware units (such as
DSPs)
* Hardware accelerated video decoding/encoding using
GPUs
* Zero-copy memory management (avoiding unnecessary roundtrips between the CPU and GPU) for better performance and lower power consumption
* Dynamic pipelines
* API and code cleanups
Beyond the technical improvements, the 1.x series is also defined by a new release versioning scheme. As the GStreamer roadmap explains, all 1.x.y versions carry a -1.0 API version suffix and have a stable API/ABI. The API/ABI can only be broken by a new major release series (i.e.: 2.x); however, there are currently no plans for a 2.0 release series. Until then, the new version numbering scheme can be used to predict the intended use of each release. The roadmap cites some examples:
* 1.0.0, 1.0.1, 1.0.2, 1.0.3... stable release and follow-up bug-fix releases
* 1.1.0, 1.1.1, 1.1.2, 1.1.3... pre-releases, development version leading up to 1.2.0
* 1.2.0, 1.2.1, 1.2.2, 1.2.3... stable release and follow-up bug-fix releases
* 1.3.0...
* 1.4.0...
* etc.
In March 2013, the GStreamer project maintainers issued a statement to clarify that the 0.10 series is no longer maintained. The statement reasserted the GStreamer project's willingness to help application and plugin developers migrate to the new technology, and hinted that those for whom switching to the 1.x series was still considered impossible could seek assistance from various consulting companies.
1.2 added support for
DASH
The dash is a punctuation mark consisting of a long horizontal line. It is similar in appearance to the hyphen but is longer and sometimes higher from the baseline. The most common versions are the endash , generally longer than the hyphen ...
adaptive streaming,
JPEG2000 images,
VP9 and
Daala video, and decoding-only support for
WebP.
Version 1.14 was released on March 19, 2018,
adding support for
WebRTC,
AV1,
Nvidia NVDEC, and
Secure Reliable Transport
Secure Reliable Transport (SRT) is an open source video transport protocol that utilises the UDP transport protocol.
Overview
SRT provides connection and control, reliable transmission similar to TCP; however, it does so at the applica ...
, among other changes.
See also
*
List of software that uses GStreamer
*
Libvisual
*
OggConvert – a simple GUI front-end
*
SoundConverter – a GUI front-end based on GStreamer and GTK for transcoding digital audio files
*
Pitivi – a video editor based on GStreamer
References
External links
*
Delphi Bridge (Binding)
{{GNOME
Free multimedia software
Free software programmed in C
Freedesktop.org libraries
GNOME libraries
Multimedia frameworks
Software that uses Meson
Collabora