In computing, serialization (or serialisation) is the process of translating a
data structure
In computer science, a data structure is a data organization, management, and storage format that is usually chosen for efficient access to data. More precisely, a data structure is a collection of data values, the relationships among them, a ...
or
object
Object may refer to:
General meanings
* Object (philosophy), a thing, being, or concept
** Object (abstract), an object which does not exist at any particular time or place
** Physical object, an identifiable collection of matter
* Goal, an ai ...
state into a format that can be stored (e.g.
files in secondary storage devices,
data buffers in primary storage devices) or transmitted (e.g.
data streams over
computer networks
A computer network is a set of computers sharing resources located on or provided by network nodes. The computers use common communication protocols over digital interconnections to communicate with each other. These interconnections are ma ...
) and reconstructed later (possibly in a different computer environment). When the resulting series of bits is reread according to the serialization format, it can be used to create a semantically identical clone of the original object. For many complex objects, such as those that make extensive use of
references
Reference is a relationship between objects in which one object designates, or acts as a means by which to connect to or link to, another object. The first object in this relation is said to ''refer to'' the second object. It is called a ''name'' ...
, this process is not straightforward. Serialization of object-oriented
object
Object may refer to:
General meanings
* Object (philosophy), a thing, being, or concept
** Object (abstract), an object which does not exist at any particular time or place
** Physical object, an identifiable collection of matter
* Goal, an ai ...
s does not include any of their associated
methods
Method ( grc, μέθοδος, methodos) literally means a pursuit of knowledge, investigation, mode of prosecuting such inquiry, or system. In recent centuries it more often means a prescribed process for completing a task. It may refer to:
*Scien ...
with which they were previously linked.
This process of serializing an object is also called
marshalling an object in some situations.
The opposite operation, extracting a data structure from a series of bytes, is deserialization, (also called unserialization or
unmarshalling In computer science, marshalling or marshaling (American English spelling, US spelling) is the process of transforming the Random-access memory, memory representation of an object (computer science), object into a data format suitable for Data stora ...
).
Uses
Methods of:
* transferring data through the wires (
messaging
A message is a discrete unit of communication intended by the source for consumption by some recipient or group of recipients. A message may be delivered by various means, including courier, telegraphy, carrier pigeon and electronic bus.
A ...
).
* storing data (in
database
In computing, a database is an organized collection of data stored and accessed electronically. Small databases can be stored on a file system, while large databases are hosted on computer clusters or cloud storage. The design of databases sp ...
s, on
hard disk drive
A hard disk drive (HDD), hard disk, hard drive, or fixed disk is an electro-mechanical data storage device that stores and retrieves digital data using magnetic storage with one or more rigid rapidly rotating platters coated with magnet ...
s).
*
remote procedure call
In distributed computing, a remote procedure call (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is coded as if it were a normal (l ...
s, e.g., as in
SOAP
Soap is a salt of a fatty acid used in a variety of cleansing and lubricating products. In a domestic setting, soaps are surfactants usually used for washing, bathing, and other types of housekeeping. In industrial settings, soaps are use ...
.
* distributing objects, especially in
component-based software engineering
Component-based software engineering (CBSE), also called component-based development (CBD), is a branch of software engineering that emphasizes the separation of concerns with respect to the wide-ranging functionality available throughout a give ...
such as
COM
Com or COM may refer to:
Computing
* COM (hardware interface), a serial port interface on IBM PC-compatible computers
* COM file, or .com file, short for "command", a file extension for an executable file in MS-DOS
* .com, an Internet top-level d ...
,
CORBA
The Common Object Request Broker Architecture (CORBA) is a standard defined by the Object Management Group (OMG) designed to facilitate the communication of systems that are deployed on diverse platforms. CORBA enables collaboration between sys ...
, etc.
* detecting changes in time-varying data.
For some of these features to be useful, architecture independence must be maintained. For example, for maximal use of distribution, a computer running on a different hardware architecture should be able to reliably reconstruct a serialized data stream, regardless of
endianness
In computing, endianness, also known as byte sex, is the order or sequence of bytes of a word of digital data in computer memory. Endianness is primarily expressed as big-endian (BE) or little-endian (LE). A big-endian system stores the most sig ...
. This means that the simpler and faster procedure of directly copying the memory layout of the data structure cannot work reliably for all architectures. Serializing the data structure in an architecture-independent format means preventing the problems of
byte ordering
In computing, endianness, also known as byte sex, is the order or sequence of bytes of a word of digital data in computer memory. Endianness is primarily expressed as big-endian (BE) or little-endian (LE). A big-endian system stores the most sig ...
, memory layout, or simply different ways of representing data structures in different
programming language
A programming language is a system of notation for writing computer programs. Most programming languages are text-based formal languages, but they may also be graphical. They are a kind of computer language.
The description of a programming ...
s.
Inherent to any serialization scheme is that, because the encoding of the data is by definition serial, extracting one part of the serialized data structure requires that the entire object be read from start to end, and reconstructed. In many applications, this linearity is an asset, because it enables simple, common I/O interfaces to be utilized to hold and pass on the state of an object. In applications where higher performance is an issue, it can make sense to expend more effort to deal with a more complex, non-linear storage organization.
Even on a single machine, primitive
pointer objects are too fragile to save because the objects to which they point may be reloaded to a different location in memory. To deal with this, the serialization process includes a step called ''
unswizzling
In computer science, pointer swizzling is the conversion of references based on name or position into direct pointer references (memory addresses). It is typically performed during deserialization or loading of a relocatable object from a disk ...
'' or ''pointer unswizzling'', where direct pointer references are converted to references based on name or position. The deserialization process includes an inverse step called ''
pointer swizzling
In computer science, pointer swizzling is the conversion of references based on name or position into direct pointer references (memory addresses). It is typically performed during deserialization or loading of a relocatable object from a disk ...
''.
Since both serializing and deserializing can be driven from common code (for example, the ''Serialize'' function in
Microsoft Foundation Classes
Microsoft Foundation Class Library (MFC) is a C++ object-oriented library for developing desktop applications for Windows.
MFC was introduced by Microsoft in 1992 and quickly gained widespread use. While Microsoft has introduced alternative app ...
), it is possible for the common code to do both at the same time, and thus, 1) detect differences between the objects being serialized and their prior copies, and 2) provide the input for the next such detection. It is not necessary to actually build the prior copy because differences can be detected on the fly, a technique called differential execution. This is useful in the programming of user interfaces whose contents are time-varying — graphical objects can be created, removed, altered, or made to handle input events without necessarily having to write separate code to do those things.
Drawbacks
Serialization breaks the opacity of an
abstract data type
In computer science, an abstract data type (ADT) is a mathematical model for data types. An abstract data type is defined by its behavior (Semantics (computer science), semantics) from the point of view of a ''User (computing), user'', of the dat ...
by potentially exposing private implementation details. Trivial implementations which serialize all data members may violate
encapsulation.
To discourage competitors from making compatible products, publishers of
proprietary software
Proprietary software is software that is deemed within the free and open-source software to be non-free because its creator, publisher, or other rightsholder or rightsholder partner exercises a legal monopoly afforded by modern copyright and int ...
often keep the details of their programs' serialization formats a
trade secret
Trade secrets are a type of intellectual property that includes formulas, practices, processes, designs, instruments, patterns, or compilations of information that have inherent economic value because they are not generally known or readily asc ...
. Some deliberately
obfuscate
Obfuscation is the obscuring of the intended meaning of communication by making the message difficult to understand, usually with confusing and ambiguous language. The obfuscation might be either unintentional or intentional (although intent us ...
or even
encrypt
In cryptography, encryption is the process of encoding information. This process converts the original representation of the information, known as plaintext, into an alternative form known as ciphertext. Ideally, only authorized parties can decip ...
the serialized data. Yet, interoperability requires that applications be able to understand each other's serialization formats. Therefore,
remote method call architectures such as
CORBA
The Common Object Request Broker Architecture (CORBA) is a standard defined by the Object Management Group (OMG) designed to facilitate the communication of systems that are deployed on diverse platforms. CORBA enables collaboration between sys ...
define their serialization formats in detail.
Many institutions, such as archives and libraries, attempt to
future proof
Future-proofing is the process of anticipating the future and developing methods of minimizing the effects of shocks and stresses of future events. Future-proofing is used in industries such as electronics, medical industry, industrial desig ...
their
backup
In information technology, a backup, or data backup is a copy of computer data taken and stored elsewhere so that it may be used to restore the original after a data loss event. The verb form, referring to the process of doing so, is "back up", w ...
archives—in particular,
database dump
A database dump contains a record of the table structure and/or the data from a database and is usually in the form of a list of SQL statements ("SQL dump"). A database dump is most often used for backing up a database so that its contents can be ...
s—by storing them in some relatively
human-readable
A human-readable medium or human-readable format is any encoding of data or information that can be naturally read by humans.
In computing, ''human-readable'' data is often encoded as ASCII or Unicode text, rather than as binary data. In most c ...
serialized format.
Serialization formats
The
Xerox Network Systems
Xerox Network Systems (XNS) is a computer networking protocol suite developed by Xerox within the Xerox Network Systems Architecture. It provided general purpose network communications, internetwork routing and packet delivery, and higher level f ...
Courier technology in the early 1980s influenced the first widely adopted standard.
Sun Microsystems
Sun Microsystems, Inc. (Sun for short) was an American technology company that sold computers, computer components, software, and information technology services and created the Java programming language, the Solaris operating system, ZFS, the ...
published the
External Data Representation
External Data Representation (XDR) is a standard data serialization format, for uses such as computer network protocols. It allows data to be transferred between different kinds of computer systems. Converting from the local representation to XD ...
(XDR) in 1987. XDR is an
open format
An open file format is a file format for storing digital data, defined by an openly published specification usually maintained by a standards organization, and which can be used and implemented by anyone. Open file format is licensed with open lic ...
, and standardized a
STD 67(RFC 4506).
In the late 1990s, a push to provide an alternative to the standard serialization protocols started:
XML
Extensible Markup Language (XML) is a markup language and file format for storing, transmitting, and reconstructing arbitrary data. It defines a set of rules for encoding documents in a format that is both human-readable and machine-readable. T ...
, an
SGML
The Standard Generalized Markup Language (SGML; ISO 8879:1986) is a standard for defining generalized markup languages for documents. ISO 8879 Annex A.1 states that generalized markup is "based on two postulates":
* Declarative: Markup should des ...
subset, was used to produce a human-readable
text-based encoding. Such an encoding can be useful for persistent objects that may be read and understood by humans, or communicated to other systems regardless of programming language. It has the disadvantage of losing the more compact, byte-stream-based encoding, but by this point larger storage and transmission capacities made file size less of a concern than in the early days of computing. In the 2000s, XML was often used for asynchronous transfer of structured data between client and server in
Ajax
Ajax may refer to:
Greek mythology and tragedy
* Ajax the Great, a Greek mythological hero, son of King Telamon and Periboea
* Ajax the Lesser, a Greek mythological hero, son of Oileus, the king of Locris
* ''Ajax'' (play), by the ancient Greek ...
web applications. XML is an open format, and standardized as
W3C recommendation
JSON
JSON (JavaScript Object Notation, pronounced ; also ) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other ser ...
is a lightweight plain-text alternative to XML, and is also commonly used for client-server communication in web applications. JSON is based on
JavaScript syntax
The syntax of JavaScript is the set of rules that define a correctly structured JavaScript program.
The examples below make use of the log function of the console object present in most browsers for standard text output.
The JavaScript standar ...
, but is independent of JavaScript and supported in many other programming languages. JSON is an open format, standardized a
STD 90()
ECMA-404 an
YAML
YAML ( and ) (''see '') is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted. YAML targets many of the same communications applications as Exte ...
is a strict superset of JSON and includes additional features such as a data type tags, support for cyclic data structures, indentation-sensitive syntax, and multiple forms of scalar data quoting. YAML is an open format.
Property list
In the macOS, iOS, NeXTSTEP, and GNUstep programming frameworks, property list files are files that store serialized objects. Property list files use the filename extension .plist, and thus are often referred to as p-list files.
Property list ...
s are used for serialization by
NeXTSTEP
NeXTSTEP is a discontinued object-oriented, multitasking operating system based on the Mach kernel and the UNIX-derived BSD. It was developed by NeXT Computer in the late 1980s and early 1990s and was initially used for its range of proprieta ...
,
GNUstep
GNUstep is a free software implementation of the Cocoa (formerly OpenStep) Objective-C frameworks, widget toolkit, and application development tools for Unix-like operating systems and Microsoft Windows. It is part of the GNU Project.
GNUstep ...
,
macOS
macOS (; previously OS X and originally Mac OS X) is a Unix operating system developed and marketed by Apple Inc. since 2001. It is the primary operating system for Apple's Mac computers. Within the market of desktop and lapt ...
, and
iOS
iOS (formerly iPhone OS) is a mobile operating system created and developed by Apple Inc. exclusively for its hardware. It is the operating system that powers many of the company's mobile devices, including the iPhone; the term also includes ...
frameworks
A framework is a generic term commonly referring to an essential supporting structure which other things are built on top of.
Framework may refer to:
Computing
* Application framework, used to implement the structure of an application for an op ...
. ''Property list'', or ''p-list'' for short, doesn't refer to a single serialization format but instead several different variants, some human-readable and one binary.
For large volume scientific datasets, such as satellite data and output of numerical climate, weather, or ocean models, specific binary serialization standards have been developed, e.g.
HDF,
netCDF
NetCDF (Network Common Data Form) is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the Unidata ...
and the older
GRIB
GRIB (GRIdded Binary or General Regularly-distributed Information in Binary form) is a concise data format commonly used in meteorology to store historical and forecast weather data. It is standardized by the World Meteorological Organization's C ...
.
Programming language support
Several
object-oriented programming
Object-oriented programming (OOP) is a programming paradigm based on the concept of "objects", which can contain data and code. The data is in the form of fields (often known as attributes or ''properties''), and the code is in the form of pr ...
languages directly support ''object serialization'' (or ''object archival''), either by
syntactic sugar
In computer science, syntactic sugar is syntax within a programming language that is designed to make things easier to read or to express. It makes the language "sweeter" for human use: things can be expressed more clearly, more concisely, or in an ...
elements or providing a standard
interface
Interface or interfacing may refer to:
Academic journals
* ''Interface'' (journal), by the Electrochemical Society
* ''Interface, Journal of Applied Linguistics'', now merged with ''ITL International Journal of Applied Linguistics''
* '' Inte ...
for doing so. The languages which do so include
Ruby
A ruby is a pinkish red to blood-red colored gemstone, a variety of the mineral corundum ( aluminium oxide). Ruby is one of the most popular traditional jewelry gems and is very durable. Other varieties of gem-quality corundum are called sa ...
,
Smalltalk
Smalltalk is an object-oriented, dynamically typed reflective programming language. It was designed and created in part for educational use, specifically for constructionist learning, at the Learning Research Group (LRG) of Xerox PARC by Alan Ka ...
,
Python
Python may refer to:
Snakes
* Pythonidae, a family of nonvenomous snakes found in Africa, Asia, and Australia
** ''Python'' (genus), a genus of Pythonidae found in Africa and Asia
* Python (mythology), a mythical serpent
Computing
* Python (pro ...
,
PHP
PHP is a general-purpose scripting language geared toward web development. It was originally created by Danish-Canadian programmer Rasmus Lerdorf in 1993 and released in 1995. The PHP reference implementation is now produced by The PHP Group ...
,
Objective-C
Objective-C is a general-purpose, object-oriented programming language that adds Smalltalk-style messaging to the C programming language. Originally developed by Brad Cox and Tom Love in the early 1980s, it was selected by NeXT for its NeXTS ...
,
Delphi
Delphi (; ), in legend previously called Pytho (Πυθώ), in ancient times was a sacred precinct that served as the seat of Pythia, the major oracle who was consulted about important decisions throughout the ancient classical world. The oracle ...
,
Java
Java (; id, Jawa, ; jv, ꦗꦮ; su, ) is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea to the north. With a population of 151.6 million people, Java is the world's List ...
, and the
.NET family of languages. There are also libraries available that add serialization support to languages that lack native support for it.
;C and C++:
C and
C++
C++ (pronounced "C plus plus") is a high-level general-purpose programming language created by Danish computer scientist Bjarne Stroustrup as an extension of the C programming language, or "C with Classes". The language has expanded significan ...
do not provide serialization as any sort of high-level construct, but both languages support writing any of the built-in
data types
In computer science and computer programming, a data type (or simply type) is a set of possible values and a set of allowed operations on it. A data type tells the compiler or interpreter how the programmer intends to use the data. Most progra ...
, as well as
plain old data
In computer science and object-oriented programming, a passive data structure (PDS, also termed a plain old data structure, or plain old data, POD) is a term for a record, to contrast with objects. It is a data structure that is represented only ...
structs, as binary data. As such, it is usually trivial to write custom serialization functions. Moreover, compiler-based solutions, such as the
ODB ODB may refer to:
People
* Ol' Dirty Bastard (1968–2004), American rapper and founding member of the Wu-Tang Clan
* ODB (wrestler) (born 1978), Stage name of American professional wrestler Jessica Kresa
* Original David Baker (born 1972), a monik ...
ORM
Orm (in Old Norse and in modern Danish, Swedish, Norwegian (bokmål and nynorsk) the word for "snake", "worm" or "dragon") became an Anglo-Saxon personal name during period of the Danelaw.
Orm may also refer to:
* Orm or Ormin, the author of t ...
system for C++ and the
gSOAP
gSOAP is a C and C++ software development toolkit for SOAP/XML web services and generic XML data bindings. Given a set of C/C++ type declarations, the compiler-based gSOAP tools generate serialization routines in source code for efficient XML s ...
toolkit for C and C++, are capable of automatically producing serialization code with few or no modifications to class declarations. Other popular serialization frameworks are Boost.Serialization from the
Boost Framework, the S11n framework, and Cereal.
MFC framework (Microsoft) also provides serialization methodology as part of its Document-View architecture.
;CFML:
CFML
ColdFusion Markup Language, more commonly known as CFML, is a server-side scripting, scripting language for web development that runs on the JVM, the .NET Framework, .NET framework, and Google App Engine. Multiple commercial and open-source soft ...
allows data structures to be serialized to
WDDX
WDDX (Web Distributed Data eXchange) is a programming language-, platform- and transport-neutral data interchange mechanism designed to pass data between different environments and different computers.
History
WDDX was created by Simeon Simeonov ...
with the
/code> tag and to JSON
JSON (JavaScript Object Notation, pronounced ; also ) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other ser ...
with th
SerializeJSON()
function.
;Delphi: Delphi
Delphi (; ), in legend previously called Pytho (Πυθώ), in ancient times was a sacred precinct that served as the seat of Pythia, the major oracle who was consulted about important decisions throughout the ancient classical world. The oracle ...
provides a built-in mechanism for serialization of components (also called persistent objects), which is fully integrated with its IDE. The component's contents are saved to a DFM file and reloaded on-the-fly.
;Go: Go natively supports unmarshalling/marshalling of JSON
JSON (JavaScript Object Notation, pronounced ; also ) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other ser ...
and XML
Extensible Markup Language (XML) is a markup language and file format for storing, transmitting, and reconstructing arbitrary data. It defines a set of rules for encoding documents in a format that is both human-readable and machine-readable. T ...
data. There are also third-party modules that support YAML
YAML ( and ) (''see '') is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted. YAML targets many of the same communications applications as Exte ...
and Protocol Buffers
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data. It is useful in developing programs to communicate with each other over a network or for storing data. The method involves an int ...
. Go also supports ''Gobs''.
;Haskell: In Haskell, serialization is supported for types that are members of the Read and Show type class
In computer science, a type class is a type system construct that supports ad hoc polymorphism. This is achieved by adding constraints to type variables in parametrically polymorphic types. Such a constraint typically involves a type class T and ...
es. Every type that is a member of the Read
type class defines a function that will extract the data from the string representation of the dumped data. The Show
type class, in turn, contains the show
function from which a string representation of the object can be generated. The programmer need not define the functions explicitly—merely declaring a type to be deriving Read or deriving Show, or both, can make the compiler generate the appropriate functions for many cases (but not all: function types, for example, cannot automatically derive Show or Read). The auto-generated instance for Show also produces valid source code, so the same Haskell value can be generated by running the code produced by show in, for example, a Haskell interpreter. For more efficient serialization, there are haskell libraries that allow high-speed serialization in binary format, e.g
binary
;Java: Java provides automatic serialization which requires that the object be marked
In linguistics and social sciences, markedness is the state of standing out as nontypical or divergent as opposed to regular or common. In a marked–unmarked relation, one term of an opposition is the broader, dominant one. The dominant defau ...
by implementing the interface
Interface or interfacing may refer to:
Academic journals
* ''Interface'' (journal), by the Electrochemical Society
* ''Interface, Journal of Applied Linguistics'', now merged with ''ITL International Journal of Applied Linguistics''
* '' Inte ...
. Implementing the interface marks the class as "okay to serialize", and Java then handles serialization internally. There are no serialization methods defined on the Serializable
interface, but a serializable class can optionally define methods with certain special names and signatures that if defined, will be called as part of the serialization/deserialization process. The language also allows the developer to override the serialization process more thoroughly by implementing another interface, the interface, which includes two special methods that are used to save and restore the object's state.
There are three primary reasons why objects are not serializable by default and must implement the Serializable
interface to access Java's serialization mechanism.
Firstly, not all objects capture useful semantics in a serialized state. For example, a object is tied to the state of the current JVM
A Java virtual machine (JVM) is a virtual machine that enables a computer to run Java programs as well as programs written in other languages that are also compiled to Java bytecode. The JVM is detailed by a specification that formally describes ...
. There is no context in which a deserialized Thread
object would maintain useful semantics.
Secondly, the serialized state of an object forms part of its classes' compatibility contract. Maintaining compatibility between versions of serializable classes requires additional effort and consideration. Therefore, making a class serializable needs to be a deliberate design decision and not a default condition.
Lastly, serialization allows access to non-transient
ECHELON, originally a secret government code name, is a surveillance program ( signals intelligence/SIGINT collection and analysis network) operated by the five signatory states to the UKUSA Security Agreement:Given the 5 dialects that ...
private members of a class that are not otherwise accessible. Classes containing sensitive information (for example, a password) should not be serializable nor externalizable. The standard encoding method uses a recursive graph-based translation of the object's class descriptor and serializable fields into a byte stream. Primitives as well as non-transient, non-static referenced objects are encoded into the stream. Each object that is referenced by the serialized object via a field that is not marked as transient
must also be serialized; and if any object in the complete graph of non-transient object references is not serializable, then serialization will fail. The developer can influence this behavior by marking objects as transient, or by redefining the serialization for an object so that some portion of the reference graph is truncated and not serialized.
Java does not use constructor to serialize objects. It is possible to serialize Java objects through JDBC
Java Database Connectivity (JDBC) is an application programming interface (API) for the programming language Java, which defines how a client may access a database. It is a Java-based data access technology used for Java database connectivity. I ...
and store them into a database. While Swing components do implement the Serializable interface, they are not guaranteed to be portable between different versions of the Java Virtual Machine. As such, a Swing component, or any component which inherits it, may be serialized to a byte stream, but it is not guaranteed that this will be re-constitutable on another machine.
;JavaScript: Since ECMAScript 5.1, JavaScript
JavaScript (), often abbreviated as JS, is a programming language that is one of the core technologies of the World Wide Web, alongside HTML and CSS. As of 2022, 98% of Website, websites use JavaScript on the Client (computing), client side ...
has included the built-in JSON
object and its methods JSON.parse()
and JSON.stringify()
. Although JSON is originally based on a subset of JavaScript, there are boundary cases where JSON is not valid JavaScript. Specifically, JSON allows the Unicode line terminators and to appear unescaped in quoted strings, while ECMAScript 2018 and older does not. See the main article on JSON.
;Julia: Julia
Julia is usually a feminine given name. It is a Latinate feminine form of the name Julio and Julius. (For further details on etymology, see the Wiktionary entry "Julius".) The given name ''Julia'' had been in use throughout Late Antiquity (e.g. ...
implements serialization through the serialize()
/ deserialize()
modules, intended to work within the same version of Julia, and/or instance of the same system image. The HDF5.jl
package offers a more stable alternative, using a documented format and common library with wrappers for different languages, while the default serialization format is suggested to have been designed rather with maximal performance for network communication in mind.
;Lisp: Generally a Lisp
A lisp is a speech impairment in which a person misarticulates sibilants (, , , , , , , ). These misarticulations often result in unclear speech.
Types
* A frontal lisp occurs when the tongue is placed anterior to the target. Interdental lisping ...
data structure can be serialized with the functions "read
" and "print
". A variable foo containing, for example, a list of arrays would be printed by (print foo)
. Similarly an object can be read from a stream named s by (read s)
. These two parts of the Lisp implementation are called the Printer and the Reader. The output of "print
" is human readable; it uses lists demarked by parentheses, for example: . In many types of Lisp, including Common Lisp
Common Lisp (CL) is a dialect of the Lisp programming language, published in ANSI standard document ''ANSI INCITS 226-1994 (S20018)'' (formerly ''X3.226-1994 (R1999)''). The Common Lisp HyperSpec, a hyperlinked HTML version, has been derived fro ...
, the printer cannot represent every type of data because it is not clear how to do so. In Common Lisp for example the printer cannot print CLOS objects. Instead the programmer may write a method on the generic function print-object
, this will be invoked when the object is printed. This is somewhat similar to the method used in Ruby. Lisp code itself is written in the syntax of the reader, called read syntax. Most languages use separate and different parsers to deal with code and data, Lisp only uses one. A file containing lisp code may be read into memory as a data structure, transformed by another program, then possibly executed or written out, such as in a read–eval–print loop
A read–eval–print loop (REPL), also termed an interactive toplevel or language shell, is a simple interactive computer programming environment that takes single user inputs, executes them, and returns the result to the user; a program written ...
. Not all readers/writers support cyclic, recursive or shared structures.
;.NET Framework: .NET Framework
The .NET Framework (pronounced as "''dot net"'') is a proprietary software framework developed by Microsoft that runs primarily on Microsoft Windows. It was the predominant implementation of the Common Language Infrastructure (CLI) until bein ...
has several serializers designed by Microsoft
Microsoft Corporation is an American multinational technology corporation producing computer software, consumer electronics, personal computers, and related services headquartered at the Microsoft Redmond campus located in Redmond, Washing ...
. There are also many serializers by third parties. More than a dozen serializers are discussed and teste
here
an
here
The list is constantly growing.
;OCaml: OCaml
OCaml ( , formerly Objective Caml) is a general-purpose programming language, general-purpose, multi-paradigm programming language which extends the Caml dialect of ML (programming language), ML with object-oriented programming, object-oriented ...
's standard library provides marshalling through the Marshal
module[ and the Pervasives functions ]output_value
and input_value
. While OCaml programming is statically type-checked, uses of the Marshal
module may break type guarantees, as there is no way to check whether an unmarshalled stream represents objects of the expected type. In OCaml it is difficult to marshal a function or a data structure which contains a function (e.g. an object which contains a method), because executable code in functions cannot be transmitted across different programs. (There is a flag to marshal the code position of a function but it can only be unmarshalled in exactly the same program). The standard marshalling functions can preserve sharing and handle cyclic data, which can be configured by a flag.
;Perl: Several Perl
Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offici ...
modules available from CPAN
The Comprehensive Perl Archive Network (CPAN) is a repository of over 250,000 software modules and accompanying documentation for 39,000 distributions, written in the Perl programming language by over 12,000 contributors. ''CPAN'' can denote eith ...
provide serialization mechanisms, including Storable
, JSON::XS
and FreezeThaw
. Storable includes functions to serialize and deserialize Perl data structures to and from files or Perl scalars. In addition to serializing directly to files, Storable
includes the freeze
function to return a serialized copy of the data packed into a scalar, and thaw
to deserialize such a scalar. This is useful for sending a complex data structure over a network socket or storing it in a database. When serializing structures with Storable
, there are network safe functions that always store their data in a format that is readable on any computer at a small cost of speed. These functions are named nstore
, nfreeze
, etc. There are no "n" functions for deserializing these structures — the regular thaw
and retrieve
deserialize structures serialized with the "n
" functions and their machine-specific equivalents.
;PHP: PHP
PHP is a general-purpose scripting language geared toward web development. It was originally created by Danish-Canadian programmer Rasmus Lerdorf in 1993 and released in 1995. The PHP reference implementation is now produced by The PHP Group ...
originally implemented serialization through the built-in serialize()
and unserialize()
functions. PHP can serialize any of its data types except resources (file pointers, sockets, etc.). The built-in unserialize()
function is often dangerous when used on completely untrusted data. For objects, there are two "magic
Magic or Magick most commonly refers to:
* Magic (supernatural), beliefs and actions employed to influence supernatural beings and forces
* Ceremonial magic, encompasses a wide variety of rituals of magic
* Magical thinking, the belief that unrela ...
methods" that can be implemented within a class — __sleep()
and __wakeup()
— that are called from within serialize()
and unserialize()
, respectively, that can clean up and restore an object. For example, it may be desirable to close a database connection on serialization and restore the connection on deserialization; this functionality would be handled in these two magic methods. They also permit the object to pick which properties are serialized. Since PHP 5.1, there is an object-oriented serialization mechanism for objects, the Serializable
interface.
;Prolog: Prolog
Prolog is a logic programming language associated with artificial intelligence and computational linguistics.
Prolog has its roots in first-order logic, a formal logic, and unlike many other programming languages, Prolog is intended primarily ...
's ''term'' structure, which is the only data structure of the language, can be serialized out through the built-in predicate write_term/3
and serialized-in through the built-in predicates read/1
and read_term/2
. The resulting stream is uncompressed text (in some encoding determined by configuration of the target stream), with any free variables in the term represented by placeholder variable names. The predicate write_term/3
is standardized in the ISO Specification for Prolog (ISO/IEC 13211-1) on pages 59 ff. ("Writing a term, § 7.10.5"). Therefore it is expected that terms serialized-out by one implementation can be serialized-in by another without ambiguity or surprises. In practice, implementation-specific extensions (e.g. SWI-Prolog's dictionaries) may use non-standard term structures, so interoperability may break in edge cases. As examples, see the corresponding manual pages for SWI-Prolog, SICStus Prolog, GNU Prolog. Whether and how serialized terms received over the network are checked against a specification (after deserialization from the character stream has happened) is left to the implementer. Prolog's built-in Definite Clause Grammars can be applied at that stage.
;Python: The core general serialization mechanism is the pickle
standard library
In computer programming, a standard library is the library made available across implementations of a programming language. These libraries are conventionally described in programming language specifications; however, contents of a language's as ...
module, alluding to the database systems term ''pickling'' to describe data serialization (''unpickling'' for ''deserializing''). Pickle uses a simple stack-based virtual machine
In computing, a virtual machine (VM) is the virtualization/emulation of a computer system. Virtual machines are based on computer architectures and provide functionality of a physical computer. Their implementations may involve specialized hardw ...
that records the instructions used to reconstruct the object. It is a cross-versio
customisable
but unsafe (not secure against erroneous or malicious data) serialization format. Malformed or maliciously constructed data, may cause the deserializer to import arbitrary modules and instantiate any object. The standard library also includes modules serializing to standard data formats:
/code> (with built-in support for basic scalar and collection types and able to support arbitrary types vi
.
/code> (with support for both binary and XML property list
In the macOS, iOS, NeXTSTEP, and GNUstep programming frameworks, property list files are files that store serialized objects. Property list files use the filename extension .plist, and thus are often referred to as p-list files.
Property list ...
formats). xdrlib
/code> (with support for the External Data Representation (XDR) standard as described in RFC 1014). Finally, it is recommended that an object's
/code> be evaluable in the right environment, making it a rough match for Common Lisp's
/code>. Not all object types can be pickled automatically, especially ones that hold operating system
An operating system (OS) is system software that manages computer hardware, software resources, and provides common services for computer programs.
Time-sharing operating systems schedule tasks for efficient use of the system and may also in ...
resources like file handle
In Unix and Unix-like computer operating systems, a file descriptor (FD, less frequently fildes) is a process-unique identifier (handle) for a file or other input/output resource, such as a pipe or network socket.
File descriptors typically have ...
s, but users can register custom "reduction" and construction functions to support the pickling and unpickling of arbitrary types. Pickle was originally implemented as the pure Python pickle
module, but, in versions of Python prior to 3.0, the cPickle
module (also a built-in) offers improved performance (up to 1000 times faster). The cPickle
was adapted from the Unladen Swallow
CPython is the reference implementation of the Python programming language. Written in C and Python, CPython is the default and most widely used implementation of the Python language.
CPython can be defined as both an interpreter and a compi ...
project. In Python 3, users should always import the standard version, which attempts to import the accelerated version and falls back to the pure Python version.
;R: R has the function dput
which writes an ASCII text representation of an R object to a file or connection. A representation can be read from a file using dget
. More specific, the function serialize
serializes an R object to a connection, the output being a raw vector coded in hexadecimal format. The unserialize
function allows to read an object from a connection or a raw vector.
;REBOL: REBOL will serialize to file (save/all
) or to a string!
(mold/all
). Strings and files can be deserialized using the polymorphic load
function. RProtoBuf
provides cross-language data serialization in R, using Protocol Buffers
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data. It is useful in developing programs to communicate with each other over a network or for storing data. The method involves an int ...
.
;Ruby: Ruby
A ruby is a pinkish red to blood-red colored gemstone, a variety of the mineral corundum ( aluminium oxide). Ruby is one of the most popular traditional jewelry gems and is very durable. Other varieties of gem-quality corundum are called sa ...
includes the standard module Marshal
/code> with 2 methods dump
and load
, akin to the standard Unix utilities dump
and restore
. These methods serialize to the standard class String
, that is, they effectively become a sequence of bytes. Some objects cannot be serialized (doing so would raise a TypeError
exception): bindings, procedure objects, instances of class IO, singleton objects and interfaces. If a class requires custom serialization (for example, it requires certain cleanup actions done on dumping / restoring), it can be done by implementing 2 methods: _dump
and _load
. The instance method _dump
should return a String
object containing all the information necessary to reconstitute objects of this class and all referenced objects up to a maximum depth given as an integer parameter (a value of -1 implies that depth checking should be disabled). The class method _load
should take a String
and return an object of this class.
;Rust: Serde
/code> is the most widely used library, or crate, for serialization in Rust
Rust is an iron oxide, a usually reddish-brown oxide formed by the reaction of iron and oxygen in the catalytic presence of water or air moisture. Rust consists of hydrous iron(III) oxides (Fe2O3·nH2O) and iron(III) oxide-hydroxide (FeO(OH ...
.
;Smalltalk: In general, non-recursive and non-sharing objects can be stored and retrieved in a human readable form using the storeOn:
/readFrom:
protocol. The storeOn:
method generates the text of a Smalltalk expression which - when evaluated using readFrom:
- recreates the original object. This scheme is special, in that it uses a procedural description of the object, not the data itself. It is therefore very flexible, allowing for classes to define more compact representations. However, in its original form, it does not handle cyclic data structures or preserve the identity of shared references (i.e. two references a single object will be restored as references to two equal, but not identical copies). For this, various portable and non-portable alternatives exist. Some of them are specific to a particular Smalltalk implementation or class library. There are several ways in Squeak Smalltalk to serialize and store objects. The easiest and most used are storeOn:/readFrom:
and binary storage formats based on SmartRefStream
serializers. In addition, bundled objects can be stored and retrieved using ImageSegments
. Both provide a so-called "binary-object storage framework", which support serialization into and retrieval from a compact binary form. Both handle cyclic, recursive and shared structures, storage/retrieval of class and metaclass info and include mechanisms for "on the fly" object migration (i.e. to convert instances which were written by an older version of a class with a different object layout). The APIs are similar (storeBinary/readBinary), but the encoding details are different, making these two formats incompatible. However, the Smalltalk/X code is open source and free and can be loaded into other Smalltalks to allow for cross-dialect object interchange. Object serialization is not part of the ANSI Smalltalk specification. As a result, the code to serialize an object varies by Smalltalk implementation. The resulting binary data also varies. For instance, a serialized object created in Squeak Smalltalk cannot be restored in Ambrai Smalltalk. Consequently, various applications that do work on multiple Smalltalk implementations that rely on object serialization cannot share data between these different implementations. These applications include the MinneStore object database and some RPC packages. A solution to this problem is SIXX, which is a package for multiple Smalltalks that uses an XML
Extensible Markup Language (XML) is a markup language and file format for storing, transmitting, and reconstructing arbitrary data. It defines a set of rules for encoding documents in a format that is both human-readable and machine-readable. T ...
-based format for serialization.
;Swift: The Swift
Swift or SWIFT most commonly refers to:
* SWIFT, an international organization facilitating transactions between banks
** SWIFT code
* Swift (programming language)
* Swift (bird), a family of birds
It may also refer to:
Organizations
* SWIFT, ...
standard library provides two protocols, Encodable
and Decodable
(composed together as Codable
), which allow instances of conforming types to be serialized to or deserialized from JSON
JSON (JavaScript Object Notation, pronounced ; also ) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other ser ...
, property list
In the macOS, iOS, NeXTSTEP, and GNUstep programming frameworks, property list files are files that store serialized objects. Property list files use the filename extension .plist, and thus are often referred to as p-list files.
Property list ...
s, or other formats. Default implementations of these protocols can be generated by the compiler for types whose stored properties are also Decodable
or Encodable
.
;Windows PowerShell: Windows PowerShell
PowerShell is a task automation and configuration management program from Microsoft, consisting of a command-line shell and the associated scripting language. Initially a Windows component only, known as Windows PowerShell, it was made open-sourc ...
implements serialization through the built-in
Built-in, builtin, or built in may refer to:
Computing
* Shell builtin, a command or a function executed directly in the shell itself
* Builtin function, in computer software and compiler theory
Other uses
* Built-in behavior, of a living organis ...
cmdlet Export-CliXML
. Export-CliXML
serializes .NET objects and stores the resulting XML in a file. To reconstitute the objects, use the Import-CliXML
cmdlet, which generates a deserialized object from the XML in the exported file. Deserialized objects, often known as "property bags" are not live objects; they are snapshots that have properties, but no methods. Two dimensional data structures can also be (de)serialized in CSV format using the built-in cmdlets Import-CSV
and Export-CSV
.
See also
* Commutation (telemetry)
In telemetry, commutation is a process whereby multiple data streams ("measurands"), possibly with differing data rates, are combined into a single Frame (networking), frame-based stream for transmission (telecommunications), transmission, before ...
* Comparison of data serialization formats
This is a comparison of data serialization formats, various ways to convert complex objects to sequences of bits. It does not include markup languages used exclusively as document file format
A document file format is a text or binary file form ...
* Container format
A container format (informally, sometimes called a wrapper) or metafile is a file format that allows multiple data streams to be embedded into a single file, usually along with metadata for identifying and further detailing those streams. Notab ...
* Hibernate (Java)
Hibernate ORM (or simply Hibernate) is an object–relational mapping tool for the Java programming language. It provides a framework for mapping an object-oriented domain model to a relational database. Hibernate handles object–relational impe ...
* XML Schema
An XML schema is a description of a type of Extensible Markup Language, XML document, typically expressed in terms of constraints on the structure and content of documents of that type, above and beyond the basic syntactical constraints imposed ...
* Basic Encoding Rules X.690 is an ITU-T standard specifying several ASN.1 encoding formats:
* Basic Encoding Rules (BER)
* Canonical Encoding Rules (CER)
* Distinguished Encoding Rules (DER)
The Basic Encoding Rules (BER) were the original rules laid out by the ASN.1 s ...
* Google Protocol Buffers
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data. It is useful in developing programs to communicate with each other over a network or for storing data. The method involves an i ...
* Wikibase
Wikibase is a set of MediaWiki extensions for working with versioned semi-structured data in a central repository based upon JSON instead of the unstructured data of MediaWiki wikitext. Its primary components are the ''Wikibase Repository'', an e ...
* Apache Avro
Avro is a row-oriented remote procedure call and data serialization framework developed within Apache's Hadoop project. It uses JSON for defining data types and protocols, and serializes data in a compact binary format. Its primary use is in Ap ...
References
External links
*
Java 1.4 Object Serialization documentation
Databoard
- Binary serialization with partial and random access, type system, RPC, type adaption, and text format
{{Authority control
*
Persistence
Articles with example code