External metadata
A final way of storing the format of a file is to explicitly store information about the format in the file system, rather than within the file itself. This approach keeps the metadata separate from both the main data and the name, but is also lessMac OS type-codes
TheMac OS X uniform type identifiers (UTIs)
A Uniform Type Identifier (UTI) is a method used inOS/2 extended attributes
The HPFS, FAT12, and FAT16 (but not FAT32) filesystems allow the storage of "extended attributes" with files. These comprise an arbitrary set of triplets with a name, a coded type for the value, and a value, where the names are unique and values can be up to 64 KB long. There are standardized meanings for certain types and names (underPOSIX extended attributes
On Unix andPRONOM unique identifiers (PUIDs)
The PRONOM Persistent Unique Identifier (PUID) is an extensible scheme of persistent, unique, and unambiguous identifiers for file formats, which has been developed by The National Archives of the UK as part of its PRONOM technical registry service. PUIDs can be expressed as Uniform Resource Identifiers using the namespace. Although not yet widely used outside of the UK government and some digital preservation programs, the PUID scheme does provide greater granularity than most alternative schemes.MIME types
MIME types are widely used in many Internet-related applications, and increasingly elsewhere, although their usage for on-disc type information is rare. These consist of a standardised system of identifiers (managed by Internet Assigned Numbers Authority, IANA) consisting of a ''type'' and a ''sub-type'', separated by a slash (punctuation), slash—for instance, or . These were originally intended as a way of identifying what type of file was attached to an e-mail, independent of the source and target operating systems. MIME types identify files on BeOS, AmigaOS 4.0 and MorphOS, as well as store unique application signatures for application launching. In AmigaOS and MorphOS, the Mime type system works in parallel with Amiga specific Datatype system. There are problems with the MIME types though; several organizations and people have created their own MIME types without registering them properly with IANA, which makes the use of this standard awkward in some cases.File format identifiers (FFIDs)
File format identifiers are another, not widely used way to identify file formats according to their origin and their file category. It was created for the Description Explorer suite of software. It is composed of several digits of the form . The first part indicates the organization origin/maintainer (this number represents a value in a company/standards organization database), and the 2 following digits categorize the type of file inFile content based format identification
Another but less popular way to identify the file format is to examine the file contents for distinguishable patterns among file types. The contents of a file are a sequence of bytes and a byte has 256 unique permutations (0–255). Thus, counting the occurrence of byte patterns that is often referred to as byte frequency distribution gives distinguishable patterns to identify file types. There are many content-based file type identification schemes that use a byte frequency distribution to build the representative models for file type and use any statistical and data mining techniques to identify file typesFile structure
There are several types of ways to structure data in a file. The most usual ones are described below.Unstructured formats (raw memory dumps)
Earlier file formats used raw data formats that consisted of directly dumping the memory images of one or more structures into the file. This has several drawbacks. Unless the memory images also have reserved spaces for future extensions, extending and improving this type of structured file is very difficult. It also creates files that might be specific to one platform or programming language (for example a structure containing a Pascal (programming language), Pascal string is not recognized as such in C (programming language), C). On the other hand, developing tools for reading and writing these types of files is very simple. The limitations of the unstructured formats led to the development of other types of file formats that could be easily extended and be backward compatible at the same time.Chunk-based formats
In this kind of file structure, each piece of data is embedded in a container that somehow identifies the data. The container's scope can be identified by start- and end-markers of some kind, by an explicit length field somewhere, or by fixed requirements of the file format's definition. Throughout the 1970s, many programs used formats of this general kind. For example, word-processors such as troff, SCRIPT (markup), Script, and Scribe (markup language), Scribe, and database export files such as comma-separated values, CSV. Electronic Arts and Commodore International, Commodore-Amiga also used this type of file format in 1985, with their IFF (Interchange File Format) file format. A container is sometimes called a ''"chunk"'', although "chunk" may also imply that each piece is small, and/or that chunks do not contain other chunks; many formats do not impose those requirements. The information that identifies a particular "chunk" may be called many different things, often terms including "field name", "identifier", "label", or "tag". The identifiers are often human-readable, and classify parts of the data: for example, as a "surname", "address", "rectangle", "font name", etc. These are not the same thing as identifiers in the sense of a database key or serial number (although an identifier may well identify its associated data as such a key). With this type of file structure, tools that do not know certain chunk identifiers simply skip those that they do not understand. Depending on the actual meaning of the skipped data, this may or may not be useful (CSS explicitly defines such behavior). This concept has been used again and again by RIFF (File format), RIFF (Microsoft-IBM equivalent of IFF), PNG, JPEG storage, DER (Distinguished Encoding Rules) encoded streams and files (which were originally described in CCITT X.409:1984 and therefore predate IFF), and SDXF, Structured Data Exchange Format (SDXF). Indeed, any data format must somehow identify the significance of its component parts, and embedded boundary-markers are an obvious way to do so: * MIME#MIME headers, MIME headers do this with a colon-separated label at the start of each logical line. MIME headers cannot contain other MIME headers, though the data content of some headers has sub-parts that can be extracted by other conventions. * comma-separated values, CSV and similar files often do this using a header records with field names, and with commas to mark the field boundaries. Like MIME, CSV has no provision for structures with more than one level. *Directory-based formats
This is another extensible format, that closely resembles a file system (Object Linking and Embedding, OLE Documents are actual filesystems), where the file is composed of 'directory entries' that contain the location of the data within the file itself as well as its signatures (and in certain cases its type). Good examples of these types of file structures are disk images, executables, OLE documents TIFF, library (computing), libraries. Some file formats like ODT and DOCX, being PKZIP-based, are both chunked and carry a directory. The structure of a directory-based file format lends itself to modifications more easily than unstructured or chunk-based formats. The nature of this type of format allows users to carefully construct files that causes reader software to do things the authors of the format never intended to happen. An example of this is the zip bomb. Directory-based file formats also use values that point at other areas in the file but if some later data value points back at data that was read earlier, it can result in an infinite loop for any reader software that assumes the input file is valid and blindly follows the loop.See also
* Audio file format * Chemical file format * Comparison of executable file formats * Digital container format * Document file format * PRONOM technical registry#DROID, DROID file format identification utility * File (command), a file type identification utility * File conversion * Future proofing * Graphics file format summary * Image file formats * List of archive formats * List of file formats * List of file signatures, or "magic numbers" * List of filename extensions (alphabetical) * List of free file formats * List of motion and gesture file formats *References
:* :* :*External links
* * ("The file formats you use have a direct impact on your ability to open those files at a later date and on the ability of other people to access those data") {{DEFAULTSORT:File format Computer file formats,