High Sierra File System

ISO 9660 (also known as ECMA-119) is a file system for optical disc media. Being sold by the International Organization for Standardization (ISO) the file system is considered an international technical standard. Since the specification is available for anybody to purchase, implementations have been written for many operating systems.

ISO 9660 traces its roots to the ''High Sierra Format'', which arranged file information in a dense, sequential layout to minimize nonsequential access by using a hierarchical (eight levels of directories deep) tree file system arrangement, similar to UNIX and FAT. To facilitate cross platform compatibility, it defined a minimal set of common file attributes (directory or ordinary file and time of recording) and name attributes (name, extension, and version), and used a separate system use area where future optional extensions for each file may be specified. High Sierra was adopted in December 1986 (with changes) as an international standard by Ecma International as ECMA-119 and submitted for fast tracking to the ISO, where it was eventually accepted as ISO 9660:1988. Subsequent amendments to the standard were published in 2013 and 2020.

The first 16 sectors of the file system are empty and reserved for other uses. The rest begins with a ''volume descriptor set'' (a header block which describes the subsequent layout) and then the path tables, directories and files on the disc. An ISO 9660 compliant disc must contain at least one ''primary volume descriptor'' describing the file system and a ''volume descriptor set terminator'' which is a volume descriptor that marks the end of the descriptor set. The primary volume descriptor provides information about the volume, characteristics and metadata, including a root directory record that indicates in which sector the root directory is located. Other fields contain metadata such as the volume's name and creator, along with the size and number of logical blocks used by the file system. Path tables summarize the directory structure of the relevant directory hierarchy. For each directory in the image, the path table provides the directory identifier, the location of the extent in which the directory is recorded, the length of any extended attributes associated with the directory, and the index of its parent directory path table entry.

There are several extensions to ISO 9660 that relax some of its limitations. Notable examples include ''Rock Ridge'' (Unix-style permissions and longer names), ''Joliet'' ( Unicode, allowing non- Latin scripts to be used), ''El Torito'' (enables CDs to be bootable) and the ''Apple ISO 9660 Extensions'' (file characteristics specific to the classic Mac OS and macOS, such as resource forks, file backup date and more).

History

Compact discs were originally developed for recording musical data, but soon were used for storing additional digital data types because they were equally effective for archival mass data storage. Called CD-ROMs, the lowest level format for these type of compact discs was defined in the '' Yellow Book'' specification in 1983. However, this book did not define any format for organizing data on CD-ROMs into logical units such as files, which led to every CD-ROM maker creating their own format. In order to develop a CD-ROM file system standard (''Z39.60'' - ''Volume and File Structure of CDROM for Information Interchange''), the National Information Standards Organization (NISO) set up Standards Committee SC EE (Compact Disc Data Format) in July 1985. In September/ October 1985 several companies invited experts to participate in the development of a working paper for such a standard.

In November 1985, representatives of computer hardware manufacturers gathered at the High Sierra Hotel and Casino (currently called the Hard Rock Hotel and Casino) near Lake Tahoe, California. This group became known as the ''High Sierra Group'' (''HSG''). Present at the meeting were representatives from Apple Computer, AT&T, Digital Equipment Corporation (DEC), Hitachi, LaserData, Microware, Microsoft, 3M, Philips, Reference Technology Inc., Sony Corporation, TMS Inc., VideoTools (later Meridian), Xebec, and Yelick. The meeting report evolved from the ''Yellow Book'' CD-ROM standard, which was so open ended it was leading to diversification and creation of many incompatible data storage methods. The ''High Sierra Group Proposal'' (''HSGP'') was released in May 1986, defining a file system for CD-ROMs commonly known as the High Sierra Format.

A draft version of this proposal was submitted to the European Computer Manufacturers Association (ECMA) for standardization. With some changes, this led to the issue of the initial edition of the ECMA-119 standard in December 1986. The ECMA submitted their standard to the International Standards Organization (ISO) for ''fast tracking'', where it was further refined into the ISO 9660 standard. For compatibility the second edition of ECMA-119 was revised to be equivalent to ISO 9660 in December 1987.https://web.archive.org/web/20161118181416/http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-119.pdf -->
/ref> ''ISO 9660:1988'' was published in 1988. The main changes from the High Sierra Format in the ECMA-119 and ISO 9660 standards were international extensions to allow the format to work better on non-US markets.

In order not to create incompatibilities, NISO suspended further work on Z39.60, which had been adopted by NISO members on 28 May 1987. It was withdrawn before final approval, in favour of ISO 9660.

In 2013, ISO published Amendment 1 to the ISO 9660 standard, introducing new data structures and relaxed file name rules intended to "bring harmonization between ISO 9660 and widely used ' Joliet Specification'." In December 2017, a 3rd Edition of ECMA-119 was published that is technically identical with ISO 9660, Amendment 1.

In 2020, ISO published Amendment 2, which adds some minor clarifying matter, but does not add or correct any technical information of the standard.

Specifications

The following is the rough overall structure of the ISO 9660 file system.

Multi-byte values can be stored in three different formats: little-endian, big-endian, and in a concatenation of both types in what the specification calls "both-byte" order. Both-byte order is required in several fields in the volume descriptors and directory records, while path tables can be either little-endian or big-endian.

Top level

The ''system area'', the first 32,768 data bytes of the disc (16 sectors of 2,048 bytes each), is unused by ISO 9660 and therefore available for other uses. While it is suggested that they are reserved for use by bootable media, a CD-ROM may contain an alternative file system descriptor in this area, and it is often used by hybrid CDs to offer classic Mac OS-specific and macOS-specific content.

Volume descriptor set

The ''data area'' begins with the ''volume descriptor set'', a set of one or more ''volume descriptors'' terminated with a ''volume descriptor set terminator''. These collectively act as a header for the data area, describing its content (similar to the BIOS parameter block used by FAT, HPFS and NTFS formatted disks).

Each volume descriptor is 2048 bytes in size, fitting perfectly into a single Mode 1 or Mode 2 Form 1 sector. They have the following structure:

The data field of a volume descriptor may be subdivided into several fields, with the exact content depending on the type. Redundant copies of each volume descriptor can also be included in case the first copy of the descriptor becomes corrupt.

Standard volume descriptor types are the following:

An ISO 9660 compliant disc must contain at least one ''primary volume descriptor'' describing the file system and a ''volume descriptor set terminator'' for indicating the end of the descriptor sequence. The ''volume descriptor set terminator'' is simply a particular type of volume descriptor with the purpose of marking the end of this set of structures. The primary volume descriptor provides information about the volume, characteristics and metadata, including a root directory record that indicates in which sector the root directory is located. Other fields contain the description or name of the volume, and information about who created it and with which application. The size of the logical blocks which the file system uses to segment the volume is also stored in a field inside the primary volume descriptor, as well as the amount of space occupied by the volume (measured in number of logical blocks).

In addition to the primary volume descriptor(s), ''supplementary volume descriptors'' or ''enhanced volume descriptors'' may be present. Supplementary volume descriptors describe the same volume as the primary volume descriptor does, and are normally used for providing additional code page support when the standard code tables are insufficient. The standard specifies that ISO 2022 is used for managing code sets that are wider than 8 bytes, and that ISO 2375 escape sequences are used to identify each particular code page used. Consequently, ISO 9660 supports international single-byte and multi-byte character sets, provided they fit into the framework of the referenced standards. However, ISO 9660 does not specify any code pages that are guaranteed to be supported: all use of code tables other than those defined in the standard itself are subject to agreement between the originator and the recipient of the volume. Enhanced volume descriptors were introduced in ISO 9660, Amendment 1. They relax some of the requirements of the other volume descriptors and the directory records referenced by them: for example, the directory depth can exceed eight, file identifiers need not contain '.' or file version number, the length of a file and directory identifier is maximized to 207.

Path tables

Path tables summarize the directory structure of the relevant directory hierarchy. For each directory in the image, the path table provides the directory identifier, the location of the extent in which the directory is recorded, the length of any extended attributes associated with the directory, and the index of its parent directory path table entry. The parent directory number is a 16-bit number, limiting its range from 1 to 65,535.

Directories and files

Directory entries are stored following the location of the root directory entry, where evaluation of filenames is begun. Both directories and files are stored as extents, which are sequential series of sectors. Files and directories are differentiated only by a file attribute that indicates its nature (similar to Unix). The attributes of a file are stored in the directory entry that describes the file, and optionally in the extended attribute record. To locate a file, the directory names in the file's path can be checked sequentially, going to the location of each directory to obtain the location of the subsequent subdirectory. However, a file can also be located through the path table provided by the file system. This path table stores information about each directory, its parent, and its location on disc. Since the path table is stored in a contiguous region, it can be searched much faster than jumping to the particular locations of each directory in the file's path, thus reducing seek time.

The standard specifies three nested levels of interchange (paraphrased from section 10):

* Level 1: File names are limited to eight characters with a three-character extension. Directory names are limited to eight characters. Files may contain one single file section.
* Level 2: Files may contain one single file section.
* Level 3: No additional restrictions than those stipulated in the main body of the standard. That is, directory identifiers may not exceed 31 characters in length, and file name + '.' + file name extension may not exceed 30 characters in length (sections 7.5 and 7.6). Files are also allowed to consist of multiple non-contiguous sections (with some restrictions as to order).

Additional restrictions in the body of the standard: The depth of the directory hierarchy must not exceed 8 (root directory being at level 1), and the path length of any file must not exceed 255. (section 6.8.2.1).

The standard also specifies the following name restrictions (sections 7.5 and 7.6):

* All levels restrict file names in the mandatory file hierarchy to upper case letters, digits, underscores ("_"), and a dot. (See also section 7.4.4 and Annex A.)
* If no characters are specified for the File Name then the File Name Extension shall consist of at least one character.
* If no characters are specified for the File Name Extension then the File Name shall consist of at least one character.
* File names shall not have more than one dot.
* Directory names shall not use dots at all.

A CD-ROM producer may choose one of the lower Levels of Interchange specified in chapter 10 of the standard, and further restrict file name length from 30 characters to only 8+3 in file identifiers, and 8 in directory identifiers in order to promote interchangeability with implementations that do not implement the full standard.

All numbers in ISO 9660 file systems except the single byte value used for the GMT offset are unsigned numbers. As the length of a file's extent on disc is stored in a 32 bit value, it allows for a maximum length of just over 4.2  GB (more precisely, one byte less than 4  GiB). It is possible to circumvent this limitation by using the multi-extent (fragmentation) feature of ISO 9660 Level 3 to create ISO 9660 file systems and single files up to 8 TB. With this, files larger than 4 GiB can be split up into multiple extents (sequential series of sectors), each not exceeding the 4 GiB limit. For example, the free software such as InfraRecorder, ImgBurn and mkisofs as well as Roxio Toast are able to create ISO 9660 file systems that use multi-extent files to store files larger than 4 GiB on appropriate media such as recordable DVDs. Linux supports multiple extents.

Extensions and improvements

There are several extensions to ISO 9660 that relax some of its limitations. Notable examples include ''Rock Ridge'' (Unix-style permissions and longer names), ''Joliet'' ( Unicode, allowing non- Latin scripts to be used), ''El Torito'' (enables CDs to be bootable) and the ''Apple ISO 9660 Extensions'' (file characteristics specific to the classic Mac OS and macOS, such as resource forks, file backup date and more).

SUSP

''System Use Sharing Protocol'' (SUSP, IEEE P1281) provides a generic way of including additional properties for any directory entry reachable from the primary volume descriptor (PVD). In an ISO 9660 volume, every directory entry has an optional ''system use area'' whose contents are undefined and left to be interpreted by the system. SUSP defines a method to subdivide that area into multiple system use fields, each identified by a two-character signature tag. The idea behind SUSP was that it would enable any number of independent extensions to ISO 9660 to be created and included on a volume without conflicting. It also allows for the inclusion of property data that would otherwise be too large to fit within the limits of the system use area.

SUSP defines several common tags and system use fields:

* CE: Continuation area
* PD: Padding field
* SP: System use sharing protocol indicator
* ST: System use sharing protocol terminator
* ER: Extensions reference
* ES: Extension selector

Other known SUSP fields include:

* AA: Apple extension, preferred
* BA: Apple extension, old (length attribute is missing)
* AS: Amiga file properties
* ZF: zisofs compressed file, usually produced by program mkzftree or by libisofs. Transparently decompressed by Linux kernel if built with CONFIG_ZISOFS.
* AL: records Extended File Attributes, including ACLs. Proposed by libburnia, supported by libisofs.

The Apple extensions do not technically follow the SUSP standard; however the basic structure of the AA and AB fields defined by Apple are forward compatible with SUSP; so that, with care, a volume can use both Apple extensions as well as RRIP extensions.

Rock Ridge

The ''Rock Ridge Interchange Protocol'' (RRIP, IEEE P1282) is an extension which adds POSIX file system semantics. The availability of these extension properties allows for better integration with Unix and Unix-like operating systems. The standard takes its name from the fictional town ''Rock Ridge'' in Mel Brooks' film ''Blazing Saddles''. The RRIP extensions are, briefly:

* Longer file names (up to 255 bytes) and fewer restrictions on allowed characters (support for lowercase, etc.)
* UNIX-style file modes, user ids and group ids, and file timestamps
* Support for Symbolic links and device files
* Deeper directory hierarchy (more than 8 levels)
* Efficient storage of sparse files

The RRIP extensions are built upon SUSP, defining additional tags for support of POSIX semantics, along with the format and meaning of the corresponding system use fields:

* RR: Rock Ridge extensions in-use indicator (note: dropped from standard after version 1.09)
* PX: POSIX file attributes
* PN: POSIX device numbers
* SL: symbolic link
* NM: alternate name
* CL: child link
* PL: parent link
* RE: relocated directory
* TF: time stamp
* SF: sparse file data

''Amiga Rock Ridge'' is similar to RRIP, except it provides additional properties used by AmigaOS. It too is built on the SUSP standard by defining an "AS"-tagged system use field. Thus both Amiga Rock Ridge and the POSIX RRIP may be used simultaneously on the same volume. Some of the specific properties supported by this extension are the additional Amiga-bits for files. There is support for attribute "P" that stands for "pure" bit (indicating re-entrant command) and attribute "S" for script bit (indicating batch file). This includes the protection flags plus an optional comment field. These extensions were introduced by Angela Schmidt with the help of Andrew Young, the primary author of the Rock Ridge Interchange Protocol and System Use Sharing Protocol. The first publicly available software to master a CD-ROM with Amiga extensions was MakeCD, an Amiga software which Angela Schmidt developed together with Patrick Ohly.

El Torito

''El Torito'' is an extension designed to allow booting a computer from a CD-ROM. It was announced in November 1994 and first issued in January 1995 as a joint proposal by IBM and BIOS manufacturer Phoenix Technologies. According to legend, the El Torito CD/DVD extension to ISO 9660 got its name because its design originated in an El Torito

HOME


Content is Copyleft
Website design, code, and AI is Copyrighted (c) 2014-2017 by Stephen Payne


Consider donating to Wikimedia



As an Amazon Associate I earn from qualifying purchases