Record separator

The C0 and C1 control code or control character sets define control codes for use in text by computer systems that use ASCII and derivatives of ASCII. The codes represent additional information about the text, such as the position of a cursor, an instruction to start a new line, or a message that the text has been received.

C0 codes are the range 00 _HEX–1F_HEX and the default C0 set was originally defined in ISO 646 (ASCII). C1 codes are the range 80_HEX–9F_HEX and the default C1 set was originally defined in ECMA-48 (harmonized later with ISO 6429). The ISO/IEC 2022 system of specifying control and graphic characters allows other C0 and C1 sets to be available for specialized applications, but they are rarely used.

C0 controls

ASCII defined 32 control characters, plus a necessary extra character for the DEL character, 7F_HEX or 01111111_BIN (needed to punch out all the holes on a paper tape and erase it).

This large number of codes was desirable at the time, as multi-byte controls would require implementation of a state machine in the terminal, which was very difficult with contemporary electronics and mechanical terminals. Since then, only a few of the original controls have maintained their use: the "whitespace" range of BS, TAB, LF, VT, FF, and CR; the BEL code; and ESC (but almost always as part of an ESC,' __representation_starting_an_ANSI_escape_sequence).__Others_are_unused_or_have_acquired_different_meanings_such_as_NUL_being_the_C_(programming_language).html" ;"title="ANSI_escape_sequence.html" ;"title=" representation starting an ANSI escape sequence"> representation starting an ANSI escape sequence). Others are unused or have acquired different meanings such as NUL being the C (programming language)">C string terminator.

Some serial transmission protocols such as ANPA-1312, Kermit (protocol), Kermit, and XMODEM do make extensive use of control characters SOH, STX, ETX, EOT, ACK, NAK and SYN for purposes approximating their original definitions.

Basic ASCII control codes

These are the standard ASCII control codes, originally defined in ANSI X3.4. If using the ISO/IEC 2022 extension mechanism, they are designated as the active C0 control character set with the octet sequence 0x1B 0x21 0x40 (ESC ! @).

Category-Number names

Several of the basic ASCII control codes are classified into a few categories, and sometimes given alternative abbreviated names consisting of that category and a number:
* Transmission controls: TC₁ (SOH), TC₂ (STX), TC₃ (ETX), TC₄ (EOT), TC₅ (ENQ), TC₆ (ACK), TC₇ (DLE), TC₈ (NAK), TC₉ (SYN), TC₁₀ (ETB).
* Format effectors: FE₀ (BS), FE₁ (HT), FE₂ (LF), FE₃ (VT), FE₄ (FF), FE₅ (CR).
* Device controls: DC₁, DC₂, DC₃, DC₄.
* Information separators: IS₁ (US), IS₂ (RS), IS₃ (GS), IS₄ (FS).
* Locking shifts: LS₀ (SI), LS₁ (SO).
* Others: NUL, BEL, CAN, EM, SUB, ESC.

ISO/IEC 2022 (ECMA-35) refers to the C0 locking shifts as LS0 and LS1 in 8-bit environments, and as SI and SO in 7-bit environments.

The first, 1963 edition of ASCII classified as a device control, rather than a transmission control, and gave it the abbreviation DC0 ("device control reserved for data link escape").

Format (FE) codes define and actuate formatting (such as line breaks) which affects how graphical characters are laid out and rendered, as opposed to controlling other functions of hardware devices or having other side effects. The C0 format effectors are permitted in ISO/IEC 6429 , , and sequences. The information separators and C0 format effectors (minus ) are the only C0 control codes with semantics defined by the Unicode Standard, the interpretation of the remainder of the C0 controls being left to higher-level protocols.

ISO/IEC 2022 (ECMA-35) requires that if C0 control code sets include the ten ASCII transmission control (TC) codes, they must be encoded at their ASCII locations. It also prohibits those ten transmission controls from being included in a C1 control code set, and prohibits transmission controls besides those ten from being included in a C0 control set.

Modified C0 control code sets

Although C0 control code sets usually preserve most of the ASCII control codes unchanged, a number are registered which replace certain control functions with alternatives. A selection of these, excluding those related to Videotex, are shown below.

Other C0 control code sets

Teletext defines an entirely different set of control codes. In formats where compatibility with ECMA-48's C0 control codes is not required, these control codes are sometimes mapped transparently to the Unicode C0 control code range (U+0000 through U+001F).

C1 controls

In parallel to the development of the 1972 edition of ISO 646, which revised the standard to introduce the concept of national versions of the code in addition to the US-originated ASCII, work was also underway with the purpose of defining extension mechanisms for ASCII, applicable to both 7-bit and 8-bit environments, which would be published as ECMA-35 and ISO 2022.

These mechanisms were designed so that any conformant 8-bit code could be converted to a corresponding 7-bit code, and ''vice versa''. In a 7-bit environment, the Shift Out () control would change the meaning of the 94 bytes 0x21 through 0x7E (i.e. the graphical codes, excluding the space) to invoke characters from an alternative set, and the Shift In () control would change them back. In an 8-bit environment, instead of using shift codes, the eighth bit was set on a byte referencing the additional graphic character set. This meant that bytes 0xA1 through 0xFE were used for the additional graphic characters. The C0 control characters, being unaffected by the shift state of a 7-bit code, were to always be represented in an 8-bit code with the eighth bit unset. The consequently otherwise-unused bytes in the range 0x80 through 0x9F could be used for additional control codes, which would instead be represented as 0x1B 0x40 through 0x1B 0x5F (ESC @ through ESC _) in a 7-bit code. These additional control codes become known as the C1 control codes. To retain compatibility with the 7-bit representation, the behaviour of bytes 0xA0 and 0xFF was originally left undefined.

The first C1 control code set to be registered for use with ISO 2022 was DIN 31626, a specialised set for bibliographic use which was registered in 1979. The general-use ISO/IEC 6429 set was registered in 1983, although the ECMA-48 specification upon which it was based had been first published in 1976.

Further editions of the standards altered the provisions to an extent. For instance, a further revision to ECMA-35 and ISO 2022 in 1985 introduced the concept of a 96-code graphical character set. In an 8-bit code, this allowed the entire range from 0xA0 to 0xFF to be used for graphical characters. Use of 96-code sets also meant that the meaning of the bytes 0x20 and 0x7F in the corresponding 7-bit code could differ from "Space" and "Delete", unless the code was in the Shift In state. Using 96-code sets for the G0 (Shift In) set was not made possible.

In accordance with this revised 8-bit ISO 2022 code structure, ISO 8859 defines sets of characters to be encoded over 0xA0–FF, in combination with the ASCII graphical characters over 0x20–7E, and reserves the bytes outside of these ranges for use as non-graphical codes by other specifications such as ISO/IEC 6429. Unicode inherits its first 256 code points from ISO 8859-1, hence also incorporating a range reserved for a C1 control code set, although it mostly leaves their function to be defined by higher level protocols, with ISO/IEC 6429 suggested as a default.

C1 control codes for general use

These are the most common extended control codes, and are defined in ISO/IEC 6429, ECMA-48 and JIS X 0211 (formerly JIS C 6323). If using the ISO/IEC 2022 extension mechanism, they are designated as the active C1 control character set with the sequence 0x1B 0x22 0x43 (ESC " C). Although Unicode does not require a particular C1 control code set, leaving their interpretation to be specified by higher-level protocols, and only specifies a behaviour for U+0085, it suggests interpreting C1 control codes as specified in ISO/IEC 6429 in the absence of use for other purposes. Also listed in the table below are three control codes listed alongside the ISO/IEC 6429 codes in , but not actually defined by ISO/IEC 6429 (, and ).

Except for and in EUC-JP text, and in text transcoded from EBCDIC, the 8-bit forms of these codes are almost never used. , and are used to control text terminals and terminal emulators, but almost always by using their 7-bit escape code representations. Their ISO/IEC 2022 compliant single-byte representations are invalid in UTF-8, and the UTF-8 encodings of their corresponding codepoints are two bytes long like their escape code forms (for instance, CSI at U+009B is encoded as the bytes 0xC2, 0x9B in UTF-8), so there is no advantage to using them rather than the equivalent two-byte escape sequence. When these codes appear in modern documents, web pages, e-mail messages, etc., they are usually intended to be printing characters at that position in a proprietary encoding such as Windows-1252 or Mac OS Roman that use the C1 codes to provide additional graphic characters.

The official English language names of some C1 codes were revised in the most recent edition of the standard for control codes in general (ISO 6429:1992 or ECMA-48:1991) to be neutral with respect to the graphic characters used with them, and to not assume that, as in the Latin script, lines are written on a page from top to bottom and that characters are written on a line from left to right. The abbreviations used were not changed, as the standard had already specified that those would remain unchanged when the standard is translated to other languages. Where the name has been changed, the original name from which the abbreviation was derived is also given in parenthesis in the tables below.

C1 control codes for bibliographic use

The following alternative C1 control code set is defined for bibliographic applications such as library systems. It is mostly concerned with string collation, and with markup of bibliographic fields. Slightly different variants are defined in the German standard DIN 31626 (published in 1978 and since withdrawn) and the ISO standard ISO 6630, the latter of which has also been adopted in Germany as DIN ISO 6630. Where these differ is noted in the table below where applicable. MARC-8 uses the coding of and from this set, and adds some additional format effectors in locations not used by the ISO version; however, MARC 21 uses this control set only in MARC-8 records, not in Unicode-format records.

If using the ISO/IEC 2022 extension mechanism, the DIN 31626 set is designated as the active C1 control character set with the sequence 0x1B 0x22 0x45 (ESC " E), and the ISO 6630 / DIN ISO 6630 set is designated with the sequence 0x1B 0x22 0x42 (ESC " B). The 1985 expansion of the ISO 6630 set can also be explicitly specified by using the sequence 0x1B 0x26 0x40 0x1B 0x22 0x42 (ESC & @ ESC " B).

Other C1 control code sets

EBCDIC defines 16 additional control codes, besides those present in ASCII. When mapped to Unicode or to ISO 8859, these codes are mapped to C1 control characters in a manner specified by IBM's Character Data Representation Architecture (CDRA).

Although the default mapping of the New Line (NL) control does correspond to the ISO/IEC 6429 (0x85; although its mapping is sometimes swapped with LF, following UNIX line ending convention), the remainder of the control codes do not correspond to ISO/IEC 6429. For example, the EBCDIC control (0x09, mapped to 0x8D) and the ECMA-48 control (0x8C) are both used to begin a superscript or end a subscript, but are not mapped to one another. Extended-ASCII-mapped EBCDIC can therefore be regarded as having its own C1 set, although it is not registered with the ISO-IR registry for use with ISO/IEC 2022.

Various specialised C1 control code sets are registered for use by various Videotex formats.

Unicode

Unicode sets aside 65 code points in the general category (control) for compatibility with ISO/IEC 2022. These are:
* U+0000–U+001F (C0 controls) and U+007F (delete) assigned to the Basic Latin block, and
* U+0080–U+009F (C1 controls) assigned to the Latin-1 Supplement block.
Unicode only specifies semantics for U+0009–U+000D, U+001C–U+001F, and U+0085. The rest of the control codes are transparent to Unicode and their meanings are left to higher-level protocols.

Besides these, Unicode include additional format effector characters besides these, such as marks, embeds, isolates and pops for explicit bidirectional formatting, and the zero-width joiner and non-joiner for controlling ligature use. However these are given the general category (format) rather than .

See also

* Control Pictures
* ANSI escape code

Footnotes

References

* The Unicode Standard
*
C0 Controls and Basic Latin
*
C1 Controls and Latin-1 Supplement
*
Control Pictures
** The Unicode Standard, Version 6.1.0
Chapter 16: Special Areas and Format Characters

ATIS Telecom Glossary 2007

''De litteris regentibus C1 quaestiones septem'' or ''Are C1 characters legal in XHTML 1.0?''

W3C I18N FAQ: HTML, XHTML, XML and Control Codes

International register of coded character sets to be used with escape sequences

{{character encoding

Control characters

de:Steuerzeichen