Western Latin Character Sets (computing)

Several binary representations of 8-bit character sets for common Western European languages are compared in this article. These encodings were designed for representation of Italian, Spanish, Portuguese, French, German, Dutch, English, Danish, Swedish, Norwegian, and Icelandic, which use the Latin alphabet, a few additional letters and ones with precomposed diacritics, some punctuation, and various symbols (including some Greek letters). Although they're called "Western European" many of these languages are spoken all over the world. Also, these character sets happen to support many other languages such as Malay, Swahili, and Classical Latin.

''This material is technically obsolete, having been functionally replaced by Unicode. However it continues to have historical interest.''

Summary

The ISO-8859 series of 8-bit character sets encodes all Latin character sets used in Europe, albeit that the same code points have multiple uses that caused some difficulty (including mojibake, or garbled characters, and communication issues). The arrival of Unicode, with a unique code point for every glyph, resolved these issues.

* ISO/IEC 8859-1 or Latin-1 is the most used and also defines the first 256 codes in Unicode.
* ISO/IEC 8859-15 modifies ISO-8859-1 to fully support Estonian, Finnish and French and add the euro sign.
* Windows-1252 is a superset of ISO-8859-1 that includes the printable characters from ISO/IEC 8859-15 and popular punctuation such as curved quotation marks (also known as smart quotes, such as in Microsoft Word settings and similar programs). It is common that web page tools for Windows use Windows-1252 but label the web page as using ISO-8859-1, this has been addressed in HTML5, which mandates that pages labeled as ISO-8859-1 must be interpreted as Windows-1252.
* IBM CP437, being intended for English only, has very little in the way of accented letters (particularly uppercase) but has far more graphics characters than the other IBM code pages listed here and also some mathematical and Greek characters that are useful as technical symbols.
* IBM CP850 has all the printable characters that ISO-8859-1 has (albeit arranged differently) and still manages to have enough graphics characters to build a usable text-mode user interface.
* IBM CP858 differs from CP850 only by one character — a ''dotless i'' ( ı), rarely used outside Turkey and with no uppercase equivalent provided, was replaced by ''euro currency sign'' ( €).
* IBM CP859 contains all the printable characters that ISO/IEC 8859-15 has, so unlike CP850 it supports the euro sign, Estonian, Finnish and French.
* IBM code pages 037, 500, and 1047 are EBCDIC encodings that include all of the ISO-8859-1 characters.
* The Mac OS Roman character set (often referred to as MacRoman and known by the IANA as simply MACINTOSH) has most, but not all, of the same characters as ISO/IEC 8859-1 but in a very different arrangement; and it also adds many technical and mathematical characters (though it lacks the important ×) and more diacritics. Older Macintosh web browsers were known to munge the few characters that were in ISO/IEC 8859-1 but not their native Macintosh character set when editing text from Web sites. Conversely, in Web material prepared on an older Macintosh, many characters were displayed incorrectly when read by other operating systems. The Macintosh Latin encoding, a modification of Mac OS Roman to support ISO/IEC 8859-1, was created by the creators of Kermit (protocol) to solve this problem.

History

The earlier seven- bit U.S. American Standard Code for Information Interchange ('ASCII') encoding has characters sufficient to properly represent only a few languages such as English, Latin, Malay and Swahili. It is missing some letters and letter-diacritic combinations used in other Latin-alphabet languages. However, since there was no other choice on most US-supplied computer platforms, use of ASCII was unavoidable except where there was a strong national computing industry. There was the ISO 646 group of encodings which replaced some of the symbols in ASCII with local characters, but space was very limited, and some of the symbols replaced were quite common in things like programming languages.

Most computers internally used eight-bit bytes but communication (seen as inherently unreliable) used seven data bits plus one parity bit. In time, it became common to use all eight bits for data, creating space for another 128 characters. In the early days most of these were system specific, but gradually the ISO/IEC 8859 standards emerged to provide some cross-platform similarity to enable information interchange.

Towards the end of the 20th century, as storage and memory costs fell, the issues associated with multiple meanings of a given eight-bit code (there are seven ISO-Latin code sets alone) have ceased to be justified. All major operating systems have moved to Unicode as their main internal representation. However, as Windows did not support the UTF-8 method of encoding Unicode (preferring UTF-16), many applications continued to be restricted to these legacy character sets.

The euro sign

The introduction of the euro and its associated euro sign () introduced significant pressure on computer systems developers to support this new symbol, and most 8-bit character sets had to be adapted in some way.
* Apple with MacRoman and Sun Microsystems with Solaris OS simply replaced the generic currency sign (). This caused difficulty in some places because organisations had found other uses for its code point, such as the company logo.
* ISO introduced a further variant of ISO 8859, ISO 8859-15, which replaced the generic currency sign with the euro sign as well as making some other replacements of symbols with letters with diacritics. ISO 8859-15 never received widespread adoption.
* With Windows-1252, Microsoft placed the euro sign in a gap (position 80_hex) in the existing C1 control codes, a decision that other vendors considered counter-architectural.
Whilst these decisions had limited effect for documents that were only used within a single computer (or at least within a single vendor's "digital ecosystem"), it meant that documents containing a euro sign would fail to render as expected when interchanged between ecosystems.

All of these issues have been resolved as operating systems have been upgraded to support Unicode as standard, which encodes the euro sign at U+20AC (decimal 8364).

Comparison table

Code points to U+007F are not shown in this table currently, as they are directly mapped in all character sets listed here. The ASCII coding standard defines the original specification for the mapping of the first 0-127 characters.

The table is arranged by Unicode code point. Character sets are referred to here by their IANA names in upper case.

* The mappings for the IBM code pages are from the Unicode site supplied by Microsoft. The Unicode Consortium's document has links to sources giving the differences between IBM's and Microsoft's mappings for these code pages.
* IBM437 and IBM850 defined printable characters for the control code ranges. While these could not be used when printing text through DOS, as they would be trapped before reaching the screen, they could be used by applications that used screen memory directly.
* Macintosh has an Apple logo at 0xF0, and translates it to U+F8FF in the Private Use Area for Unicode.

Notes

References

{{DEFAULTSORT:Western Latin Character Sets (Computing)
Character sets
Articles with unsupported PUA characters
History of computing