![Illustration of a newline](https://upload.wikimedia.org/wikipedia/commons/b/b3/Illustration_of_a_newline.png)
Newline (frequently called line ending, end of line (EOL), next line (NEL) or line break) is a
control character
In computing and telecommunication, a control Character (computing), character or non-printing character (NPC) is a code point (a number) in a character encoding, character set, that does not represent a written symbol. They are used as in-band ...
or sequence of control characters in
character encoding
Character encoding is the process of assigning numbers to Graphics, graphical character (computing), characters, especially the written characters of Language, human language, allowing them to be Data storage, stored, Data communication, transmi ...
specifications such as
ASCII
ASCII ( ), abbreviated from American Standard Code for Information Interchange, is a character encoding standard for electronic communication. ASCII codes represent text in computers, telecommunications equipment, and other devices. Because of ...
,
EBCDIC
Extended Binary Coded Decimal Interchange Code (EBCDIC; ) is an eight-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding six- ...
,
Unicode
Unicode, formally The Unicode Standard,The formal version reference is is an information technology Technical standard, standard for the consistent character encoding, encoding, representation, and handling of Character (computing), text expre ...
, etc. This character, or a sequence of characters, is used to signify the end of a
line of text
{{Unreferenced, date=May 2008
In computing, a line is a unit of organization for text files. A line consists of a sequence of zero or more Character (computing), characters, usually displayed within a single horizontal sequence.
Depending on the ...
and the start of a new one.
History
In the mid-1800s, long before the advent of
teleprinter
A teleprinter (teletypewriter, teletype or TTY) is an electromechanical device that can be used to send and receive typed messages through various communications channels, in both point-to-point and point-to-multipoint configurations. Initia ...
s and teletype machines,
Morse code
Morse code is a method used in telecommunication to encode text characters as standardized sequences of two different signal durations, called ''dots'' and ''dashes'', or ''dits'' and ''dahs''. Morse code is named after Samuel Morse, one of ...
operators or
telegraphist
A telegraphist (British English), telegrapher (American English), or telegraph operator is an operator who uses a telegraph key to send and receive the Morse code in order to communicate by land lines or radio.
During the Great War the Roya ...
s invented and used
Morse code prosigns to encode white space text formatting in formal written text messages. In particular the
Morse
Morse may refer to:
People
* Morse (surname)
* Morse Goodman (1917-1993), Anglican Bishop of Calgary, Canada
* Morse Robb (1902–1992), Canadian inventor and entrepreneur
Geography Antarctica
* Cape Morse, Wilkes Land
* Mount Morse, Churchi ...
prosign (mnemonic reak ext) represented by the concatenation of literal textual Morse codes "B" and "T" characters sent without the normal inter-character spacing is used in Morse code to encode and indicate a ''new line'' or ''new section'' in a formal text message.
Later, in the age of modern
teleprinter
A teleprinter (teletypewriter, teletype or TTY) is an electromechanical device that can be used to send and receive typed messages through various communications channels, in both point-to-point and point-to-multipoint configurations. Initia ...
s, standardized character set control codes were developed to aid in white space text formatting. ASCII was developed simultaneously by the
International Organization for Standardization
The International Organization for Standardization (ISO ) is an international standard development organization composed of representatives from the national standards organizations of member countries. Membership requirements are given in Ar ...
(ISO) and the American Standards Association (ASA), the latter being the predecessor organization to
American National Standards Institute
The American National Standards Institute (ANSI ) is a private non-profit organization that oversees the development of voluntary consensus standards for products, services, processes, systems, and personnel in the United States. The organi ...
(ANSI). During the period of 1963 to 1968, the ISO draft standards supported the use of either
+ or
alone as a newline, while the ASA drafts supported only +.
The sequence + was commonly used on many early computer systems that had adopted
Teletype
A teleprinter (teletypewriter, teletype or TTY) is an electromechanical device that can be used to send and receive typed messages through various communications channels, in both point-to-point and point-to-multipoint configurations. Initia ...
machines—typically a
Teletype Model 33
The Teletype Model 33 is an electromechanical teleprinter designed for light-duty office use. It is less rugged and cost less than earlier Teletype machines. The Teletype Corporation introduced the Model 33 as a commercial product in 1963 after ...
ASR—as a console device, because this sequence was required to position those printers at the start of a new line. The separation of newline into two functions concealed the fact that the print head could not return from the far right to the beginning of the next line in time to print the next character. Any character printed after a would often print as a smudge in the middle of the page while the print head was still moving the carriage back to the first position. "The solution was to make the newline two characters: to move the carriage to column one, and to move the paper up." In fact, it was often necessary to send extra characters—extraneous CRs or NULs—which are ignored but give the print head time to move to the left margin. Many early video displays also required multiple character times to
scroll
A scroll (from the Old French ''escroe'' or ''escroue''), also known as a roll, is a roll of papyrus, parchment, or paper containing writing.
Structure
A scroll is usually partitioned into pages, which are sometimes separate sheets of papyrus ...
the display.
On such systems, applications had to talk directly to the Teletype machine and follow its conventions since the concept of
device driver
In computing, a device driver is a computer program that operates or controls a particular type of device that is attached to a computer or automaton. A driver provides a software interface to hardware devices, enabling operating systems and ot ...
s hiding such hardware details from the application was not yet well developed. Therefore, text was routinely composed to satisfy the needs of Teletype machines. Most minicomputer systems from
DEC used this convention.
CP/M
CP/M, originally standing for Control Program/Monitor and later Control Program for Microcomputers, is a mass-market operating system created in 1974 for Intel 8080/ 85-based microcomputers by Gary Kildall of Digital Research, Inc. Initial ...
also used it in order to print on the same terminals that minicomputers used. From there
MS-DOS
MS-DOS ( ; acronym for Microsoft Disk Operating System, also known as Microsoft DOS) is an operating system for x86-based personal computers mostly developed by Microsoft. Collectively, MS-DOS, its rebranding as IBM PC DOS, and a few ope ...
(1981) adopted
CP/M
CP/M, originally standing for Control Program/Monitor and later Control Program for Microcomputers, is a mass-market operating system created in 1974 for Intel 8080/ 85-based microcomputers by Gary Kildall of Digital Research, Inc. Initial ...
's + in order to be compatible, and this convention was inherited by Microsoft's later
Windows
Windows is a group of several proprietary graphical operating system families developed and marketed by Microsoft. Each family caters to a certain sector of the computing industry. For example, Windows NT for consumers, Windows Server for serv ...
operating system.
The
Multics
Multics ("Multiplexed Information and Computing Service") is an influential early time-sharing operating system based on the concept of a single-level memory.Dennis M. Ritchie, "The Evolution of the Unix Time-sharing System", Communications of t ...
operating system began development in 1964 and used alone as its newline. Multics used a device driver to translate this character to whatever sequence a printer needed (including extra padding characters), and the single byte was more convenient for programming. What seems like a more obvious choice——was not used, as provided the useful function of overprinting one line with another to create
boldface
In typography, emphasis is the strengthening of words in a text with a font in a different style from the rest of the text, to highlight them. It is the equivalent of prosody stress in speech.
Methods and use
The most common methods in W ...
,
underscore
An underscore, ; also called an underline, low line, or low dash; is a line drawn under a segment of text. In proofreading, underscoring is a convention that says "set this text in italic type", traditionally used on Manuscript (publishing), man ...
and
strikethrough
Strikethrough is a typographical presentation of words with a horizontal line through their center, resulting in . Contrary to censored or sanitized (redacted) texts, the words remain readable. This presentation signifies one of two meanings. In ...
effects. Perhaps more importantly, the use of alone as a line terminator had already been incorporated into drafts of the eventual
ISO/IEC 646
ISO/IEC 646 is a set of International Organization for Standardization, ISO/International Electrotechnical Commission, IEC standards, described as ''Information technology — ISO 7-bit coded character set for information interchange' ...
standard.
Unix
Unix (; trademarked as UNIX) is a family of multitasking, multiuser computer operating systems that derive from the original AT&T Unix, whose development started in 1969 at the Bell Labs research center by Ken Thompson, Dennis Ritchie, and ot ...
followed the Multics practice, and later
Unix-like
A Unix-like (sometimes referred to as UN*X or *nix) operating system is one that behaves in a manner similar to a Unix system, although not necessarily conforming to or being certified to any version of the Single UNIX Specification. A Unix-li ...
systems followed Unix. This created conflicts between Windows and Unix-like
operating system
An operating system (OS) is system software that manages computer hardware, software resources, and provides common services for computer programs.
Time-sharing operating systems schedule tasks for efficient use of the system and may also in ...
s, whereby files composed on one operating system cannot be properly formatted or interpreted by another operating system (for example a
UNIX shell script written in a Windows text editor like
Notepad
A notebook (also known as a notepad, writing pad, drawing pad, or legal pad) is a book or stack of paper pages that are often Ruled paper, ruled and used for purposes such as note-taking, diary, journaling or other writing, drawing, or scrapbook ...
).
Representation
The concepts of
carriage return
A carriage return, sometimes known as a cartridge return and often shortened to CR, or return, is a control character or mechanism used to reset a device's position to the beginning of a line of text. It is closely associated with the line feed a ...
(CR) and line feed (LF) are closely associated and can be considered either separately or together. In the physical media of
typewriter
A typewriter is a mechanical or electromechanical machine for typing characters. Typically, a typewriter has an array of keys, and each one causes a different single character to be produced on paper by striking an inked ribbon selectivel ...
s and
printer
Printer may refer to:
Technology
* Printer (publishing), a person or a company
* Printer (computing), a hardware device
* Optical printer for motion picture films
People
* Nariman Printer (fl. c. 1940), Indian journalist and activist
* James ...
s, two
axes
Axes, plural of ''axe'' and of ''axis'', may refer to
* ''Axes'' (album), a 2005 rock album by the British band Electrelane
* a possibly still empty plot (graphics)
See also
*Axess (disambiguation)
*Axxess (disambiguation) Axxess may refer to:
...
of motion, "down" and "across", are needed to create a new line on the
page
Page most commonly refers to:
* Page (paper), one side of a leaf of paper, as in a book
Page, PAGE, pages, or paging may also refer to:
Roles
* Page (assistance occupation), a professional occupation
* Page (servant), traditionally a young m ...
. Although the design of a machine (typewriter or printer) must consider them separately, the abstract logic of software can combine them together as one event. This is why a newline in
character encoding
Character encoding is the process of assigning numbers to Graphics, graphical character (computing), characters, especially the written characters of Language, human language, allowing them to be Data storage, stored, Data communication, transmi ...
can be defined as and combined into one (commonly called or ).
Some
character sets
Character encoding is the process of assigning numbers to graphical characters, especially the written characters of human language, allowing them to be stored, transmitted, and transformed using digital computers. The numerical values that ...
provide a separate newline character code.
EBCDIC
Extended Binary Coded Decimal Interchange Code (EBCDIC; ) is an eight-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding six- ...
, for example, provides an character code in addition to the and codes.
Unicode
Unicode, formally The Unicode Standard,The formal version reference is is an information technology Technical standard, standard for the consistent character encoding, encoding, representation, and handling of Character (computing), text expre ...
, in addition to providing the
ASCII
ASCII ( ), abbreviated from American Standard Code for Information Interchange, is a character encoding standard for electronic communication. ASCII codes represent text in computers, telecommunications equipment, and other devices. Because of ...
and
control codes, also provides a "next line" () control code, as well as control codes for "line separator" and "paragraph separator" markers.
*
EBCDIC
Extended Binary Coded Decimal Interchange Code (EBCDIC; ) is an eight-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding six- ...
systems—mainly
IBM mainframe systems, including
z/OS
z/OS is a 64-bit operating system for IBM z/Architecture mainframes, introduced by IBM in October 2000. It derives from and is the successor to OS/390, which in turn was preceded by a string of MVS versions.Starting with the earliest:
* O ...
(
OS/390
OS/390 is an IBM operating system for the System/390 IBM mainframe computers.
Overview
OS/390 was introduced in late 1995 in an effort to simplify the packaging and ordering for the key, entitled elements needed to complete a fully functional ...
) and
IBM i
IBM i (the ''i'' standing for ''integrated'') is an operating system developed by IBM for IBM Power Systems. It was originally released in 1988 as OS/400, as the sole operating system of the IBM AS/400 line of systems. It was renamed to i5/OS in ...
(
OS/400
IBM i (the ''i'' standing for ''integrated'') is an operating system developed by IBM for IBM Power Systems. It was originally released in 1988 as OS/400, as the sole operating system of the IBM AS/400 line of systems. It was renamed to i5/OS ...
)—use (New Line, ) as the character combining the functions of line feed and carriage return. The equivalent Unicode character () is called (Next Line). EBCDIC also has control characters called and , but the numerical value of () differs from the one used by ASCII (). Additionally, some EBCDIC variants also use but assign a different numeric code to the character. However, those operating systems use a
record-based file system, which stores text files as one record per line. In most file formats, no line terminators are actually stored.
*Operating systems for the
CDC 6000 series The CDC 6000 series is a discontinued family of mainframe computers manufactured by Control Data Corporation in the 1960s. It consisted of the CDC 6200, CDC 6300, CDC 6400, CDC 6500, CDC 6600 and CDC 6700 computers, which were all extremely rapid a ...
defined a newline as two or more zero-valued six-bit characters at the end of a 60-bit word. Some configurations also defined a zero-valued character as a
colon character, with the result that multiple colons could be interpreted as a newline depending on position.
*
RSX-11
RSX-11 is a discontinued family of multi-user real-time operating systems for PDP-11 computers created by Digital Equipment Corporation. In widespread use through the late 1970s and early 1980s, RSX-11 was influential in the development of later ...
and
OpenVMS
OpenVMS, often referred to as just VMS, is a multi-user, multiprocessing and virtual memory-based operating system. It is designed to support time-sharing, batch processing, transaction processing and workstation applications. Customers using Ope ...
also use a record-based file system, which stores text files as one record per line. In most file formats, no line terminators are actually stored, but the
Record Management Services
Record Management Services (RMS) are procedures in the VMS, RSTS/E, RT-11 and RSX-11M operating systems that programs may call to process files and records within files.
Its file formats and procedures are similar to of those in some IBM acces ...
facility can transparently add a terminator to each line when it is retrieved by an application. The records themselves can contain the same line terminator characters, which can either be considered a feature or a nuisance depending on the application. RMS not only stores records, but also stores metadata about the record separators in different bits for the file to complicate matters even more (since files can have fixed length records, records that are prefixed by a count or records that are terminated by a specific character). The bits are not generic, so while they can specify that or or even is the line terminator, they can not substitute some other code.
*''Fixed line length'' was used by some early
mainframe operating systems. In such a system, an implicit end-of-line was assumed every 72 or 80 characters, for example. No newline character was stored. If a file was imported from the outside world, lines shorter than the line length had to be padded with spaces, while lines longer than the line length had to be truncated. This mimicked the use of
punched card
A punched card (also punch card or punched-card) is a piece of stiff paper that holds digital data represented by the presence or absence of holes in predefined positions. Punched cards were once common in data processing applications or to di ...
s, on which each line was stored on a separate card, usually with 80 columns on each card, often with sequence numbers in columns 73–80. Many of these systems added a
carriage control character to the start of the ''next'' record; this could indicate whether the next record was a continuation of the line started by the previous record, or a new line, or should overprint the previous line (similar to a ). Often this was a normal printing character such as that thus could not be used as the first character in a line. Some early line printers interpreted these characters directly in the records sent to them.
Unicode
The
Unicode
Unicode, formally The Unicode Standard,The formal version reference is is an information technology Technical standard, standard for the consistent character encoding, encoding, representation, and handling of Character (computing), text expre ...
standard defines a number of characters that conforming applications should recognize as line terminators:
:: Line Feed,
::
Vertical Tab
The tab key (abbreviation of tabulator key or tabular key) on a keyboard is used to advance the cursor to the next tab stop.
History
The word ''tab'' derives from the word ''tabulate'', which means "to arrange data in a tabular, or table, fo ...
,
::
Form Feed
A page break is a marker in an electronic document that tells the document interpreter that the content which follows is part of a new page. A page break causes a form feed to be sent to the printer during spooling of the document to the printer. ...
,
::
Carriage Return
A carriage return, sometimes known as a cartridge return and often shortened to CR, or return, is a control character or mechanism used to reset a device's position to the beginning of a line of text. It is closely associated with the line feed a ...
,
:+: () followed by ()
:: Next Line,
:: Line Separator,
:: Paragraph Separator,
This may seem overly complicated compared to an approach such as converting all line terminators to a single character, for example . However, Unicode was designed to preserve all information when converting a text file from any existing encoding to Unicode and back. Therefore, Unicode should contain characters included in existing encodings.
For example: is part of
EBCDIC
Extended Binary Coded Decimal Interchange Code (EBCDIC; ) is an eight-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding six- ...
, which uses code ; it is normally mapped to Unicode , , which is a control character in the C1 control set. As such, it is defined by ECMA 48, and recognized by encodings compliant with
ISO/IEC 2022
ISO/IEC 2022 ''Information technology—Character code structure and extension techniques'', is an ISO/IEC standard (equivalent to the ECMA standard ECMA-35, the ANSI standard ANSI X3.41 and the Japanese Industrial Standard JIS X 0202) in the ...
(which is equivalent to ECMA 35). C1 control set is also compatible with
ISO-8859-1
ISO/IEC 8859-1:1998, ''Information technology — 8-bit single-byte coded graphic character sets — Part 1: Latin alphabet No. 1'', is part of the ISO/IEC 8859 series of ASCII-based standard character encodings, first edition published in 1 ...
. The approach taken in the Unicode standard allows round-trip transformation to be information-preserving while still enabling applications to recognize all possible types of line terminators.
Recognizing and using the newline codes greater than (, and ) is not often done. They are multiple bytes in
UTF-8
UTF-8 is a variable-width encoding, variable-length character encoding used for electronic communication. Defined by the Unicode Standard, the name is derived from ''Unicode'' (or ''Universal Coded Character Set'') ''Transformation Format 8-bit'' ...
, and the code for has been used as the
ellipsis
The ellipsis (, also known informally as dot dot dot) is a series of dots that indicates an intentional omission of a word, sentence, or whole section from a text without altering its original meaning. The plural is ellipses. The term origin ...
() character in
Windows-1252
Windows-1252 or CP-1252 ( code page 1252) is a single-byte character encoding of the Latin alphabet, used by default in the legacy components of Microsoft Windows for English and many European languages including Spanish, French, and German.
It ...
. For instance:
*
ECMAScript
ECMAScript (; ES) is a JavaScript standard intended to ensure the interoperability of web pages across different browsers. It is standardized by Ecma International in the documenECMA-262
ECMAScript is commonly used for client-side scripting o ...
accepts and as line-breaks, but considers ()
whitespace instead of a line-break.
*
Windows 10
Windows 10 is a major release of Microsoft's Windows NT operating system. It is the direct successor to Windows 8.1, which was released nearly two years earlier. It was released to manufacturing on July 15, 2015, and later to retail on J ...
does not treat any of , , or as line-breaks in its default text editor,
Notepad
A notebook (also known as a notepad, writing pad, drawing pad, or legal pad) is a book or stack of paper pages that are often Ruled paper, ruled and used for purposes such as note-taking, diary, journaling or other writing, drawing, or scrapbook ...
.
*
gedit
gedit ( or ) is a text editor designed for the GNOME desktop environment. It was GNOME's default text editor and part of the GNOME Core Applications until GNOME version 42 in March 2022, which changed the default text editor to GNOME Text E ...
, the default
text editor
A text editor is a type of computer program that edits plain text. Such programs are sometimes known as "notepad" software (e.g. Windows Notepad). Text editors are provided with operating systems and software development packages, and can be us ...
of the
GNOME
A gnome is a mythological creature and diminutive spirit in Renaissance magic and alchemy, first introduced by Paracelsus in the 16th century and later adopted by more recent authors including those of modern fantasy literature. Its characte ...
desktop environment
In computing, a desktop environment (DE) is an implementation of the desktop metaphor made of a bundle of programs running on top of a computer operating system that share a common graphical user interface (GUI), sometimes described as a graphica ...
, treats and as newlines but does not for .
*
JSON
JSON (JavaScript Object Notation, pronounced ; also ) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other ser ...
allows and characters within strings, while ECMAScript prior to treated them as newlines, and therefore illegal syntax.
*
YAML
YAML ( and ) (''see '') is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted. YAML targets many of the same communications applications as Exte ...
no longer recognizes them as special as of version 1.2, in order to be compatible with
JSON
JSON (JavaScript Object Notation, pronounced ; also ) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other ser ...
.
Note well that the Unicode special characters (, ), (, ), (, ) and (, ) are
glyph
A glyph () is any kind of purposeful mark. In typography, a glyph is "the specific shape, design, or representation of a character". It is a particular graphical representation, in a particular typeface, of an element of written language. A g ...
s intended for presenting a user-visible character to the reader of the document, and are thus not recognized themselves as a newline.
In programming languages
To facilitate the creation of
portable
Portable may refer to:
General
* Portable building, a manufactured structure that is built off site and moved in upon completion of site and utility work
* Portable classroom, a temporary building installed on the grounds of a school to provide a ...
programs, programming languages provide some abstractions to deal with the different types of newline sequences used in different environments.
The
C programming language
''The C Programming Language'' (sometimes termed ''K&R'', after its authors' initials) is a computer programming book written by Brian Kernighan and Dennis Ritchie, the latter of whom originally designed and implemented the language, as well as ...
provides the
escape sequence
In computer science, an escape sequence is a combination of characters that has a meaning other than the literal characters contained therein; it is marked by one or more preceding (and possibly terminating) characters.
Examples
* In C and man ...
s (newline) and (carriage return). However, these are not required to be equivalent to the ASCII and control characters. The C standard only guarantees two things:
# Each of these escape sequences maps to a unique implementation-defined number that can be stored in a single value.
# When writing to a file, device node, or socket/fifo in ''text mode'', is transparently translated to the native newline sequence used by the system, which may be longer than one character. When reading in text mode, the native newline sequence is translated back to . In ''binary mode'', no translation is performed, and the internal representation produced by is output directly.
On Unix platforms, where C originated, the native newline sequence is ASCII (), so was simply defined to be that value. With the internal and external representation being identical, the translation performed in text mode is a
no-op
In computer science, a NOP, no-op, or NOOP (pronounced "no op"; short for no operation) is a machine language instruction and its assembly language mnemonic, programming language statement, or computer protocol command that does nothing.
Mac ...
, and Unix has no notion of text mode or binary mode. This has caused many programmers who developed their software on Unix systems simply to ignore the distinction completely, resulting in code that is not portable to different platforms.
The C library function is best avoided in binary mode because any file not written with the Unix newline convention will be misread. Also, in text mode, any file not written with the system's native newline sequence (such as a file created on a Unix system, then copied to a Windows system) will be misread as well.
Another common problem is the use of when communicating using an Internet protocol that mandates the use of ASCII + for ending lines. Writing to a text mode stream works correctly on Windows systems, but produces only on Unix, and something completely different on more exotic systems. Using in binary mode is slightly better.
Many languages, such as
C++
C++ (pronounced "C plus plus") is a high-level general-purpose programming language created by Danish computer scientist Bjarne Stroustrup as an extension of the C programming language, or "C with Classes". The language has expanded significan ...
,
Perl
Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offici ...
, and
Haskell
Haskell () is a general-purpose, statically-typed, purely functional programming language with type inference and lazy evaluation. Designed for teaching, research and industrial applications, Haskell has pioneered a number of programming lan ...
provide the same interpretation of as C. C++ has an
alternative I/O model where the manipulator can be used to output a newline (and flushes the stream buffer).
Java
Java (; id, Jawa, ; jv, ꦗꦮ; su, ) is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea to the north. With a population of 151.6 million people, Java is the world's List ...
,
PHP
PHP is a general-purpose scripting language geared toward web development. It was originally created by Danish-Canadian programmer Rasmus Lerdorf in 1993 and released in 1995. The PHP reference implementation is now produced by The PHP Group ...
, and
Python
Python may refer to:
Snakes
* Pythonidae, a family of nonvenomous snakes found in Africa, Asia, and Australia
** ''Python'' (genus), a genus of Pythonidae found in Africa and Asia
* Python (mythology), a mythical serpent
Computing
* Python (pro ...
provide the sequence (for ASCII +). In contrast to C, these are guaranteed to represent the values and , respectively.
The Java I/O libraries do not transparently translate these into platform-dependent newline sequences on input or output. Instead, they provide functions for writing a full line that automatically add the native newline sequence, and functions for reading lines that accept any of , , or + as a line terminator (se
. The method can be used to retrieve the underlying line separator.
Example:
String eol = System.lineSeparator();
String lineColor = "Color: Red" + eol;
Python permits "Universal Newline Support" when opening a file for reading, when importing modules, and when executing a file.
Some languages have created special
variables,
constants, and
subroutine
In computer programming, a function or subroutine is a sequence of program instructions that performs a specific task, packaged as a unit. This unit can then be used in programs wherever that particular task should be performed.
Functions may ...
s to facilitate newlines during program execution. In some languages such as
PHP
PHP is a general-purpose scripting language geared toward web development. It was originally created by Danish-Canadian programmer Rasmus Lerdorf in 1993 and released in 1995. The PHP reference implementation is now produced by The PHP Group ...
and
Perl
Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offici ...
,
double quotes
Quotation marks (also known as quotes, quote marks, speech marks, inverted commas, or talking marks) are punctuation marks used in pairs in various writing systems to set off direct speech, a quotation, or a phrase. The pair consists of an ...
are required to perform escape substitution for all escape sequences, including and . In PHP, to avoid portability problems, newline sequences should be issued using the PHP_EOL constant.
Example in
C#:
string eol = Environment.NewLine;
string lineColor = "Color: Red" + eol;
string eol2 = "\n";
string lineColor2 = "Color: Blue" + eol2;
Issues with different newline formats
![Newline hex 0A](https://upload.wikimedia.org/wikipedia/commons/2/27/Newline_hex_0A.png)
The different newline conventions cause text files that have been transferred between systems of different types to be displayed incorrectly.
Text in files created with programs which are common on
Unix-like
A Unix-like (sometimes referred to as UN*X or *nix) operating system is one that behaves in a manner similar to a Unix system, although not necessarily conforming to or being certified to any version of the Single UNIX Specification. A Unix-li ...
or
classic Mac OS
Mac OS (originally System Software; retronym: Classic Mac OS) is the series of operating systems developed for the Macintosh family of personal computers by Apple Computer from 1984 to 2001, starting with System 1 and ending with Mac OS 9. The ...
, appear as a single long line on most programs common to
MS-DOS
MS-DOS ( ; acronym for Microsoft Disk Operating System, also known as Microsoft DOS) is an operating system for x86-based personal computers mostly developed by Microsoft. Collectively, MS-DOS, its rebranding as IBM PC DOS, and a few ope ...
and
Microsoft Windows
Windows is a group of several proprietary graphical operating system families developed and marketed by Microsoft. Each family caters to a certain sector of the computing industry. For example, Windows NT for consumers, Windows Server for serv ...
because these do not display a single or a single as a line break.
Conversely, when viewing a file originating from a Windows computer on a Unix-like system, the extra may be displayed as a second line break, as , or as at the end of each line.
Furthermore, programs other than text editors may not accept a file, e.g. some configuration file, encoded using the foreign newline convention, as a valid file.
The problem can be hard to spot because some programs handle the foreign newlines properly while others do not. For example, a
compiler
In computing, a compiler is a computer program that translates computer code written in one programming language (the ''source'' language) into another language (the ''target'' language). The name "compiler" is primarily used for programs that ...
may fail with obscure syntax errors even though the source file looks correct when displayed on the
console
Console may refer to:
Computing and video games
* System console, a physical device to operate a computer
** Virtual console, a user interface for multiple computer consoles on one device
** Command-line interface, a method of interacting with ...
or in an
editor
Editing is the process of selecting and preparing written, photographic, visual, audible, or cinematic material used by a person or an entity to convey a message or information. The editing process can involve correction, condensation, orga ...
. Modern text editors generally recognize all flavours of + newlines and allow users to convert between the different standards.
Web browser
A web browser is application software for accessing websites. When a user requests a web page from a particular website, the browser retrieves its files from a web server and then displays the page on the user's screen. Browsers are used on ...
s are usually also capable of displaying text files and websites which use different types of newlines.
Even if a program supports different newline conventions, these features are often not sufficiently labeled, described, or documented. Typically a menu or combo-box enumerating different newline conventions will be displayed to users without an indication if the selection will re-interpret, temporarily convert, or permanently convert the newlines. Some programs will implicitly convert on open, copy, paste, or save—often inconsistently.
Most textual
Internet
The Internet (or internet) is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices. It is a '' network of networks'' that consists of private, pub ...
protocols
Protocol may refer to:
Sociology and politics
* Protocol (politics), a formal agreement between nation states
* Protocol (diplomacy), the etiquette of diplomacy and affairs of state
* Etiquette, a code of personal behavior
Science and technology
...
(including
HTTP
The Hypertext Transfer Protocol (HTTP) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, ...
,
SMTP
The Simple Mail Transfer Protocol (SMTP) is an Internet standard communication protocol for electronic mail transmission. Mail servers and other message transfer agents use SMTP to send and receive mail messages. User-level email clients typical ...
,
FTP
The File Transfer Protocol (FTP) is a standard communication protocol used for the transfer of computer files from a server to a client on a computer network. FTP is built on a client–server model architecture using separate control and data ...
,
IRC
Internet Relay Chat (IRC) is a text-based chat system for instant messaging. IRC is designed for group communication in discussion forums, called '' channels'', but also allows one-on-one communication via private messages as well as chat an ...
, and many others) mandate the use of ASCII + (, ) on the protocol level, but recommend that tolerant applications recognize lone (, ) as well. Despite the dictated standard, many applications erroneously use the
C newline escape sequence () instead of the correct combination of carriage return escape and newline escape sequences (+) (see section
Newline in programming languages above). This accidental use of the wrong escape sequences leads to problems when trying to communicate with systems adhering to the stricter interpretation of the standards instead of the suggested tolerant interpretation. One such intolerant system is the
qmail
qmail is a mail transfer agent (MTA) that runs on Unix. It was written, starting December 1995, by Daniel J. Bernstein as a more secure replacement for the popular Sendmail program. Originally license-free software, qmail's source code was ...
mail transfer agent
The mail or post is a system for physically transporting postcards, letters, and parcels. A postal service can be private or public, though many governments place restrictions on private systems. Since the mid-19th century, national postal syst ...
that actively refuses to accept messages from systems that send bare instead of the required +.
The standard Internet Message Format for email states: "CR and LF MUST only occur together as CRLF; they MUST NOT appear independently in the body".
The
File Transfer Protocol
The File Transfer Protocol (FTP) is a standard communication protocol used for the transfer of computer files from a server to a client on a computer network. FTP is built on a client–server model architecture using separate control and data ...
can automatically convert newlines in files being transferred between
systems
A system is a group of interacting or interrelated elements that act according to a set of rules to form a unified whole. A system, surrounded and influenced by its environment, is described by its boundaries, structure and purpose and express ...
with different newline representations when the transfer is done in "ASCII mode". However, transferring binary files in this mode usually has disastrous results: any occurrence of the newline byte sequence—which does not have line terminator semantics in this context, but is just part of a normal sequence of bytes—will be translated to whatever newline representation the other system uses, effectively
corrupting the file. FTP clients often employ some
heuristic
A heuristic (; ), or heuristic technique, is any approach to problem solving or self-discovery that employs a practical method that is not guaranteed to be optimal, perfect, or rational, but is nevertheless sufficient for reaching an immediate, ...
s (for example, inspection of
filename extension
A filename extension, file name extension or file extension is a suffix to the name of a computer file (e.g., .txt, .docx, .md). The extension indicates a characteristic of the file contents or its intended use. A filename extension is typically d ...
s) to automatically select either binary or ASCII mode, but in the end it is up to users to make sure their files are transferred in the correct mode. If there is any doubt as to the correct mode, binary mode should be used, as then no files will be altered by FTP, though they may display incorrectly.
Conversion between newline formats
Text editor
A text editor is a type of computer program that edits plain text. Such programs are sometimes known as "notepad" software (e.g. Windows Notepad). Text editors are provided with operating systems and software development packages, and can be us ...
s are often used for converting a text file between different newline formats; most modern editors can read and write files using at least the different ASCII / conventions.
For example, the editor
Vim can make a file compatible with the Windows Notepad text editor. Within vim
:set fileformat=dos
:wq
Editors can be unsuitable for converting larger files or bulk conversion of many files. For larger files (on Windows NT/2000/XP) the following command is often used:
D:\>TYPE unix_file , FIND /V "" > dos_file
Special purpose programs to convert files between different newline conventions include
and , and , and , and .
The command is available on virtually every
Unix-like
A Unix-like (sometimes referred to as UN*X or *nix) operating system is one that behaves in a manner similar to a Unix system, although not necessarily conforming to or being certified to any version of the Single UNIX Specification. A Unix-li ...
system and can be used to perform arbitrary replacement operations on single characters. A DOS/Windows text file can be converted to Unix format by simply removing all ASCII characters with
$
tr -d '\r' < ''inputfile'' > ''outputfile''
or, if the text has only newlines, by converting all newlines to with
$
tr '\r' '\n' < ''inputfile'' > ''outputfile''
The same tasks are sometimes performed with
awk
AWK (''awk'') is a domain-specific language designed for text processing and typically used as a data extraction and reporting tool. Like sed and grep, it is a filter, and is a standard feature of most Unix-like operating systems.
The AWK lang ...
,
sed
sed ("stream editor") is a Unix utility that parses and transforms text, using a simple, compact programming language. It was developed from 1973 to 1974 by Lee E. McMahon of Bell Labs,
and is available today for most operating systems.
sed w ...
, or in
Perl
Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. "Perl" refers to Perl 5, but from 2000 to 2019 it also referred to its redesigned "sister language", Perl 6, before the latter's name was offici ...
if the platform has a Perl interpreter:
$ awk '' inputfile > outputfile # UNIX to DOS (adding CRs on Linux and BSD based OS that haven't GNU extensions)
$ awk '' inputfile > outputfile # DOS to UNIX (removing CRs on Linux and BSD based OS that haven't GNU extensions)
$ sed -e 's/$/\r/' inputfile > outputfile # UNIX to DOS (adding CRs on Linux based OS that use GNU extensions)
$ sed -e 's/\r$//' inputfile > outputfile # DOS to UNIX (removing CRs on Linux based OS that use GNU extensions)
$ perl -pe 's/\r?\n, \r/\r\n/g' inputfile > outputfile # Convert to DOS
$ perl -pe 's/\r?\n, \r/\n/g' inputfile > outputfile # Convert to UNIX
$ perl -pe 's/\r?\n, \r/\r/g' inputfile > outputfile # Convert to old Mac
The command can identify the type of line endings:
$ file myfile.txt
myfile.txt: ASCII English text, with CRLF line terminators
The Unix
egrep
grep is a command-line utility for searching plain-text data sets for lines that match a regular expression. Its name comes from the ed command ''g/re/p'' (''globally search for a regular expression and print matching lines''), which has the sam ...
(extended grep) command can be used to print filenames of Unix or DOS files (assuming Unix and DOS-style files only, no classic Mac OS-style files):
$ egrep -L '\r\n' myfile.txt # show UNIX style file (LF terminated)
$ egrep -l '\r\n' myfile.txt # show DOS style file (CRLF terminated)
Other tools permit the user to visualise the EOL characters:
$ od -a myfile.txt
$ cat -e myfile.txt
$ cat -v myfile.txt
$ hexdump -c myfile.txt
Interpretation
Two ways to view newlines, both of which are
self-consistent
In classical deductive logic, a consistent theory is one that does not lead to a logical contradiction. The lack of contradiction can be defined in either semantic or syntactic terms. The semantic definition states that a theory is consistent ...
, are that newlines either ''separate'' lines or that they ''terminate'' lines. If a newline is considered a separator, there will be no newline after the last line of a file. Some programs have problems processing the last line of a file if it is not terminated by a newline. On the other hand, programs that expect newline to be used as a separator will interpret a final newline as starting a new (empty) line. Conversely, if a newline is considered a terminator, all text lines including the last are expected to be terminated by a newline. If the final character sequence in a text file is not a newline, the final line of the file may be considered to be an improper or incomplete text line, or the file may be considered to be improperly truncated.
In text intended primarily to be read by humans using software which implements the
word wrap
Line breaking, also known as word wrapping, is breaking a section of text into lines so that it will fit into the available width of a page, window or other display area. In text display, line wrap is continuing on a new line when a line is ful ...
feature, a newline character typically only needs to be stored if a line break is required independent of whether the next word would fit on the same line, such as between
paragraph
A paragraph () is a self-contained unit of discourse in writing dealing with a particular point or idea. Though not required by the orthographic conventions of any language with a writing system, paragraphs are a conventional means of organizing e ...
s and in vertical lists. Therefore, in the logic of
word processing
A word is a basic element of language that carries an objective or practical meaning, can be used on its own, and is uninterruptible. Despite the fact that language speakers often have an intuitive grasp of what a word is, there is no consen ...
and most
text editor
A text editor is a type of computer program that edits plain text. Such programs are sometimes known as "notepad" software (e.g. Windows Notepad). Text editors are provided with operating systems and software development packages, and can be us ...
s, newline is used as a ''paragraph break'' and is known as a "hard return", in contrast to "soft returns" which are dynamically created to implement word wrapping and are changeable with each display instance. In many applications a separate
control character
In computing and telecommunication, a control Character (computing), character or non-printing character (NPC) is a code point (a number) in a character encoding, character set, that does not represent a written symbol. They are used as in-band ...
called "manual line break" exists for forcing line breaks inside a single paragraph. The
glyph
A glyph () is any kind of purposeful mark. In typography, a glyph is "the specific shape, design, or representation of a character". It is a particular graphical representation, in a particular typeface, of an element of written language. A g ...
for the control character for a hard return is usually a
pilcrow
The pilcrow, ¶, is a handwritten or typographical character used to identify a paragraph. It is also called the paragraph mark (or sign or symbol), paraph, or blind P.
The pilcrow may be used at the start of separate paragraphs or t ...
(¶), and for the manual line break is usually a carriage return arrow (↵).
Reverse and partial line feeds
, (
U+008D REVERSE LINE FEED,
ISO/IEC 6429
ISO/IEC JTC 1, entitled "Information technology", is a joint technical committee (JTC) of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). Its purpose is to develop, maintain and pr ...
8D, decimal 141) is used to move the printing position back one line (by reverse feeding the paper, or by moving a display cursor up one line) so that other characters may be printed over existing text. This may be done to make them bolder, or to add underlines, strike-throughs or other characters such as
diacritics
A diacritic (also diacritical mark, diacritical point, diacritical sign, or accent) is a glyph added to a letter or to a basic glyph. The term derives from the Ancient Greek (, "distinguishing"), from (, "to distinguish"). The word ''diacritic ...
.
Similarly, (
U+008B PARTIAL LINE FORWARD, decimal 139) and (
U+008C PARTIAL LINE BACKWARD, decimal 140) can be used to advance or reverse the text printing position by some fraction of the vertical line spacing (typically, half). These can be used in combination for subscripts (by advancing and then reversing) and superscripts (by reversing and then advancing), and may also be useful for printing diacritics.
See also
*
ASA carriage control characters ASA control characters are simple printing command characters used to control the movement of paper through line printers. These commands are presented as special characters in the first column of each text line to be printed, and affect how the pa ...
*
C0 and C1 control codes
The C0 and C1 control code or control character sets define control codes for use in text by computer systems that use ASCII and derivatives of ASCII. The codes represent additional information about the text, such as the position of a cursor, ...
*
End-of-file
In computing, end-of-file (EOF) is a condition in a computer operating system where no more data can be read from a data source. The data source is usually called a file or stream.
Details
In the C standard library, the character reading functio ...
*
Line starve
*
Page break
A page break is a marker in an electronic document that tells the document interpreter that the content which follows is part of a new page. A page break causes a form feed to be sent to the printer during spooling of the document to the printer. ...
*
Carriage return
A carriage return, sometimes known as a cartridge return and often shortened to CR, or return, is a control character or mechanism used to reset a device's position to the beginning of a line of text. It is closely associated with the line feed a ...
*
Enter key
On computer keyboards, the enter key and return key are two closely related keys with overlapping and distinct functions dependent on operating system and application.
Functions
The return key has its origins in two typewriter functions: ca ...
References
External links
*The Unicode reference, see paragraph 5.8 i
Chapter 5of the Unicode 4.0 standard (PDF)
*
*{{webarchive, url=https://web.archive.org/web/20060820133536/http://onlamp.com/pub/a/onlamp/2006/08/17/understanding-newlines.html, title=Understanding Newlines, date=20 August 2006
"The End-of-Line Story"
Control characters
Whitespace