The Brown University Standard Corpus of Present-Day American English (or just Brown Corpus) is an electronic collection of text samples of American English, the first major structured

corpus Corpus is Latin for "body". It may refer to: Linguistics * Text corpus, in linguistics, a large and structured set of texts * Speech corpus, in linguistics, a large set of speech audio files * Corpus linguistics, a branch of linguistics Music * ...

of varied genres. This corpus first set the bar for the scientific study of the frequency and distribution of word categories in everyday language use. Compiled by

Henry Kučera Henry Kučera (15 February 1925 – 20 February 2010), born Jindřich Kučera () was a Czech-American linguist who pioneered corpus linguistics, linguistic software, a major contributor to the ''American Heritage Dictionary'', and a pioneer in ...

and

W. Nelson Francis W. Nelson Francis (October 23, 1910 – June 14, 2002) was an American author, linguist, and university professor. He served as a member of the faculties of Franklin & Marshall College and Brown University, where he specialized in Engl ...

Brown University Brown University is a private research university in Providence, Rhode Island. Brown is the seventh-oldest institution of higher education in the United States, founded in 1764 as the College in the English Colony of Rhode Island and Providenc ...

, in

Rhode Island Rhode Island (, like ''road'') is a U.S. state, state in the New England region of the Northeastern United States. It is the List of U.S. states by area, smallest U.S. state by area and the List of states and territories of the United States ...

, it is a general language corpus containing 500 samples of English, totaling roughly one million words, compiled from works published in the United States in 1961.

History

In 1967, Kučera and Francis published their classic work ''Computational Analysis of Present-Day American English'', which provided basic statistics on what is known today simply as the ''Brown Corpus''. The Brown Corpus was a carefully compiled selection of current American English, totalling about a million words drawn from a wide variety of sources. Kučera and Francis subjected it to a variety of computational analyses, from which they compiled a rich and variegated opus, combining elements of linguistics, psychology, statistics, and sociology. It has been very widely used in computational linguistics, and was for many years among the most-cited resources in the field. Shortly after publication of the first

lexicostatistical Lexicostatistics is a method of comparative linguistics that involves comparing the percentage of lexical cognates between languages to determine their relationship. Lexicostatistics is related to the comparative method but does not reconstruct a p ...

analysis,

Boston Boston (), officially the City of Boston, is the state capital and most populous city of the Commonwealth of Massachusetts, as well as the cultural and financial center of the New England region of the United States. It is the 24th- mo ...

publisher Houghton-Mifflin approached Kučera to supply a million word, three-line citation base for its new ''

American Heritage Dictionary American(s) may refer to: * American, something of, from, or related to the United States of America, commonly known as the "United States" or "America" ** Americans, citizens and nationals of the United States of America ** American ancestry, pe ...

''. This ground-breaking new dictionary, which first appeared in 1969, was the first dictionary to be compiled using corpus linguistics for word frequency and other information. The initial Brown Corpus had only the words themselves, plus a location identifier for each. Over the following several years part-of-speech tags were applied. The Greene and Rubin tagging program (see under

part of speech tagging In corpus linguistics, part-of-speech tagging (POS tagging or PoS tagging or POST), also called grammatical tagging is the process of marking up a word in a text (corpus) as corresponding to a particular part of speech, based on both its definit ...

) helped considerably in this, but the high error rate meant that extensive manual proofreading was required. The tagged Brown Corpus used a selection of about 80 parts of speech, as well as special indicators for compound forms, contractions, foreign words and a few other phenomena, and formed the model for many later corpora such as the

Lancaster-Oslo-Bergen Corpus The Lancaster-Oslo/Bergen (LOB) Corpus is a million-word collection of British English texts which was compiled in the 1970s in collaboration between the University of Lancaster, the University of Oslo, and the Norwegian Computing Centre for the ...

(British English from the early 1990s) and the Freiburg-Brown Corpus of American English (FROWN) (American English from the early 1990s). Tagging the corpus enabled far more sophisticated statistical analysis, such as the work programmed by Andrew Mackie, and documented in books on English grammar. One interesting result is that even for quite large samples, graphing words in order of decreasing frequency of occurrence shows a

hyperbola In mathematics, a hyperbola (; pl. hyperbolas or hyperbolae ; adj. hyperbolic ) is a type of smooth curve lying in a plane, defined by its geometric properties or by equations for which it is the solution set. A hyperbola has two pieces, cal ...

: the frequency of the ''n''-th most frequent word is roughly proportional to 1/''n''. Thus "the" constitutes nearly 7% of the Brown Corpus, "to" and "of" more than another 3% each; while about half the total vocabulary of about 50,000 words are '' hapax legomena'': words that occur only once in the corpus.Kirsten Malmkjær,
The Linguistics Encyclopedia
', 2nd ed, Routledge, 2002, , p. 87. This simple rank-vs.-frequency relationship was noted for an extraordinary variety of phenomena by

George Kingsley Zipf George Kingsley Zipf (; January 7, 1902 – September 25, 1950), was an American linguist and philologist who studied statistical occurrences in different languages.. Zipf earned his bachelors, masters, and doctoral degrees from Harvard Universi ...

(for example, see his ''The Psychobiology of Language''), and is known as Zipf's law. Although the Brown Corpus pioneered the field of corpus linguistics, by now typical corpora (such as the

Corpus of Contemporary American English The Corpus of Contemporary American English (COCA) is a one-billion-word corpus of contemporary American English. It was created by Mark Davies, retired professor of corpus linguistics at Brigham Young University (BYU). Content The Corpus of C ...

, the

British National Corpus The British National Corpus (BNC) is a 100-million-word text corpus of samples of written and spoken English from a wide range of sources. The corpus covers British English of the late 20th century from a wide variety of genres, with the intention ...

or the

International Corpus of English The International Corpus of English (ICE) is a set of corpora representing varieties of English from around the world. Over twenty countries or groups of countries where English is the first language or an official second language are included. His ...

) tend to be much larger, on the order of 100 million words.

Sample distribution

The Corpus consists of 500 samples, distributed across 15 genres in rough proportion to the amount published in 1961 in each of those genres. All works sampled were published in 1961; as far as could be determined they were ''first'' published then, and were written by native speakers of American English. Each sample began at a random sentence-boundary in the article or other unit chosen, and continued up to the first sentence boundary after 2,000 words. In a very few cases miscounts led to samples being just under 2,000 words. The original data entry was done on upper-case only

keypunch A keypunch is a device for precisely punching holes into stiff paper cards at specific locations as determined by keys struck by a human operator. Other devices included here for that same function include the gang punch, the pantograph punch, ...

machines; capitals were indicated by a preceding asterisk, and various special items such as formulae also had special codes. The corpus originally (1961) contained 1,014,312 words sampled from 15 text categories: * A. PRESS: Reportage (''44 texts'') ** Political ** Sports ** Society ** Spot News ** Financial ** Cultural * B. PRESS: Editorial (''27 texts'') ** Institutional Daily ** Personal ** Letters to the Editor * C. PRESS: Reviews (''17 texts'') ** ''theatre'' ** ''books'' ** ''music'' ** ''dance'' * D. RELIGION (''17 texts'') ** Books ** Periodicals ** Tracts * E. SKILL AND HOBBIES (''36 texts'') ** Books ** Periodicals * F. POPULAR LORE (''48 texts'') ** Books ** Periodicals * G. BELLES-LETTRES - Biography, Memoirs, etc. (''75 texts'') ** Books ** Periodicals * H. MISCELLANEOUS: US Government & House Organs (''30 texts'') ** Government Documents ** Foundation Reports ** Industry Reports ** College Catalog ** Industry House organ * J. LEARNED (''80 texts'') ** Natural Sciences ** Medicine ** Mathematics ** Social and Behavioral Sciences ** Political Science, Law, Education ** Humanities ** Technology and Engineering * K. FICTION: General (''29 texts'') ** Novels ** Short Stories * L. FICTION: Mystery and Detective Fiction (''24 texts'') ** Novels ** Short Stories * M. FICTION: Science (''6 texts'') ** Novels ** Short Stories * N. FICTION: Adventure and Western (''29 texts'') ** Novels ** Short Stories * P. FICTION: Romance and Love Story (''29 texts'') ** Novels ** Short Stories * R. HUMOR (''9 texts'') ** Novels ** Essays, etc.

Part-of-speech tags used

References

External links

Brown Corpus Manual

History

Sample distribution

Part-of-speech tags used

See also

References

External links