A bigram or digram is a sequence of two adjacent elements from a
string of
tokens, which are typically letters, syllables, or words. A bigram is an
''n''-gram for ''n''=2. The frequency distribution of every bigram in a string is commonly used for simple statistical analysis of text in many applications, including in computational linguistics, cryptography, speech recognition, and so on.
''Gappy bigrams'' or ''skipping bigrams'' are word pairs which allow gaps (perhaps avoiding connecting words, or allowing some simulation of dependencies, as in a
dependency grammar
Dependency grammar (DG) is a class of modern grammatical theories that are all based on the dependency relation (as opposed to the ''constituency relation'' of phrase structure) and that can be traced back primarily to the work of Lucien TesniĂ ...
).
''Head word bigrams'' are gappy bigrams with an explicit dependency relationship.
Details
Bigrams help provide the conditional probability of a token given the preceding token, when the relation of the
conditional probability
In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occu ...
is applied:
That is, the probability
of a token
given the preceding token
is equal to the probability of their bigram, or the co-occurrence of the two tokens
, divided by the probability of the preceding token.
Applications
Bigrams are used in most successful
language model
A language model is a probability distribution over sequences of words. Given any sequence of words of length , a language model assigns a probability P(w_1,\ldots,w_m) to the whole sequence. Language models generate probabilities by training on ...
s for
speech recognition
Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the ...
. They are a special case of
N-gram
In the fields of computational linguistics and probability, an ''n''-gram (sometimes also called Q-gram) is a contiguous sequence of ''n'' items from a given sample of text or speech. The items can be phonemes, syllables, letters, words or ...
.
Bigram frequency attacks can be used in
cryptography
Cryptography, or cryptology (from grc, , translit=kryptĂłs "hidden, secret"; and ''graphein'', "to write", or '' -logia'', "study", respectively), is the practice and study of techniques for secure communication in the presence of adv ...
to solve
cryptograms. See
frequency analysis
In cryptanalysis, frequency analysis (also known as counting letters) is the study of the frequency of letters or groups of letters in a ciphertext. The method is used as an aid to breaking classical ciphers.
Frequency analysis is based on ...
.
Bigram frequency is one approach to
statistical language identification.
Some activities in
logology or recreational linguistics involve bigrams. These include attempts to find English words beginning with every possible bigram, or words containing a string of repeated bigrams, such as ''logogogue''.
Bigram frequency in the English language
The frequency of the most common letter bigrams, rounded to the closest centesimal, in a large English corpus is:
th 3.56% of 1.17% io 0.83%
he 3.07% ed 1.17% le 0.83%
in 2.43% is 1.13% ve 0.83%
er 2.05% it 1.12% co 0.79%
an 1.99% al 1.09% me 0.79%
re 1.85% ar 1.07% de 0.76%
on 1.76% st 1.05% hi 0.76%
at 1.49% to 1.05% ri 0.73%
en 1.45% nt 1.04% ro 0.73%
nd 1.35% ng 0.95% ic 0.70%
ti 1.34% se 0.93% ne 0.69%
es 1.34% ha 0.93% ea 0.69%
or 1.28% as 0.87% ra 0.69%
te 1.20% ou 0.87% ce 0.65%
Bigram frequencies for a different corpus is available.
See also
*
Digraph (orthography)
A digraph or digram (from the grc, δίς , "double" and , "to write") is a pair of characters used in the orthography of a language to write either a single phoneme (distinct sound), or a sequence of phonemes that does not correspond to ...
*
N-gram
In the fields of computational linguistics and probability, an ''n''-gram (sometimes also called Q-gram) is a contiguous sequence of ''n'' items from a given sample of text or speech. The items can be phonemes, syllables, letters, words or ...
*
Letter frequency
Letter frequency is the number of times letters of the alphabet appear on average in written language. Letter frequency analysis dates back to the Arab mathematician Al-Kindi (c. 801–873 AD), who formally developed the method to brea ...
*
Sørensen–Dice coefficient
References
{{Natural Language Processing
Formal languages
Classical cryptography
Natural language processing