HOME

TheInfoList



OR:

''The Art of Computer Programming'' (''TAOCP'') is a comprehensive
monograph A monograph is a specialist work of writing (in contrast to reference works) or exhibition on a single subject or an aspect of a subject, often by a single author or artist, and usually on a scholarly subject. In library cataloging, ''monogra ...
written by the computer scientist
Donald Knuth Donald Ervin Knuth ( ; born January 10, 1938) is an American computer scientist, mathematician, and professor emeritus at Stanford University. He is the 1974 recipient of the ACM Turing Award, informally considered the Nobel Prize of computer ...
presenting programming
algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
s and their analysis. Volumes 1–5 are intended to represent the central core of computer programming for sequential machines. When Knuth began the project in 1962, he originally conceived of it as a single book with twelve chapters. The first three volumes of what was then expected to be a seven-volume set were published in 1968, 1969, and 1973. Work began in earnest on Volume 4 in 1973, but was suspended in 1977 for work on typesetting prompted by the second edition of Volume 2. Writing of the final copy of Volume 4A began in longhand in 2001, and the first online pre-fascicle, 2A, appeared later in 2001. The first published installment of Volume 4 appeared in paperback as Fascicle 2 in 2005. The hardback Volume 4A, combining Volume 4, Fascicles 0–4, was published in 2011. Volume 4, Fascicle 6 ("Satisfiability") was released in December 2015; Volume 4, Fascicle 5 ("Mathematical Preliminaries Redux; Backtracking; Dancing Links") was released in November 2019. Volume 4B consists of material evolved from Fascicles 5 and 6. The manuscript was sent to the publisher on August 1, 2022 and the volume was published in September 2022. Fascicle 7, planned for Volume 4C, was the subject of Knuth's talk on August 3, 2022.


History

After winning a Westinghouse Talent Search scholarship, Knuth enrolled at the Case Institute of Technology (now
Case Western Reserve University Case Western Reserve University (CWRU) is a Private university, private research university in Cleveland, Cleveland, Ohio. Case Western Reserve was established in 1967, when Western Reserve University, founded in 1826 and named for its location i ...
), where his performance was so outstanding that the faculty voted to award him a
master of science A Master of Science ( la, Magisterii Scientiae; abbreviated MS, M.S., MSc, M.Sc., SM, S.M., ScM or Sc.M.) is a master's degree in the field of science awarded by universities in many countries or a person holding such a degree. In contrast t ...
upon his completion of the
bachelor degree A bachelor's degree (from Middle Latin ''baccalaureus'') or baccalaureate (from New Latin, Modern Latin ''baccalaureatus'') is an Undergraduate degree, undergraduate academic degree awarded by colleges and universities upon completion of a course ...
. During his summer vacations, Knuth was hired by the
Burroughs Corporation The Burroughs Corporation was a major American manufacturer of business equipment. The company was founded in 1886 as the American Arithmometer Company. In 1986, it merged with Sperry UNIVAC to form Unisys. The company's history paralleled many ...
to write
compiler In computing, a compiler is a computer program that translates computer code written in one programming language (the ''source'' language) into another language (the ''target'' language). The name "compiler" is primarily used for programs tha ...
s, earning more in his summer months than full professors did for an entire year. Such exploits made Knuth a topic of discussion among the mathematics department, which included Richard S. Varga. In January 1962, when he was a graduate student in the mathematics department at Caltech, Knuth was approached by
Addison-Wesley Addison-Wesley is an American publisher of textbooks and computer literature. It is an imprint of Pearson PLC, a global publishing and education company. In addition to publishing books, Addison-Wesley also distributes its technical titles throug ...
to write a book about compiler design, and he proposed a larger scope. He came up with a list of twelve chapter titles the same day. In the summer of 1962 he worked on a FORTRAN compiler for
UNIVAC UNIVAC (Universal Automatic Computer) was a line of electronic digital stored-program computers starting with the products of the Eckert–Mauchly Computer Corporation. Later the name was applied to a division of the Remington Rand company an ...
. During this time, he also came up with a mathematical analysis of
linear probing Linear probing is a scheme in computer programming for resolving collisions in hash tables, data structures for maintaining a collection of key–value pairs and looking up the value associated with a given key. It was invented in 1954 by Gene ...
, which convinced him to present the material with a quantitative approach. After receiving his Ph.D. in June 1963, he began working on his manuscript, of which he finished his first draft in June 1965, at hand-written pages. He had assumed that about five hand-written pages would translate into one printed page, but his publisher said instead that about hand-written pages translated to one printed page. This meant he had approximately printed pages of material, which closely matches the size of the first three published volumes. At this point, Knuth received support from Richard S. Varga, who was the scientific adviser to the publisher. Varga was visiting
Olga Taussky-Todd Olga Taussky-Todd (August 30, 1906, Olomouc, Austria-Hungary (present-day Olomouc, Czech Republic) – October 7, 1995, Pasadena, California) was an Austrian and later Czech-American mathematician. She published more than 300 research papers ...
and
John Todd John Todd or Tod may refer to: Clergy * John Todd (abolitionist) (1818–1894), preacher and 'conductor' on the Underground Railroad * John Todd (author) (1800–1873), American minister and author * John Todd (bishop), Anglican bishop in the early ...
at
Caltech The California Institute of Technology (branded as Caltech or CIT)The university itself only spells its short form as "Caltech"; the institution considers other spellings such a"Cal Tech" and "CalTech" incorrect. The institute is also occasional ...
. With Varga's enthusiastic endorsement, the publisher accepted Knuth's expanded plans. In its expanded version, the book would be published in seven volumes, each with just one or two chapters. Due to the growth in Chapter 7, which was fewer than 100 pages of the 1965 manuscript, per Vol. 4A p. vi, the plan for Volume 4 has since expanded to include Volumes 4A, 4B, 4C, 4D, and possibly more. In 1976, Knuth prepared a second edition of Volume 2, requiring it to be
typeset Typesetting is the composition of text by means of arranging physical ''type'' (or ''sort'') in mechanical systems or ''glyphs'' in digital systems representing ''characters'' (letters and other symbols).Dictionary.com Unabridged. Random H ...
again, but the style of type used in the first edition (called
hot type ''Hot Type'' was a Canadian television series, which aired weekly on CBC Newsworld. Hosted by Evan Solomon, the program was a cultural talk and interview show focused primarily on books and literature."TV takes new look at the printed word". ''Th ...
) was no longer available. In 1977, he decided to spend some time creating something more suitable. Eight years later, he returned with TEX, which is currently used for all volumes. The offer of a so-called
Knuth reward check Knuth reward checks are checks or check-like certificates awarded by computer scientist Donald Knuth for finding technical, typographical, or historical errors, or making substantial suggestions for his publications. The ''MIT Technology Review'' ...
worth "one hexadecimal dollar" (100 HEX
base 16 In mathematics and computing, the hexadecimal (also base-16 or simply hex) numeral system is a positional numeral system that represents numbers using a radix (base) of 16. Unlike the decimal system representing numbers using 10 symbols, hexad ...
cents, in
decimal The decimal numeral system (also called the base-ten positional numeral system and denary or decanary) is the standard system for denoting integer and non-integer numbers. It is the extension to non-integer numbers of the Hindu–Arabic numer ...
, is $2.56) for any errors found, and the correction of these errors in subsequent printings, has contributed to the highly polished and still-authoritative nature of the work, long after its first publication. Another characteristic of the volumes is the variation in the difficulty of the exercises. Knuth even has a numerical difficulty scale for rating those exercises, varying from 0 to 50, where 0 is trivial, and 50 is an open question in contemporary research. Knuth's dedication reads:
This series of books is affectionately dedicated
to the Type 650 computer once installed at
Case Institute of Technology Case Western Reserve University (CWRU) is a private research university in Cleveland, Ohio. Case Western Reserve was established in 1967, when Western Reserve University, founded in 1826 and named for its location in the Connecticut Western Rese ...
,
with whom I have spent many pleasant evenings.The dedication was worded slightly differently in the first edition.


Assembly language in the book

All examples in the books use a language called " MIX assembly language", which runs on the hypothetical MIX computer. Currently, the MIX computer is being replaced by the
MMIX MMIX (pronounced ''em-mix'') is a 64-bit reduced instruction set computing (RISC) architecture designed by Donald Knuth, with significant contributions by John L. Hennessy (who contributed to the design of the MIPS architecture) and Richard L. ...
computer, which is a
RISC In computer engineering, a reduced instruction set computer (RISC) is a computer designed to simplify the individual instructions given to the computer to accomplish tasks. Compared to the instructions given to a complex instruction set comput ...
version. Software such as GNU MDK exists to provide emulation of the MIX architecture. Knuth considers the use of
assembly language In computer programming, assembly language (or assembler language, or symbolic machine code), often referred to simply as Assembly and commonly abbreviated as ASM or asm, is any low-level programming language with a very strong correspondence b ...
necessary for the speed and memory usage of algorithms to be judged.


Critical response

Knuth was awarded the 1974
Turing Award The ACM A. M. Turing Award is an annual prize given by the Association for Computing Machinery (ACM) for contributions of lasting and major technical importance to computer science. It is generally recognized as the highest distinction in comput ...
"for his major contributions to the
analysis of algorithms In computer science, the analysis of algorithms is the process of finding the computational complexity of algorithms—the amount of time, storage, or other resources needed to execute them. Usually, this involves determining a function that re ...
�� and in particular for his contributions to the 'art of computer programming' through his well-known books in a continuous series by this title." ''
American Scientist __NOTOC__ ''American Scientist'' (informally abbreviated ''AmSci'') is an American bimonthly science and technology magazine published since 1913 by Sigma Xi, The Scientific Research Society. In the beginning of 2000s the headquarters was in New ...
'' has included this work among "100 or so Books that shaped a Century of Science", referring to the twentieth century, Covers of the third edition of Volume 1 quote
Bill Gates William Henry Gates III (born October 28, 1955) is an American business magnate and philanthropist. He is a co-founder of Microsoft, along with his late childhood friend Paul Allen. During his career at Microsoft, Gates held the positions ...
as saying, "If you think you're a really good programmer… read (Knuth's) ''Art of Computer Programming''… You should definitely send me a résumé if you can read the whole thing." ''
The New York Times ''The New York Times'' (''the Times'', ''NYT'', or the Gray Lady) is a daily newspaper based in New York City with a worldwide readership reported in 2020 to comprise a declining 840,000 paid print subscribers, and a growing 6 million paid ...
'' referred to it as "the profession's defining treatise".


Volumes


Completed

* Volume 1 â€“ Fundamental Algorithms ** Chapter 1 â€“ Basic concepts ** Chapter 2 â€“ Information
structures A structure is an arrangement and organization of interrelated elements in a material object or system, or the object or system so organized. Material structures include man-made objects such as buildings and machines and natural objects such a ...
* Volume 2 â€“ Seminumerical Algorithms ** Chapter 3 â€“ Random numbers ** Chapter 4 â€“
Arithmetic Arithmetic () is an elementary part of mathematics that consists of the study of the properties of the traditional operations on numbers—addition, subtraction, multiplication, division, exponentiation, and extraction of roots. In the 19th ce ...
* Volume 3 â€“
Sorting Sorting refers to ordering data in an increasing or decreasing manner according to some linear relationship among the data items. # ordering: arranging items in a sequence ordered by some criterion; # categorizing: grouping items with similar p ...
and
Searching Searching or search may refer to: Computing technology * Search algorithm, including keyword search ** :Search algorithms * Search and optimization for problem solving in artificial intelligence * Search engine technology, software for findi ...
** Chapter 5 â€“
Sorting Sorting refers to ordering data in an increasing or decreasing manner according to some linear relationship among the data items. # ordering: arranging items in a sequence ordered by some criterion; # categorizing: grouping items with similar p ...
** Chapter 6 â€“
Searching Searching or search may refer to: Computing technology * Search algorithm, including keyword search ** :Search algorithms * Search and optimization for problem solving in artificial intelligence * Search engine technology, software for findi ...
* Volume 4A â€“
Combinatorial Combinatorics is an area of mathematics primarily concerned with counting, both as a means and an end in obtaining results, and certain properties of finite structures. It is closely related to many other areas of mathematics and has many app ...
Algorithms ** Chapter 7 â€“ Combinatorial searching (part 1) * Volume 4B â€“
Combinatorial Combinatorics is an area of mathematics primarily concerned with counting, both as a means and an end in obtaining results, and certain properties of finite structures. It is closely related to many other areas of mathematics and has many app ...
Algorithms ** Chapter 7 â€“ Combinatorial searching (part 2)


Planned

* Volume 4C... â€“ Combinatorial Algorithms (chapters 7 & 8 released in several subvolumes) ** Chapter 7 â€“ Combinatorial searching (continued) ** Chapter 8 â€“
Recursion Recursion (adjective: ''recursive'') occurs when a thing is defined in terms of itself or of its type. Recursion is used in a variety of disciplines ranging from linguistics to logic. The most common application of recursion is in mathematics ...
* Volume 5 â€“ Syntactic Algorithms ** Chapter 9 â€“ Lexical scanning (also includes
string search In computer science, string-searching algorithms, sometimes called string-matching algorithms, are an important class of string algorithms that try to find a place where one or several strings (also called patterns) are found within a larger str ...
and
data compression In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compressio ...
) ** Chapter 10 â€“
Parsing Parsing, syntax analysis, or syntactic analysis is the process of analyzing a string of symbols, either in natural language, computer languages or data structures, conforming to the rules of a formal grammar. The term ''parsing'' comes from Lati ...
techniques * Volume 6 â€“ The Theory of
Context-Free Languages In formal language theory, a context-free language (CFL) is a language generated by a context-free grammar (CFG). Context-free languages have many applications in programming languages, in particular, most arithmetic expressions are generated by ...
* Volume 7 â€“
Compiler In computing, a compiler is a computer program that translates computer code written in one programming language (the ''source'' language) into another language (the ''target'' language). The name "compiler" is primarily used for programs tha ...
Techniques


Chapter outlines


Completed


Volume 1 â€“ Fundamental Algorithms

* Chapter 1 â€“ Basic concepts ** 1.1.
Algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
s ** 1.2. Mathematical Preliminaries *** 1.2.1.
Mathematical Induction Mathematical induction is a method for proving that a statement ''P''(''n'') is true for every natural number ''n'', that is, that the infinitely many cases ''P''(0), ''P''(1), ''P''(2), ''P''(3), ...  all hold. Informal metaphors help ...
*** 1.2.2. Numbers, Powers, and
Logarithms In mathematics, the logarithm is the inverse function to exponentiation. That means the logarithm of a number  to the base  is the exponent to which must be raised, to produce . For example, since , the ''logarithm base'' 10 o ...
*** 1.2.3. Sums and Products *** 1.2.4. Integer Functions and Elementary
Number Theory Number theory (or arithmetic or higher arithmetic in older usage) is a branch of pure mathematics devoted primarily to the study of the integers and integer-valued functions. German mathematician Carl Friedrich Gauss (1777–1855) said, "Mathe ...
*** 1.2.5.
Permutation In mathematics, a permutation of a set is, loosely speaking, an arrangement of its members into a sequence or linear order, or if the set is already ordered, a rearrangement of its elements. The word "permutation" also refers to the act or pr ...
s and
Factorial In mathematics, the factorial of a non-negative denoted is the product of all positive integers less than or equal The factorial also equals the product of n with the next smaller factorial: \begin n! &= n \times (n-1) \times (n-2) ...
s *** 1.2.6.
Binomial Coefficients In mathematics, the binomial coefficients are the positive integers that occur as coefficients in the binomial theorem. Commonly, a binomial coefficient is indexed by a pair of integers and is written \tbinom. It is the coefficient of the te ...
*** 1.2.7.
Harmonic Numbers In mathematics, the -th harmonic number is the sum of the reciprocals of the first natural numbers: H_n= 1+\frac+\frac+\cdots+\frac =\sum_^n \frac. Starting from , the sequence of harmonic numbers begins: 1, \frac, \frac, \frac, \frac, \dot ...
*** 1.2.8.
Fibonacci Numbers In mathematics, the Fibonacci numbers, commonly denoted , form a sequence, the Fibonacci sequence, in which each number is the sum of the two preceding ones. The sequence commonly starts from 0 and 1, although some authors start the sequence from ...
*** 1.2.9.
Generating Functions In mathematics, a generating function is a way of encoding an infinite sequence of numbers () by treating them as the coefficients of a formal power series. This series is called the generating function of the sequence. Unlike an ordinary series ...
*** 1.2.10. Analysis of an Algorithm *** 1.2.11. Asymptotic Representations **** 1.2.11.1. The O-notation **** 1.2.11.2. Euler's summation formula **** 1.2.11.3. Some asymptotic calculations ** 1.3
MMIX MMIX (pronounced ''em-mix'') is a 64-bit reduced instruction set computing (RISC) architecture designed by Donald Knuth, with significant contributions by John L. Hennessy (who contributed to the design of the MIPS architecture) and Richard L. ...
( MIX in the hardback copy but updated by fascicle 1) *** 1.3.1. Description of MMIX *** 1.3.2. The MMIX Assembly Language *** 1.3.3. Applications to
Permutations In mathematics, a permutation of a set is, loosely speaking, an arrangement of its members into a sequence or linear order, or if the set is already ordered, a rearrangement of its elements. The word "permutation" also refers to the act or pro ...
** 1.4. Some Fundamental Programming Techniques *** 1.4.1.
Subroutines In computer programming, a function or subroutine is a sequence of program instructions that performs a specific task, packaged as a unit. This unit can then be used in programs wherever that particular task should be performed. Functions may ...
*** 1.4.2.
Coroutine Coroutines are computer program components that generalize subroutines for non-preemptive multitasking, by allowing execution to be suspended and resumed. Coroutines are well-suited for implementing familiar program components such as cooperativ ...
s *** 1.4.3. Interpretive Routines **** 1.4.3.1. A MIX simulator **** 1.4.3.2. Trace routines *** 1.4.4.
Input and Output In computing, input/output (I/O, or informally io or IO) is the communication between an information processing system, such as a computer, and the outside world, possibly a human or another information processing system. Inputs are the signals ...
*** 1.4.5. History and Bibliography * Chapter 2 â€“ Information Structures ** 2.1. Introduction ** 2.2. Linear Lists *** 2.2.1. Stacks, Queues, and Deques *** 2.2.2. Sequential Allocation *** 2.2.3. Linked Allocation (
topological sorting In computer science, a topological sort or topological ordering of a directed graph is a linear ordering of its vertices such that for every directed edge ''uv'' from vertex ''u'' to vertex ''v'', ''u'' comes before ''v'' in the ordering. For ins ...
) *** 2.2.4. Circular Lists *** 2.2.5. Doubly Linked Lists *** 2.2.6. Arrays and Orthogonal Lists ** 2.3.
Trees In botany, a tree is a perennial plant with an elongated stem, or trunk, usually supporting branches and leaves. In some usages, the definition of a tree may be narrower, including only woody plants with secondary growth, plants that ar ...
*** 2.3.1. Traversing Binary Trees *** 2.3.2. Binary Tree Representation of Trees *** 2.3.3. Other Representations of Trees *** 2.3.4. Basic Mathematical Properties of Trees **** 2.3.4.1. Free trees **** 2.3.4.2. Oriented trees **** 2.3.4.3. The "infinity lemma" **** 2.3.4.4. Enumeration of trees **** 2.3.4.5. Path length **** 2.3.4.6. History and bibliography *** 2.3.5. Lists and Garbage Collection ** 2.4. Multilinked Structures ** 2.5. Dynamic Storage Allocation ** 2.6. History and Bibliography


Volume 2 â€“ Seminumerical Algorithms

* Chapter 3 â€“ Random Numbers ** 3.1. Introduction ** 3.2. Generating Uniform Random Numbers *** 3.2.1. The Linear Congruential Method **** 3.2.1.1. Choice of modulus **** 3.2.1.2. Choice of multiplier **** 3.2.1.3. Potency *** 3.2.2. Other Methods ** 3.3. Statistical Tests *** 3.3.1. General Test Procedures for Studying Random Data *** 3.3.2. Empirical Tests *** 3.3.3. Theoretical Tests *** 3.3.4. The Spectral Test ** 3.4. Other Types of Random Quantities *** 3.4.1. Numerical Distributions *** 3.4.2. Random Sampling and
Shuffling Shuffling is a procedure used to randomize a deck of playing cards to provide an element of chance in card games. Shuffling is often followed by a cut, to help ensure that the shuffler has not manipulated the outcome. __TOC__ Techniques Over ...
** 3.5. What Is a
Random Sequence The concept of a random sequence is essential in probability theory and statistics. The concept generally relies on the notion of a sequence of random variables and many statistical discussions begin with the words "let ''X''1,...,''Xn'' be independ ...
? ** 3.6. Summary * Chapter 4 â€“ Arithmetic ** 4.1. Positional Number Systems ** 4.2.
Floating Point In computing, floating-point arithmetic (FP) is arithmetic that represents real numbers approximately, using an integer with a fixed precision, called the significand, scaled by an integer exponent of a fixed base. For example, 12.345 can be r ...
Arithmetic *** 4.2.1. Single-Precision Calculations *** 4.2.2. Accuracy of Floating Point Arithmetic *** 4.2.3. Double-Precision Calculations *** 4.2.4. Distribution of Floating Point Numbers ** 4.3. Multiple Precision Arithmetic *** 4.3.1. The Classical Algorithms *** 4.3.2. Modular Arithmetic *** 4.3.3. How Fast Can We Multiply? ** 4.4.
Radix In a positional numeral system, the radix or base is the number of unique digits, including the digit zero, used to represent numbers. For example, for the decimal/denary system (the most common system in use today) the radix (base number) is t ...
Conversion ** 4.5.
Rational Rationality is the quality of being guided by or based on reasons. In this regard, a person acts rationally if they have a good reason for what they do or a belief is rational if it is based on strong evidence. This quality can apply to an abili ...
Arithmetic *** 4.5.1. Fractions *** 4.5.2. The Greatest Common Divisor *** 4.5.3. Analysis of
Euclid's Algorithm In mathematics, the Euclidean algorithm,Some widely used textbooks, such as I. N. Herstein's ''Topics in Algebra'' and Serge Lang's ''Algebra'', use the term "Euclidean algorithm" to refer to Euclidean division or Euclid's algorithm, is an effi ...
*** 4.5.4. Factoring into Primes ** 4.6.
Polynomial In mathematics, a polynomial is an expression consisting of indeterminates (also called variables) and coefficients, that involves only the operations of addition, subtraction, multiplication, and positive-integer powers of variables. An examp ...
Arithmetic *** 4.6.1. Division of Polynomials *** 4.6.2. Factorization of Polynomials *** 4.6.3. Evaluation of Powers (
addition-chain exponentiation In mathematics and computer science, optimal addition-chain exponentiation is a method of exponentiation by a positive integer power that requires a minimal number of multiplications. Using ''the form of'' the shortest addition chain, with multipli ...
) *** 4.6.4. Evaluation of Polynomials ** 4.7. Manipulation of
Power Series In mathematics, a power series (in one variable) is an infinite series of the form \sum_^\infty a_n \left(x - c\right)^n = a_0 + a_1 (x - c) + a_2 (x - c)^2 + \dots where ''an'' represents the coefficient of the ''n''th term and ''c'' is a const ...


Volume 3 â€“ Sorting and Searching

* Chapter 5 â€“
Sorting Sorting refers to ordering data in an increasing or decreasing manner according to some linear relationship among the data items. # ordering: arranging items in a sequence ordered by some criterion; # categorizing: grouping items with similar p ...
** 5.1. Combinatorial Properties of
Permutation In mathematics, a permutation of a set is, loosely speaking, an arrangement of its members into a sequence or linear order, or if the set is already ordered, a rearrangement of its elements. The word "permutation" also refers to the act or pr ...
s *** 5.1.1. Inversions *** 5.1.2. Permutations of a Multiset *** 5.1.3. Runs *** 5.1.4. Tableaux and Involutions ** 5.2. Internal sorting *** 5.2.1. Sorting by Insertion *** 5.2.2. Sorting by Exchanging *** 5.2.3. Sorting by Selection *** 5.2.4. Sorting by Merging *** 5.2.5. Sorting by Distribution ** 5.3. Optimum Sorting *** 5.3.1. Minimum-Comparison Sorting *** 5.3.2. Minimum-Comparison Merging *** 5.3.3. Minimum-Comparison Selection *** 5.3.4. Networks for Sorting ** 5.4.
External Sorting External sorting is a class of sorting algorithms that can handle massive amounts of data. External sorting is required when the data being sorted do not fit into the main memory of a computing device (usually RAM) and instead they must reside in t ...
*** 5.4.1. Multiway Merging and Replacement Selection *** 5.4.2. The Polyphase Merge *** 5.4.3. The Cascade Merge *** 5.4.4. Reading Tape Backwards *** 5.4.5. The Oscillating Sort *** 5.4.6. Practical Considerations for Tape Merging *** 5.4.7. External Radix Sorting *** 5.4.8. Two-Tape Sorting *** 5.4.9. Disks and Drums ** 5.5. Summary, History, and Bibliography * Chapter 6 â€“
Searching Searching or search may refer to: Computing technology * Search algorithm, including keyword search ** :Search algorithms * Search and optimization for problem solving in artificial intelligence * Search engine technology, software for findi ...
** 6.1. Sequential Searching ** 6.2. Searching by Comparison of Keys *** 6.2.1. Searching an Ordered Table *** 6.2.2. Binary Tree Searching *** 6.2.3. Balanced Trees *** 6.2.4. Multiway Trees ** 6.3. Digital Searching ** 6.4.
Hashing Hash, hashes, hash mark, or hashing may refer to: Substances * Hash (food), a coarse mixture of ingredients * Hash, a nickname for hashish, a cannabis product Hash mark *Hash mark (sports), a marking on hockey rinks and gridiron football field ...
** 6.5. Retrieval on Secondary Keys


Volume 4A â€“ Combinatorial Algorithms, Part 1

* Chapter 7 â€“ Combinatorial Searching ** 7.1. Zeros and Ones *** 7.1.1.
Boolean Any kind of logic, function, expression, or theory based on the work of George Boole is considered Boolean. Related to this, "Boolean" may refer to: * Boolean data type, a form of data with only two possible values (usually "true" and "false" ...
Basics *** 7.1.2. Boolean Evaluation *** 7.1.3. Bitwise Tricks and Techniques *** 7.1.4. Binary Decision Diagrams ** 7.2. Generating All Possibilities *** 7.2.1. Generating Basic Combinatorial Patterns **** 7.2.1.1. Generating all n-
tuple In mathematics, a tuple is a finite ordered list (sequence) of elements. An -tuple is a sequence (or ordered list) of elements, where is a non-negative integer. There is only one 0-tuple, referred to as ''the empty tuple''. An -tuple is defi ...
s **** 7.2.1.2. Generating all
permutation In mathematics, a permutation of a set is, loosely speaking, an arrangement of its members into a sequence or linear order, or if the set is already ordered, a rearrangement of its elements. The word "permutation" also refers to the act or pr ...
s **** 7.2.1.3. Generating all
combination In mathematics, a combination is a selection of items from a set that has distinct members, such that the order of selection does not matter (unlike permutations). For example, given three fruits, say an apple, an orange and a pear, there are th ...
s **** 7.2.1.4. Generating all partitions **** 7.2.1.5. Generating all set partitions **** 7.2.1.6. Generating all
trees In botany, a tree is a perennial plant with an elongated stem, or trunk, usually supporting branches and leaves. In some usages, the definition of a tree may be narrower, including only woody plants with secondary growth, plants that ar ...
**** 7.2.1.7. History and further references


Volume 4B â€“ Combinatorial Algorithms, Part 2

* Chapter 7 â€“ Combinatorial Searching (continued) ** 7.2. Generating all possibilities (continued) *** 7.2.2. Backtrack programming (published in Fascicle 5) **** 7.2.2.1. Dancing links (includes discussion of Exact cover) (published in Fascicle 5) **** 7.2.2.2.
Satisfiability In mathematical logic, a formula is ''satisfiable'' if it is true under some assignment of values to its variables. For example, the formula x+3=y is satisfiable because it is true when x=3 and y=6, while the formula x+1=x is not satisfiable over ...
(published in Fascicle 6)


Planned


Volume 4C, 4D – Combinatorial Algorithms

* Chapter 7 â€“ Combinatorial Searching (continued) ** 7.2. Generating all possibilities (continued) *** 7.2.2. Backtrack programming (continued) **** 7.2.2.3.
Constraint satisfaction In artificial intelligence and operations research, constraint satisfaction is the process of finding a solution through a set of constraints that impose conditions that the variables must satisfy. A solution is therefore a set of values for th ...
(online draft in pre-fascicle 7A) **** 7.2.2.4. Hamiltonian paths and cycles (online draft in pre-fascicle 8A) **** 7.2.2.5.
Cliques A clique ( AusE, CanE, or ), in the social sciences, is a group of individuals who interact with one another and share similar interests. Interacting with cliques is part of normative social development regardless of gender, ethnicity, or popula ...
**** 7.2.2.6. Covers (
Vertex cover In graph theory, a vertex cover (sometimes node cover) of a graph is a set of vertices that includes at least one endpoint of every edge of the graph. In computer science, the problem of finding a minimum vertex cover is a classical optimi ...
,
Set cover problem The set cover problem is a classical question in combinatorics, computer science, operations research, and complexity theory. It is one of Karp's 21 NP-complete problems shown to be NP-complete in 1972. Given a set of elements (called the uni ...
, Exact cover,
Clique cover In graph theory, a clique cover or partition into cliques of a given undirected graph is a partition of the vertices into cliques, subsets of vertices within which every two vertices are adjacent. A minimum clique cover is a clique cover that u ...
) **** 7.2.2.7. Squares **** 7.2.2.8. A potpourri of puzzles (online draft in pre-fascicle 9B) (includes Perfect digital invariant) **** 7.2.2.9. Estimating backtrack costs (chapter 6 of "Selected Papers on Analysis of Algorithms", and Fascicle 5, pp. 44−47, under the heading "Running time estimates") *** 7.2.3. Generating inequivalent patterns (includes discussion of
Pólya enumeration theorem The Pólya enumeration theorem, also known as the Redfield–Pólya theorem and Pólya counting, is a theorem in combinatorics that both follows from and ultimately generalizes Burnside's lemma on the number of orbits of a group action on a se ...
) (see "Techniques for Isomorph Rejection", chapter 4 of "Classification Algorithms for Codes and Designs" by Kaski and Östergård) ** 7.3. Shortest paths ** 7.4.
Graph algorithms The following is a list of well-known algorithms along with one-line descriptions for each. Automated planning Combinatorial algorithms General combinatorial algorithms * Brent's algorithm: finds a cycle in function value iterations using on ...
(online draft in pre-fascicle 12A) *** 7.4.1. Components and traversal (online draft in pre-fascicle 12A) **** 7.4.1.1. Union-find algorithms (online draft in pre-fascicle 12A) **** 7.4.1.2.
Depth-first search Depth-first search (DFS) is an algorithm for traversing or searching tree or graph data structures. The algorithm starts at the root node (selecting some arbitrary node as the root node in the case of a graph) and explores as far as possible al ...
(online draft in pre-fascicle 12A) **** 7.4.1.3. Vertex and edge connectivity *** 7.4.2. Special classes of graphs *** 7.4.3.
Expander graph In graph theory, an expander graph is a sparse graph that has strong connectivity properties, quantified using vertex, edge or spectral expansion. Expander constructions have spawned research in pure and applied mathematics, with several applic ...
s *** 7.4.4.
Random graph In mathematics, random graph is the general term to refer to probability distributions over graphs. Random graphs may be described simply by a probability distribution, or by a random process which generates them. The theory of random graphs l ...
s ** 7.5. Graphs and optimization *** 7.5.1. Bipartite matching (including maximum-cardinality matching,
Stable marriage problem In mathematics, economics, and computer science, the stable marriage problem (also stable matching problem or SMP) is the problem of finding a stable matching between two equally sized sets of elements given an ordering of preferences for each elem ...

Mariages Stables
(online draft in pre-fascicle 14A) *** 7.5.2. The assignment problem *** 7.5.3. Network flows *** 7.5.4. Optimum subtrees *** 7.5.5. Optimum matching *** 7.5.6. Optimum orderings ** 7.6. Independence theory *** 7.6.1. Independence structures *** 7.6.2. Efficient
matroid In combinatorics, a branch of mathematics, a matroid is a structure that abstracts and generalizes the notion of linear independence in vector spaces. There are many equivalent ways to define a matroid axiomatically, the most significant being i ...
algorithms ** 7.7. Discrete
dynamic programming Dynamic programming is both a mathematical optimization method and a computer programming method. The method was developed by Richard Bellman in the 1950s and has found applications in numerous fields, from aerospace engineering to economics. ...
(see also
Transfer-matrix method In statistical mechanics, the transfer-matrix method is a mathematical technique which is used to write the partition function into a simpler form. It was introduced in 1941 by Hans Kramers and Gregory Wannier. In many one dimensional lattice ...
) ** 7.8.
Branch-and-bound Branch and bound (BB, B&B, or BnB) is an algorithm design paradigm for discrete and combinatorial optimization problems, as well as mathematical optimization. A branch-and-bound algorithm consists of a systematic enumeration of candidate solutio ...
techniques ** 7.9. Herculean tasks (aka
NP-hard In computational complexity theory, NP-hardness ( non-deterministic polynomial-time hardness) is the defining property of a class of problems that are informally "at least as hard as the hardest problems in NP". A simple example of an NP-hard pr ...
problems) ** 7.10. Near-optimization * Chapter 8 â€“
Recursion Recursion (adjective: ''recursive'') occurs when a thing is defined in terms of itself or of its type. Recursion is used in a variety of disciplines ranging from linguistics to logic. The most common application of recursion is in mathematics ...
(chapter 22 of "Selected Papers on Analysis of Algorithms")


Volume 5 â€“ Syntactic Algorithms

* Chapter 9 â€“ Lexical scanning (includes also
string search In computer science, string-searching algorithms, sometimes called string-matching algorithms, are an important class of string algorithms that try to find a place where one or several strings (also called patterns) are found within a larger str ...
and data compression) * Chapter 10 â€“
Parsing Parsing, syntax analysis, or syntactic analysis is the process of analyzing a string of symbols, either in natural language, computer languages or data structures, conforming to the rules of a formal grammar. The term ''parsing'' comes from Lati ...
techniques


Volume 6 â€“ The Theory of Context-free Languages


Volume 7 â€“ Compiler Techniques


English editions


Current editions

These are the current editions in order by volume number: * ''The Art of Computer Programming, Volumes 1-4A Boxed Set''. Third Edition (Reading, Massachusetts: Addison-Wesley, 2011), 3168pp. * ''The Art of Computer Programming, Volumes 1-4B Boxed Set''. (Reading, Massachusetts: Addison-Wesley, 2023), 3904pp. ** ''Volume 1: Fundamental Algorithms''. Third Edition (Reading, Massachusetts: Addison-Wesley, 1997), xx+650pp. . Errata

(2011-01-08)

(2020-03-26, 27th printing run, printing). Addenda

(2011). ** ''Volume 2: Seminumerical Algorithms''. Third Edition (Reading, Massachusetts: Addison-Wesley, 1997), xiv+762pp. . Errata

(2011-01-08)

(2020-03-26, 26th printing). Addenda

(2011). ** ''Volume 3: Sorting and Searching''. Second Edition (Reading, Massachusetts: Addison-Wesley, 1998), xiv+780pp.+foldout. . Errata

(2011-01-08)

(2020-03-26, 27th printing). Addenda

(2011). ** ''Volume 4A: Combinatorial Algorithms, Part 1''. First Edition (Upper Saddle River, New Jersey: Addison-Wesley, 2011), xv+883pp. . Errata

(2020-03-26?, 22nd printing). ** ''Volume 4B: Combinatorial Algorithms, Part 2''. First Edition (Upper Saddle River, New Jersey: Addison-Wesley, 2023), xviii+714pp. (2022-11-??, 2nd printing). * ''Volume 1, Fascicle 1: MMIX â€“ A RISC Computer for the New Millennium''. (Addison-Wesley, 2005-02-14) . Errata

(2020-03-16) (will be in the fourth edition of volume 1)


Previous editions


Complete volumes

These volumes were superseded by newer editions and are in order by date. * ''Volume 1: Fundamental Algorithms''. First edition, 1968, xxi+634pp, . * ''Volume 2: Seminumerical Algorithms''. First edition, 1969, xi+624pp, . * ''Volume 3: Sorting and Searching''. First edition, 1973, xi+723pp+foldout, . Errata

* ''Volume 1: Fundamental Algorithms''. Second edition, 1973, xxi+634pp, . Errata

* ''Volume 2: Seminumerical Algorithms''. Second edition, 1981, xiii+ 688pp, . Errata

* ''The Art of Computer Programming, Volumes 1-3 Boxed Set''. Second Edition (Reading, Massachusetts: Addison-Wesley, 1998), pp.


Fascicles

Volume 4 fascicle (book), fascicles 0–4 were revised and published as Volume 4A: * ''Volume 4, Fascicle 0: Introduction to Combinatorial Algorithms and Boolean Functions''. (Addison-Wesley Professional, 2008-04-28) vi+240pp, . Errata

(2011-01-01). * ''Volume 4, Fascicle 1: Bitwise Tricks & Techniques; Binary Decision Diagrams''. (Addison-Wesley Professional, 2009-03-27) viii+260pp, . Errata

(2011-01-01). * ''Volume 4, Fascicle 2: Generating All Tuples and Permutations''. (Addison-Wesley, 2005-02-14) v+127pp, . Errata

(2011-01-01). * ''Volume 4, Fascicle 3: Generating All Combinations and Partitions''. (Addison-Wesley, 2005-07-26) vi+150pp, . Errata

(2011-01-01). * ''Volume 4, Fascicle 4: Generating All Trees; History of Combinatorial Generation''. (Addison-Wesley, 2006-02-06) vi+120pp, . Errata

(2011-01-01). Volume 4 fascicle (book), fascicles 5–6 were revised and published as Volume 4B: * ''Volume 4, Fascicle 5: Mathematical Preliminaries Redux; Backtracking; Dancing Links''. (Addison-Wesley, 2019-11-22) xiii+382pp, . Errata

(2020-03-27) * ''Volume 4, Fascicle 6: Satisfiability''. (Addison-Wesley, 2015-12-08) xiii+310pp, . Errata

(2020-03-26)


Pre-fascicles

Volume 4 pre-fascicles 5A, 5B, and 5C were revised and published as fascicle 5. Volume 4 pre-fascicle 6A was revised and published as fascicle 6. *
Volume 4, Pre-fascicle 7A: Constraint Satisfaction
' *
Volume 4, Pre-fascicle 8A: Hamiltonian Paths and Cycles
' *
Volume 4, Pre-fascicle 9B: A Potpourri of Puzzles
' *
Volume 4, Pre-fascicle 12A: Components and Traversal(PDF Version)
' *
Volume 4, Pre-fascicle 14A: Bipartite Matching
'


See also

* ''
Introduction to Algorithms ''Introduction to Algorithms'' is a book on computer programming by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. The book has been widely used as the textbook for algorithms courses at many universities and is ...
''


References

Notes Citations Sources * *


External links


Overview of topics
(Knuth's personal homepage)
Oral history interview with Donald E. Knuth
at
Charles Babbage Institute The IT History Society (ITHS) is an organization that supports the history and scholarship of information technology by encouraging, fostering, and facilitating archival and historical research. Formerly known as the Charles Babbage Foundati ...
, University of Minnesota, Minneapolis. Knuth discusses software patenting,
structured programming Structured programming is a programming paradigm aimed at improving the clarity, quality, and development time of a computer program by making extensive use of the structured control flow constructs of selection ( if/then/else) and repetition ( ...
, collaboration and his development of
TeX Tex may refer to: People and fictional characters * Tex (nickname), a list of people and fictional characters with the nickname * Joe Tex (1933–1982), stage name of American soul singer Joseph Arrington Jr. Entertainment * ''Tex'', the Italian ...
. The oral history discusses the writing of ''The Art of Computer Programming''.
"Robert W Floyd, In Memoriam", by Donald E. Knuth
- (on the influence of Bob Floyd)
''TAoCP'' and its Influence of Computer Science (Softpanorama)
{{DEFAULTSORT:Art Of Computer Programming, The 1968 non-fiction books 1969 non-fiction books 1973 non-fiction books 1981 non-fiction books 2011 non-fiction books Addison-Wesley books American non-fiction books Analysis of algorithms Books by Donald Knuth Computer arithmetic algorithms Computer programming books Computer science books English-language books Monographs