The Computer Language Benchmarks Game (formerly called The Great Computer Language Shootout) is a

free software Free software or libre software is computer software distributed under terms that allow users to run the software for any purpose as well as to study, change, and distribute it and any adapted versions. Free software is a matter of liberty, n ...

project for comparing how a given subset of simple

algorithms In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...

can be implemented in various popular

programming languages A programming language is a system of notation for writing computer programs. Most programming languages are text-based formal languages, but they may also be graphical. They are a kind of computer language. The description of a programming ...

. The project consists of: * A set of very simple algorithmic problems * Various implementations to the above problems in various programming languages * A set of unit tests to verify that the submitted implementations solve the problem statement * A framework for running and timing the implementations * A website to facilitate the interactive comparison of the results

Supported languages

Due to resource constraints, only a small subset of common programming languages are supported, up to the discretion of the game's operator.

Metrics

The following aspects of each given implementation are measured: * overall user runtime * peak

memory allocation Memory management is a form of resource management applied to computer memory. The essential requirement of memory management is to provide ways to dynamically allocate portions of memory to programs at their request, and free it for reuse when ...

gzip gzip is a file format and a software application used for file compression and decompression. The program was created by Jean-loup Gailly and Mark Adler as a free software replacement for the compress program used in early Unix systems, and i ...

'ped size of the solution's source code * sum of total CPU time over all threads * individual CPU

utilization * Rental utilization - economy * Capacity utilization - load on some process * Utilization management Utilization management (UM) or utilization review is the use of managed care techniques such as prior authorization that allow payers, particular ...

It is common to see multiple solutions in the same programming language for the same problem. This highlights that within the constraints of a given language, a solution can be given which is either of high abstraction, is memory efficient, is fast, or can be parallelized better.

Benchmark programs

It was a design choice from the start to only include very simple toy problems, each providing a different kind of programming challenge. This provides users of the Benchmark Game the opportunity to scrutinize the various implementations. * binary-trees * chameneos-redux * fannkuch-redux *

fasta FASTA is a DNA and protein sequence alignment software package first described by David J. Lipman and William R. Pearson in 1985. Its legacy is the FASTA format which is now ubiquitous in bioinformatics. History The original FASTA program ...

* k-nucleotide * mandelbrot * meteor-contest *

n-body In physics and astronomy, an ''N''-body simulation is a simulation of a dynamical system of particles, usually under the influence of physical forces, such as gravity (see ''n''-body problem for other applications). ''N''-body simulations ar ...

* pidigits * regex-redux * reverse-complement * spectral-norm * thread-ring

History

The project was known as ''The Great Computer Language Shootout'' until 2007. A port for Windows was maintained separately between 2002 and 2003. The sources have been archived on GitLab. There are also older forks on GitHub. The project is continuously evolving. The list of supported programming languages is updated approximately once per year, following market trends. Users can also submit improved solutions to any of the problems or suggest testing methodology refinement.

Caveats

The developers themselves highlight the fact that those doing research should exercise caution when using such microbenchmarks:

Impact

The benchmark results have uncovered various compiler issues. Sometimes a given compiler failed to process unusual, but otherwise grammatically valid constructs. At other times, runtime performance was shown to be below expectations, which prompted compiler developers to revise their optimization capabilities. Various research articles have been based on the benchmarks, its results and its methodology.

References

External links

* {{Official website Programming language comparisons Benchmarks (computing)