History of entropy
   HOME

TheInfoList



OR:

The concept of
entropy Entropy is a scientific concept, as well as a measurable physical property, that is most commonly associated with a state of disorder, randomness, or uncertainty. The term and the concept are used in diverse fields, from classical thermodynam ...
developed in response to the observation that a certain amount of functional energy released from combustion reactions is always lost to dissipation or friction and is thus not transformed into useful work. Early heat-powered engines such as
Thomas Savery Thomas Savery (; c. 1650 – 15 May 1715) was an English inventor and engineer. He invented the first commercially used steam-powered device, a steam pump which is often referred to as the "Savery engine". Savery's steam pump was a revolutiona ...
's (1698), the
Newcomen engine The atmospheric engine was invented by Thomas Newcomen in 1712, and is often referred to as the Newcomen fire engine (see below) or simply as a Newcomen engine. The engine was operated by condensing steam drawn into the cylinder, thereby creati ...
(1712) and the Cugnot
steam tricycle A steam tricycle is a steam-driven three-wheeled vehicle. History In the early days of motorised vehicle development, a number of experimenters built steam-powered vehicles with three wheels. The first steam tricycle – and probably the first ...
(1769) were inefficient, converting less than two percent of the input energy into useful
work output In physics, ''work output'' is the work done by a simple machine, compound machine, or any type of engine model. In common terms, it is the energy output, which for simple machines is always less than the energy input, even though the forces may be ...
; a great deal of useful energy was dissipated or lost. Over the next two centuries, physicists investigated this puzzle of lost energy; the result was the concept of entropy. In the early 1850s,
Rudolf Clausius Rudolf Julius Emanuel Clausius (; 2 January 1822 – 24 August 1888) was a German physicist and mathematician and is considered one of the central founding fathers of the science of thermodynamics. By his restatement of Sadi Carnot's principle ...
set forth the concept of the
thermodynamic system A thermodynamic system is a body of matter and/or radiation, confined in space by walls, with defined permeabilities, which separate it from its surroundings. The surroundings may include other thermodynamic systems, or physical systems that are ...
and posited the argument that in any
irreversible process In science, a process that is not reversible is called irreversible. This concept arises frequently in thermodynamics. All complex natural processes are irreversible, although a phase transition at the coexistence temperature (e.g. melting of ic ...
a small amount of
heat In thermodynamics, heat is defined as the form of energy crossing the boundary of a thermodynamic system by virtue of a temperature difference across the boundary. A thermodynamic system does not ''contain'' heat. Nevertheless, the term is al ...
energy ''δQ'' is incrementally dissipated across the system boundary. Clausius continued to develop his ideas of lost energy, and coined the term ''entropy''. Since the mid-20th century the concept of entropy has found application in the field of
information theory Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of information. The field was originally established by the works of Harry Nyquist a ...
, describing an analogous loss of data in information transmission systems. In 2019, the notion was leveraged as 'relative beam entropy' for a single-parameter characterization of beamspace randomness of 5G/6G sparse MIMO channels, e.g., 3GPP 5G cellular channels, in millimeter-wave and teraHertz bands.


Classical thermodynamic views

In 1803, mathematician
Lazare Carnot Lazare Nicolas Marguerite, Count Carnot (; 13 May 1753 – 2 August 1823) was a French mathematician, physicist and politician. He was known as the "Organizer of Victory" in the French Revolutionary Wars and Napoleonic Wars. Education and early ...
published a work entitled ''Fundamental Principles of Equilibrium and Movement''. This work includes a discussion on the efficiency of fundamental machines, i.e. pulleys and inclined planes. Carnot saw through all the details of the mechanisms to develop a general discussion on the conservation of mechanical energy. Over the next three decades, Carnot's theorem was taken as a statement that in any machine the accelerations and shocks of the moving parts all represent losses of ''moment of activity'', i.e. the useful work done. From this Carnot drew the inference that
perpetual motion Perpetual motion is the motion of bodies that continues forever in an unperturbed system. A perpetual motion machine is a hypothetical machine that can do work infinitely without an external energy source. This kind of machine is impossible, a ...
was impossible. This ''loss of moment of activity'' was the first-ever rudimentary statement of the
second law of thermodynamics The second law of thermodynamics is a physical law based on universal experience concerning heat and Energy transformation, energy interconversions. One simple statement of the law is that heat always moves from hotter objects to colder objects ( ...
and the concept of 'transformation-energy' or ''entropy'', i.e. energy lost to dissipation and friction. Carnot died in exile in 1823. During the following year his son Sadi Carnot, having graduated from the
École Polytechnique École may refer to: * an elementary school in the French educational stages normally followed by secondary education establishments (collège and lycée) * École (river), a tributary of the Seine flowing in région Île-de-France * École, Savoi ...
training school for engineers, but now living on half-pay with his brother Hippolyte in a small apartment in Paris, wrote ''
Reflections on the Motive Power of Fire ''Reflections on the Motive Power of Fire and on Machines Fitted to Develop that Power'' is a book published in 1824 by French physicist Sadi Carnot.full text of 1897 ed. ( Full text of 1897 edition on Wikisource ) The 118-page book's French t ...
''. In this book, Sadi visualized an ideal engine in which any heat (i.e.,
caloric Caloric is a brand of kitchen appliances, which dates back to 1903. History Caloric Corporation began as the Klein Stove Company in Philadelphia in 1890. The Caloric brand was introduced in 1903. It was reorganized in 1946 as the Caloric Stove C ...
) converted into
work Work may refer to: * Work (human activity), intentional activity people perform to support themselves, others, or the community ** Manual labour, physical work done by humans ** House work, housework, or homemaking ** Working animal, an animal tr ...
, could be reinstated by reversing the motion of the cycle, a concept subsequently known as
thermodynamic reversibility In thermodynamics, a reversible process is a process, involving a system and its surroundings, whose direction can be reversed by infinitesimal changes in some properties of the surroundings, such as pressure or temperature. Throughout an enti ...
. Building on his father's work, Sadi postulated the concept that "some caloric is always lost" in the conversion into work, even in his idealized reversible heat engine, which excluded frictional losses and other losses due to the imperfections of any real machine. He also discovered that this idealized efficiency was dependent only on the
temperature Temperature is a physical quantity that expresses quantitatively the perceptions of hotness and coldness. Temperature is measured with a thermometer. Thermometers are calibrated in various temperature scales that historically have relied o ...
s of the heat reservoirs between which the engine was working, and not on the types of
working fluid For fluid power, a working fluid is a gas or liquid that primarily transfers force, motion, or mechanical energy. In hydraulics, water or hydraulic fluid transfers force between hydraulic components such as hydraulic pumps, hydraulic cylinders, a ...
s. Any real
heat engine In thermodynamics and engineering, a heat engine is a system that converts heat to mechanical energy, which can then be used to do mechanical work. It does this by bringing a working substance from a higher state temperature to a lower state ...
could not realize the
Carnot cycle A Carnot cycle is an ideal thermodynamic cycle proposed by French physicist Sadi Carnot in 1824 and expanded upon by others in the 1830s and 1840s. By Carnot's theorem, it provides an upper limit on the efficiency of any classical thermodynam ...
's reversibility, and was condemned to be even less efficient. This loss of usable caloric was a precursory form of the increase in entropy as we now know it. Though formulated in terms of caloric, rather than entropy, this was an early insight into the second law of thermodynamics.


1854 definition

In his 1854 memoir, Clausius first develops the concepts of ''interior work'', i.e. that "which the atoms of the body exert upon each other", and ''exterior work'', i.e. that "which arise from foreign influences owhich the body may be exposed", which may act on a working body of fluid or gas, typically functioning to work a piston. He then discusses the three categories into which heat ''Q'' may be divided: #Heat employed in increasing the heat actually existing in the body. #Heat employed in producing the interior work. #Heat employed in producing the exterior work. Building on this logic, and following a mathematical presentation of the ''first fundamental theorem'', Clausius then presented the first-ever mathematical formulation of entropy, although at this point in the development of his theories he called it "equivalence-value", perhaps referring to the concept of the
mechanical equivalent of heat In the history of science, the mechanical equivalent of heat states that motion and heat are mutually interchangeable and that in every case, a given amount of work would generate the same amount of heat, provided the work done is totally converte ...
which was developing at the time rather than entropy, a term which was to come into use later. He stated:
the ''second fundamental theorem'' in the mechanical
theory of heat The history of thermodynamics is a fundamental strand in the history of physics, the history of chemistry, and the history of science in general. Owing to the relevance of thermodynamics in much of science and technology, its history is finely wo ...
may thus be enunciated: If two transformations which, without necessitating any other permanent change, can mutually replace one another, be called equivalent, then the generations of the quantity of heat ''Q'' from work at the temperature ''T'', has the ''equivalence-value'': :: \frac and the passage of the quantity of heat ''Q'' from the temperature ''T1'' to the temperature ''T2'', has the equivalence-value: :: Q \left( \frac - \frac \right) wherein ''T'' is a function of the temperature, independent of the nature of the process by which the transformation is effected.
In modern terminology, that is, the terminology introduced by Clausius himself in 1865, we think of this equivalence-value as "entropy", symbolized by ''S''. Thus, using the above description, we can calculate the entropy change Δ''S'' for the passage of the quantity of heat ''Q'' from the temperature ''T1'', through the "working body" of fluid, which was typically a body of steam, to the temperature ''T2'' as shown below: If we make the assignment: : S= \frac Then, the entropy change or "equivalence-value" for this transformation is: : \Delta S = S_ - S_ \, which equals: : \Delta S = \left(\frac - \frac \right) and by factoring out Q, we have the following form, as was derived by Clausius: : \Delta S = Q\left(\frac - \frac \right)


1856 definition

In 1856, Clausius stated what he called the "second fundamental theorem in the
mechanical theory of heat The history of thermodynamics is a fundamental strand in the history of physics, the history of chemistry, and the history of science in general. Owing to the relevance of thermodynamics in much of science and technology, its history is finely wov ...
" in the following form: :\int \frac = -N where ''N'' is the "equivalence-value" of all uncompensated transformations involved in a cyclical process. This equivalence-value was a precursory formulation of entropy.


1862 definition

In 1862, Clausius stated what he calls the "theorem respecting the equivalence-values of the transformations" or what is now known as the
second law of thermodynamics The second law of thermodynamics is a physical law based on universal experience concerning heat and Energy transformation, energy interconversions. One simple statement of the law is that heat always moves from hotter objects to colder objects ( ...
, as such: Quantitatively, Clausius states the mathematical expression for this theorem is follows. This was an early formulation of the second law and one of the original forms of the concept of entropy.


1865 definition

In 1865, Clausius gave irreversible heat loss, or what he had previously been calling "equivalence-value", a name: Clausius did not specify why he chose the symbol "S" to represent entropy, and it is almost certainly untrue that Clausius chose "S" in honor of Sadi Carnot; the given names of scientists are rarely if ever used this way.


Later developments

In 1876, physicist
J. Willard Gibbs Josiah Willard Gibbs (; February 11, 1839 – April 28, 1903) was an American scientist who made significant theoretical contributions to physics, chemistry, and mathematics. His work on the applications of thermodynamics was instrumental in t ...
, building on the work of Clausius,
Hermann von Helmholtz Hermann Ludwig Ferdinand von Helmholtz (31 August 1821 – 8 September 1894) was a German physicist and physician who made significant contributions in several scientific fields, particularly hydrodynamic stability. The Helmholtz Association, ...
and others, proposed that the measurement of "available energy" Δ''G'' in a thermodynamic system could be mathematically accounted for by subtracting the "energy loss" ''T''Δ''S'' from total energy change of the system Δ''H''. These concepts were further developed by
James Clerk Maxwell James Clerk Maxwell (13 June 1831 – 5 November 1879) was a Scottish mathematician and scientist responsible for the classical theory of electromagnetic radiation, which was the first theory to describe electricity, magnetism and ligh ...
871 __NOTOC__ Year 871 ( DCCCLXXI) was a common year starting on Monday (link will display the full calendar) of the Julian calendar. Events By place Europe * The English retreat onto the Berkshire Downs. The Great Heathen Army, led by the ...
and
Max Planck Max Karl Ernst Ludwig Planck (, ; 23 April 1858 – 4 October 1947) was a German theoretical physicist whose discovery of energy quanta won him the Nobel Prize in Physics in 1918. Planck made many substantial contributions to theoretical p ...
903 __NOTOC__ Year 903 ( CMIII) was a common year starting on Saturday (link will display the full calendar) of the Julian calendar. Events By place Europe * King Berengar I of Italy proceeds to issue concessions and privileges to the Lo ...


Statistical thermodynamic views

In 1877,
Ludwig Boltzmann Ludwig Eduard Boltzmann (; 20 February 1844 – 5 September 1906) was an Austrian physicist and philosopher. His greatest achievements were the development of statistical mechanics, and the statistical explanation of the second law of thermodyn ...
developed a statistical mechanical evaluation of the entropy , of a body in its own given macrostate of internal thermodynamic equilibrium. It may be written as: :S = k_ \ln \Omega \! where : denotes
Boltzmann's constant The Boltzmann constant ( or ) is the proportionality factor that relates the average relative kinetic energy of particles in a gas with the thermodynamic temperature of the gas. It occurs in the definitions of the kelvin and the gas constant, ...
and : denotes the number of microstates consistent with the given equilibrium macrostate. Boltzmann himself did not actually write this formula expressed with the named constant , which is due to Planck's reading of Boltzmann. Boltzmann saw entropy as a measure of statistical "mixedupness" or disorder. This concept was soon refined by
J. Willard Gibbs Josiah Willard Gibbs (; February 11, 1839 – April 28, 1903) was an American scientist who made significant theoretical contributions to physics, chemistry, and mathematics. His work on the applications of thermodynamics was instrumental in t ...
, and is now regarded as one of the cornerstones of the theory of
statistical mechanics In physics, statistical mechanics is a mathematical framework that applies statistical methods and probability theory to large assemblies of microscopic entities. It does not assume or postulate any natural laws, but explains the macroscopic be ...
.
Erwin Schrödinger Erwin Rudolf Josef Alexander Schrödinger (, ; ; 12 August 1887 – 4 January 1961), sometimes written as or , was a Nobel Prize-winning Austrian physicist with Irish citizenship who developed a number of fundamental results in quantum theory ...
made use of Boltzmann's work in his book ''
What is Life? ''What Is Life? The Physical Aspect of the Living Cell'' is a 1944 science book written for the lay reader by physicist Erwin Schrödinger. The book was based on a course of public lectures delivered by Schrödinger in February 1943, under the ...
'' to explain why living systems have far fewer replication errors than would be predicted from Statistical Thermodynamics. Schrödinger used the Boltzmann equation in a different form to show increase of entropy :S = k_ \ln D \! where ''D'' is the number of possible energy states in the system that can be randomly filled with energy. He postulated a local decrease of entropy for living systems when (1/D) represents the number of states that are prevented from randomly distributing, such as occurs in replication of the genetic code. :-S = k_ \ln(1/D) \! Without this correction Schrödinger claimed that statistical thermodynamics would predict one thousand mutations per million replications, and ten mutations per hundred replications following the rule for square root of n, far more mutations than actually occur. Schrödinger's separation of random and non-random energy states is one of the few explanations for why entropy could be low in the past, but continually increasing now. It has been proposed as an explanation of localized decrease of entropy in
radiant energy Radiant may refer to: Computers, software, and video games * Radiant (software), a content management system * GtkRadiant, a level editor created by id Software for their games * Radiant AI, a technology developed by Bethesda Softworks for ''The ...
focusing in parabolic reflectors and during dark current in diodes, which would otherwise be in violation of Statistical Thermodynamics.


Information theory

An analog to ''thermodynamic entropy'' is information entropy. In 1948, while working at
Bell Telephone The Bell System was a system of telecommunication companies, led by the Bell Telephone Company and later by the American Telephone and Telegraph Company (AT&T), that dominated the telephone services industry in North America for over one hundre ...
Laboratories, electrical engineer
Claude Shannon Claude Elwood Shannon (April 30, 1916 – February 24, 2001) was an American people, American mathematician, electrical engineering, electrical engineer, and cryptography, cryptographer known as a "father of information theory". As a 21-year-o ...
set out to mathematically quantify the statistical nature of "lost information" in phone-line signals. To do this, Shannon developed the very general concept of
information entropy In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent to the variable's possible outcomes. Given a discrete random variable X, which takes values in the alphabet \ ...
, a fundamental cornerstone of
information theory Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of information. The field was originally established by the works of Harry Nyquist a ...
. Although the story varies, initially it seems that Shannon was not particularly aware of the close similarity between his new quantity and earlier work in thermodynamics. In 1939, however, when Shannon had been working on his equations for some time, he happened to visit the mathematician
John von Neumann John von Neumann (; hu, Neumann János Lajos, ; December 28, 1903 – February 8, 1957) was a Hungarian-American mathematician, physicist, computer scientist, engineer and polymath. He was regarded as having perhaps the widest cove ...
. During their discussions, regarding what Shannon should call the "measure of uncertainty" or attenuation in phone-line signals with reference to his new information theory, according to one source: : According to another source, when von Neumann asked him how he was getting on with his information theory, Shannon replied: : In 1948 Shannon published his seminal paper ''
A Mathematical Theory of Communication "A Mathematical Theory of Communication" is an article by mathematician Claude E. Shannon published in '' Bell System Technical Journal'' in 1948. It was renamed ''The Mathematical Theory of Communication'' in the 1949 book of the same name, a sm ...
'', in which he devoted a section to what he calls Choice, Uncertainty, and Entropy. In this section, Shannon introduces an ''H function'' of the following form: :H = -K\sum_^k p(i) \log p(i), where ''K'' is a positive constant. Shannon then states that "any quantity of this form, where ''K'' merely amounts to a choice of a unit of measurement, plays a central role in information theory as measures of information, choice, and uncertainty." Then, as an example of how this expression applies in a number of different fields, he references R.C. Tolman's 1938 ''Principles of Statistical Mechanics'', stating that "the form of ''H'' will be recognized as that of entropy as defined in certain formulations of statistical mechanics where ''pi'' is the probability of a system being in cell ''i'' of its phase space… ''H'' is then, for example, the ''H'' in Boltzmann's famous H theorem." As such, over the last fifty years, ever since this statement was made, people have been overlapping the two concepts or even stating that they are exactly the same. Shannon's information entropy is a much more general concept than statistical thermodynamic entropy. Information entropy is present whenever there are unknown quantities that can be described only by a probability distribution. In a series of papers by
E. T. Jaynes Edwin Thompson Jaynes (July 5, 1922 – April 30, 1998) was the Wayman Crow Distinguished Professor of Physics at Washington University in St. Louis. He wrote extensively on statistical mechanics and on foundations of probability and statisti ...
starting in 1957, the statistical thermodynamic entropy can be seen as just a particular application of Shannon's information entropy to the probabilities of particular microstates of a system occurring in order to produce a particular macrostate.


Beam Entropy

The beamspace randomness of MIMO wireless channels, e.g., 3GPP 5G cellular channels, was characterized using the single parameter of 'Beam Entropy.' It facilitates the selection of the sparse MIMO channel learning algorithms in the beamspace.


Popular use

The term entropy is often used in popular language to denote a variety of unrelated phenomena. One example is the concept of corporate entropy as put forward somewhat humorously by authors Tom DeMarco and Timothy Lister in their 1987 classic publication ''Peopleware'', a book on growing and managing productive teams and successful software projects. Here, they view energy waste as red tape and business team inefficiency as a form of entropy, i.e. energy lost to waste. This concept has caught on and is now common jargon in business schools. In another example, entropy is the central theme in
Isaac Asimov yi, יצחק אזימאװ , birth_date = , birth_place = Petrovichi, Russian SFSR , spouse = , relatives = , children = 2 , death_date = , death_place = Manhattan, New York City, U.S. , nationality = Russian (1920–1922)Soviet (192 ...
's short story
The Last Question "The Last Question" is a science fiction short story by American writer Isaac Asimov. It first appeared in the November 1956 issue of Science Fiction Quarterly and was anthologized in the collections Nine Tomorrows (1959), The Best of Isaac Asi ...
(first copyrighted in 1956). The story plays with the idea that the most important question is how to stop the increase of entropy.


Terminology overlap

When necessary, to disambiguate between the statistical thermodynamic concept of entropy, and entropy-like formulae put forward by different researchers, the statistical thermodynamic entropy is most properly referred to as the
Gibbs entropy The concept entropy was first developed by German physicist Rudolf Clausius in the mid-nineteenth century as a thermodynamic property that predicts that certain spontaneous processes are irreversible or impossible. In statistical mechanics, entrop ...
. The terms ''Boltzmann–Gibbs entropy'' or ''BG entropy'', and ''Boltzmann–Gibbs–Shannon entropy'' or ''BGS entropy'' are also seen in the literature.


See also

*
Entropy Entropy is a scientific concept, as well as a measurable physical property, that is most commonly associated with a state of disorder, randomness, or uncertainty. The term and the concept are used in diverse fields, from classical thermodynam ...
*
Enthalpy Enthalpy , a property of a thermodynamic system, is the sum of the system's internal energy and the product of its pressure and volume. It is a state function used in many measurements in chemical, biological, and physical systems at a constant ...
*
History of thermodynamics The history of thermodynamics is a fundamental strand in the history of physics, the history of chemistry, and the history of science in general. Owing to the relevance of thermodynamics in much of science and technology, its history is finely wo ...
*
Thermodynamic free energy The thermodynamic free energy is a concept useful in the thermodynamics of chemical or thermal processes in engineering and science. The change in the free energy is the maximum amount of work that a thermodynamic system can perform in a process a ...


References

{{DEFAULTSORT:History Of Entropy Thermodynamic entropy History of thermodynamics