|
Max-entropy
The Hartley function is a measure of uncertainty, introduced by Ralph Hartley in 1928. If a sample from a finite set ''A'' uniformly at random is picked, the information revealed after the outcome is known is given by the Hartley function : H_0(A) := \mathrm_b \vert A \vert , where denotes the cardinality of ''A''. If the base of the logarithm is 2, then the unit of uncertainty is the shannon (more commonly known as bit). If it is the natural logarithm, then the unit is the nat. Hartley used a base-ten logarithm, and with this base, the unit of information is called the hartley (aka ban or dit) in his honor. It is also known as the Hartley entropy. Hartley function, Shannon entropy, and Rényi entropy The Hartley function coincides with the Shannon entropy (as well as with the Rényi entropies of all orders) in the case of a uniform probability distribution. It is a special case of the Rényi entropy since: :H_0(X) = \frac 1 \log \sum_^ p_i^0 = \log , X, . But it can also b ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Uncertainty
Uncertainty refers to epistemic situations involving imperfect or unknown information. It applies to predictions of future events, to physical measurements that are already made, or to the unknown. Uncertainty arises in partially observable or stochastic environments, as well as due to ignorance, indolence, or both. It arises in any number of fields, including insurance, philosophy, physics, statistics, economics, finance, medicine, psychology, sociology, engineering, metrology, meteorology, ecology and information science. Concepts Although the terms are used in various ways among the general public, many specialists in decision theory, statistics and other quantitative fields have defined uncertainty, risk, and their measurement as: Uncertainty The lack of certainty, a state of limited knowledge where it is impossible to exactly describe the existing state, a future outcome, or more than one possible outcome. ;Measurement of uncertainty: A set of possible states or outc ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Ralph Hartley
Ralph Vinton Lyon Hartley (November 30, 1888 – May 1, 1970) was an American electronics researcher. He invented the Hartley oscillator and the Hartley transform, and contributed to the foundations of information theory. Biography Hartley was born in Sprucemont, Nevada, and attended the University of Utah, receiving an A.B. degree in 1909. He became a Rhodes Scholar at St Johns, Oxford University, in 1910 and received a B.A. degree in 1912 and a B.Sc. degree in 1913. He married Florence Vail of Brooklyn on March 21, 1916. The Hartleys had no children. He returned to the United States and was employed at the Research Laboratory of the Western Electric Company. In 1915 he was in charge of radio receiver development for the Bell System transatlantic radiotelephone tests. For this he developed the Hartley oscillator and also a neutralizing circuit to eliminate triode singing resulting from internal coupling. A patent for the oscillator was filed on June 1, 1915 and awarded on Octo ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Cardinality
In mathematics, the cardinality of a set is a measure of the number of elements of the set. For example, the set A = \ contains 3 elements, and therefore A has a cardinality of 3. Beginning in the late 19th century, this concept was generalized to infinite sets, which allows one to distinguish between different types of infinity, and to perform arithmetic on them. There are two approaches to cardinality: one which compares sets directly using bijections and injections, and another which uses cardinal numbers. The cardinality of a set is also called its size, when no confusion with other notions of size is possible. The cardinality of a set A is usually denoted , A, , with a vertical bar on each side; this is the same notation as absolute value, and the meaning depends on context. The cardinality of a set A may alternatively be denoted by n(A), , \operatorname(A), or \#A. History A crude sense of cardinality, an awareness that groups of things or events compare with other grou ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Base (exponentiation)
In exponentiation, the base is the number b in an expression of the form bn. Related terms The number n is called the exponent and the expression is known formally as exponentiation of b by n or the exponential of n with base b. It is more commonly expressed as "the nth power of b", "b to the nth power" or "b to the power n". For example, the fourth power of 10 is 10,000 because . The term ''power'' strictly refers to the entire expression, but is sometimes used to refer to the exponent. Radix is the traditional term for ''base'', but usually refers then to one of the common bases: decimal (10), binary (2), hexadecimal (16), or sexagesimal (60). When the concepts of variable and constant came to be distinguished, the process of exponentiation was seen to transcend the algebraic functions. In his 1748 ''Introductio in analysin infinitorum'', Leonhard Euler referred to "base a = 10" in an example. He referred to ''a'' as a "constant number" in an extensive consideration of the ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Logarithm
In mathematics, the logarithm is the inverse function to exponentiation. That means the logarithm of a number to the base is the exponent to which must be raised, to produce . For example, since , the ''logarithm base'' 10 of is , or . The logarithm of to ''base'' is denoted as , or without parentheses, , or even without the explicit base, , when no confusion is possible, or when the base does not matter such as in big O notation. The logarithm base is called the decimal or common logarithm and is commonly used in science and engineering. The natural logarithm has the number as its base; its use is widespread in mathematics and physics, because of its very simple derivative. The binary logarithm uses base and is frequently used in computer science. Logarithms were introduced by John Napier in 1614 as a means of simplifying calculations. They were rapidly adopted by navigators, scientists, engineers, surveyors and others to perform high-a ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Shannon (unit)
The shannon (symbol: Sh) is a unit of information named after Claude Shannon, the founder of information theory. IEC 80000-13 defines the shannon as the information content associated with an event when the probability of the event occurring is . It is understood as such within the realm of information theory, and is conceptually distinct from the bit, a term used in data processing and storage to denote a single instance of a binary signal. A sequence of ''n'' binary symbols (such as contained in computer memory or a binary data transmission) is properly described as consisting of ''n'' bits, but the information content of those ''n'' symbols may be more or less than ''n'' shannons according to the ''a priori'' probability of the actual sequence of symbols. The shannon also serves as a unit of the information entropy of an event, which is defined as the expected value of the information content of the event (i.e., the probability-weighted average of all potential events). Given a ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Natural Logarithm
The natural logarithm of a number is its logarithm to the base of the mathematical constant , which is an irrational and transcendental number approximately equal to . The natural logarithm of is generally written as , , or sometimes, if the base is implicit, simply . Parentheses are sometimes added for clarity, giving , , or . This is done particularly when the argument to the logarithm is not a single symbol, so as to prevent ambiguity. The natural logarithm of is the power to which would have to be raised to equal . For example, is , because . The natural logarithm of itself, , is , because , while the natural logarithm of is , since . The natural logarithm can be defined for any positive real number as the area under the curve from to (with the area being negative when ). The simplicity of this definition, which is matched in many other formulas involving the natural logarithm, leads to the term "natural". The definition of the natural logarithm can then b ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Nat (unit)
The natural unit of information (symbol: nat), sometimes also nit or nepit, is a unit of information, based on natural logarithms and powers of ''e'', rather than the powers of 2 and base 2 logarithms, which define the shannon. This unit is also known by its unit symbol, the nat. One nat is the information content of an event when the probability of that event occurring is 1/ ''e''. One nat is equal to shannons ≈ 1.44 Sh or, equivalently, hartleys ≈ 0.434 Hart. History Boulton and Wallace used the term ''nit'' in conjunction with minimum message length, which was subsequently changed by the minimum description length community to ''nat'' to avoid confusion with the nit used as a unit of luminance. Alan Turing used the ''natural ban''. Entropy Shannon entropy (information entropy), being the expected value of the information of an event, is a quantity of the same type and with the same units as information. The International System of Uni ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Base-ten Logarithm
In mathematics, the common logarithm is the logarithm with base 10. It is also known as the decadic logarithm and as the decimal logarithm, named after its base, or Briggsian logarithm, after Henry Briggs, an English mathematician who pioneered its use, as well as standard logarithm. Historically, it was known as ''logarithmus decimalis'' or ''logarithmus decadis''. It is indicated by , , or sometimes with a capital (however, this notation is ambiguous, since it can also mean the complex natural logarithmic multi-valued function). On calculators, it is printed as "log", but mathematicians usually mean natural logarithm (logarithm with base e ≈ 2.71828) rather than common logarithm when they write "log". To mitigate this ambiguity, the ISO 80000 specification recommends that should be written , and should be . Before the early 1970s, handheld electronic calculators were not available, and mechanical calculators capable of multiplication were bulky, expensive and not widely ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Hartley (unit)
The hartley (symbol Hart), also called a ban, or a dit (short for decimal digit), is a logarithmic unit that measures information or entropy, based on base 10 logarithms and powers of 10. One hartley is the information content of an event if the probability of that event occurring is . It is therefore equal to the information contained in one decimal digit (or dit), assuming ''a priori'' equiprobability of each possible value. It is named after Ralph Hartley. If base 2 logarithms and powers of 2 are used instead, then the unit of information is the shannon or bit, which is the information content of an event if the probability of that event occurring is . Natural logarithms and powers of e define the nat. One ban corresponds to ln(10) nat = log2(10) Sh, or approximately 2.303 nat, or 3.322 bit (3.322 Sh). A deciban is one tenth of a ban (or about 0.332 Sh); the name is formed from ''ban'' by the SI prefix ''deci-''. Though there is no associated SI unit, information ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Shannon Entropy
Shannon may refer to: People * Shannon (given name) * Shannon (surname) * Shannon (American singer), stage name of singer Shannon Brenda Greene (born 1958) * Shannon (South Korean singer), British-South Korean singer and actress Shannon Arrum Williams (born 1998) * Shannon, intermittent stage name of English singer-songwriter Marty Wilde (born 1939) * Claude Shannon (1916-2001) was American mathematician, electrical engineer, and cryptographer known as a "father of information theory" Places Australia * Shannon, Tasmania, a locality * Hundred of Shannon, a cadastral unit in South Australia * Shannon, a former name for the area named Calomba, South Australia since 1916 * Shannon River (Western Australia) Canada * Shannon, New Brunswick, a community * Shannon, Quebec, a city * Shannon Bay, former name of Darrell Bay, British Columbia * Shannon Falls, a waterfall in British Columbia Ireland * River Shannon, the longest river in Ireland ** Shannon Cave, a subterranean section o ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Rényi Entropy
In information theory, the Rényi entropy is a quantity that generalizes various notions of entropy, including Hartley entropy, Shannon entropy, collision entropy, and min-entropy. The Rényi entropy is named after Alfréd Rényi, who looked for the most general way to quantify information while preserving additivity for independent events. In the context of fractal dimension estimation, the Rényi entropy forms the basis of the concept of generalized dimensions. The Rényi entropy is important in ecology and statistics as index of diversity. The Rényi entropy is also important in quantum information, where it can be used as a measure of entanglement. In the Heisenberg XY spin chain model, the Rényi entropy as a function of can be calculated explicitly because it is an automorphic function with respect to a particular subgroup of the modular group. In theoretical computer science, the min-entropy is used in the context of randomness extractors. Definition The Rényi entro ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
