|
Entropy Estimation
In various science/engineering applications, such as independent component analysis, image analysis, genetic analysis, speech recognition, manifold learning, and time delay estimationBenesty, J.; Yiteng Huang; Jingdong Chen (2007) Time Delay Estimation via Minimum Entropy. In ''Signal Processing Letters'', Volume 14, Issue 3, March 2007 157–160 it is useful to estimate the differential entropy of a system or process, given some observations. The simplest and most common approach uses histogram-based estimation, but other approaches have been developed and used, each with its own benefits and drawbacks.J. Beirlant, E. J. Dudewicz, L. Gyorfi, and E. C. van der Meulen (1997Nonparametric entropy estimation: An overview In ''International Journal of Mathematical and Statistical Sciences'', Volume 6, pp. 17– 39. The main factor in choosing a method is often a trade-off between the bias and the variance of the estimate,T. Schürmann, Bias analysis in entropy estimation. In ''J. P ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Independent Component Analysis
In signal processing, independent component analysis (ICA) is a computational method for separating a multivariate signal into additive subcomponents. This is done by assuming that at most one subcomponent is Gaussian and that the subcomponents are statistically independent from each other. ICA is a special case of blind source separation. A common example application is the "cocktail party problem" of listening in on one person's speech in a noisy room. Introduction Independent component analysis attempts to decompose a multivariate signal into independent non-Gaussian signals. As an example, sound is usually a signal that is composed of the numerical addition, at each time t, of signals from several sources. The question then is whether it is possible to separate these contributing sources from the observed total signal. When the statistical independence assumption is correct, blind ICA separation of a mixed signal gives very good results. It is also used for signals that are ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Weighted Sum
A weight function is a mathematical device used when performing a sum, integral, or average to give some elements more "weight" or influence on the result than other elements in the same set. The result of this application of a weight function is a weighted sum or weighted average. Weight functions occur frequently in statistics and analysis, and are closely related to the concept of a measure. Weight functions can be employed in both discrete and continuous settings. They can be used to construct systems of calculus called "weighted calculus" and "meta-calculus".Jane Grossma''Meta-Calculus: Differential and Integral'' , 1981. Discrete weights General definition In the discrete setting, a weight function w \colon A \to \R^+ is a positive function defined on a discrete set A, which is typically finite or countable. The weight function w(a) := 1 corresponds to the ''unweighted'' situation in which all elements have equal weight. One can then apply this weight to various concep ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Information Theory
Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering (field), information engineering, and electrical engineering. A key measure in information theory is information entropy, entropy. Entropy quantifies the amount of uncertainty involved in the value of a random variable or the outcome of a random process. For example, identifying the outcome of a fair coin flip (with two equally likely outcomes) provides less information (lower entropy) than specifying the outcome from a roll of a dice, die (with six equally likely outcomes). Some other important measures in information theory are mutual informat ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Entropy And Information
Entropy is a scientific concept, as well as a measurable physical property, that is most commonly associated with a state of disorder, randomness, or uncertainty. The term and the concept are used in diverse fields, from classical thermodynamics, where it was first recognized, to the microscopic description of nature in statistical physics, and to the principles of information theory. It has found far-ranging applications in chemistry and physics, in biological systems and their relation to life, in cosmology, economics, sociology, weather science, climate change, and information systems including the transmission of information in telecommunication. The thermodynamic concept was referred to by Scottish scientist and engineer William Rankine in 1850 with the names ''thermodynamic function'' and ''heat-potential''. In 1865, German physicist Rudolf Clausius, one of the leading founders of the field of thermodynamics, defined it as the quotient of an infinitesimal amount of heat ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Markov Chain
A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs ''now''." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC). It is named after the Russian mathematician Andrey Markov. Markov chains have many applications as statistical models of real-world processes, such as studying cruise control systems in motor vehicles, queues or lines of customers arriving at an airport, currency exchange rates and animal population dynamics. Markov processes are the basis for general stochastic simulation methods known as Markov chain Monte Carlo, which are used for simulating sampling from complex probability dist ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Dirichlet Distribution
In probability and statistics, the Dirichlet distribution (after Peter Gustav Lejeune Dirichlet), often denoted \operatorname(\boldsymbol\alpha), is a family of continuous multivariate probability distributions parameterized by a vector \boldsymbol\alpha of positive reals. It is a multivariate generalization of the beta distribution, (Chapter 49: Dirichlet and Inverted Dirichlet Distributions) hence its alternative name of multivariate beta distribution (MBD). Dirichlet distributions are commonly used as prior distributions in Bayesian statistics, and in fact, the Dirichlet distribution is the conjugate prior of the categorical distribution and multinomial distribution. The infinite-dimensional generalization of the Dirichlet distribution is the ''Dirichlet process''. Definitions Probability density function The Dirichlet distribution of order ''K'' ≥ 2 with parameters ''α''1, ..., ''α''''K'' > 0 has a probability density function with respect to Lebesgue m ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
William Bialek
William Samuel Bialek (born 1960, in Los Angeles, California) is a theoretical biophysicist and a professor at Princeton University and The Graduate Center, CUNY. Much of his work, which has ranged over a wide variety of theoretical problems at the interface of physics and biology, centers around whether various functions of living beings are optimal, and (if so) whether a precise quantification of their performance approaches limits set by basic physical principles. Best known among these is an influential series of studies applying the principles of information theory to the analysis of the neural encoding of information in the nervous system, showing that aspects of brain function can be described as essentially optimal strategies for adapting to the complex dynamics of the world, making the most of the available signals in the face of fundamental physical constraints and limitations. Bialek received his AB (1979) and PhD (1983) degrees in Biophysics from the University of Cali ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Ilya Nemenman
Ilya Mark Nemenman (born January 8, 1975 in Minsk, Belarus) is a theoretical physicist at Emory University, where he is a Winship Distinguished Research Professor of Physics and Biology. He is known for his studies of information processing in biological systems and for developing coarse-grained models of these systems. He is a Fellow of the American Physical Society for "his contributions to theoretical biological physics, especially information processing in a variety of living systems, and for the development of coarse-grained modeling methods of such systems". He is a Simons Investigator and James S. McDonnell Foundation Complex Systems Scholar. He also served in the Chair Line of the Division of Biological Physics of the American Physical Society, from 2013–2018. Nemenman also was a founder of the q-bio conference, and is a general member of the Aspen Center for Physics. Life Ilya Nemenman is the son of Mark Nemenman, a Soviet computer scientist. He studied physics at the Be ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Bayesian Estimator
In estimation theory and decision theory, a Bayes estimator or a Bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function (i.e., the posterior expected loss). Equivalently, it maximizes the posterior expectation of a utility function. An alternative way of formulating an estimator within Bayesian statistics is maximum a posteriori estimation. Definition Suppose an unknown parameter \theta is known to have a prior distribution \pi. Let \widehat = \widehat(x) be an estimator of \theta (based on some measurements ''x''), and let L(\theta,\widehat) be a loss function, such as squared error. The Bayes risk of \widehat is defined as E_\pi(L(\theta, \widehat)), where the expectation is taken over the probability distribution of \theta: this defines the risk function as a function of \widehat. An estimator \widehat is said to be a ''Bayes estimator'' if it minimizes the Bayes risk among all estimators. Equivalently, the estimator whic ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Nearest Neighbour Distribution
In probability and statistics, a nearest neighbor function, nearest neighbor distance distribution,A. Baddeley, I. Bárány, and R. Schneider. Spatial point processes and their applications. ''Stochastic Geometry: Lectures given at the CIME Summer School held in Martina Franca, Italy, September 13–18, 2004'', pages 1–75, 2007. nearest-neighbor distribution function or nearest neighbor distribution is a mathematical function that is defined in relation to mathematical objects known as point processes, which are often used as mathematical models of physical phenomena representable as randomly positioned points in time, space or both.D. Stoyan, W. S. Kendall, J. Mecke, and L. Ruschendorf. ''Stochastic geometry and its applications'', volume 2. Wiley Chichester, 1995.D. J. Daley and D. Vere-Jones. ''An introduction to the theory of point processes. Vol. I''. Probability and its Applications (New York). Springer, New York, second edition, 2003. More specifically, nearest neighbor fun ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
International Conference On Acoustics, Speech, And Signal Processing
ICASSP, the International Conference on Acoustics, Speech, and Signal Processing, is an annual flagship conference organized of IEEE Signal Processing Society. All papers included in its proceedings have been indexed by Ei Compendex. The first ICASSP was held in 1976 in Philadelphia, Pennsylvania based on the success of a conference in Massachusetts four years earlier that had focused specifically on speech signals. As ranked by Google Scholar's h-index The ''h''-index is an author-level metric that measures both the productivity and citation impact of the publications, initially used for an individual scientist or scholar. The ''h''-index correlates with obvious success indicators such as winn ... metric in 2016, ICASSP has the highest h-index of any conference in Signal Processing field. Also, It is considered a high level conference in signal processing and, for example, obtained an 'A1' rating from the Brazilian ministry of education based on its H-index. References ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers is spread out from their average value. Variance has a central role in statistics, where some ideas that use it include descriptive statistics, statistical inference, hypothesis testing, goodness of fit, and Monte Carlo sampling. Variance is an important tool in the sciences, where statistical analysis of data is common. The variance is the square of the standard deviation, the second central moment of a distribution, and the covariance of the random variable with itself, and it is often represented by \sigma^2, s^2, \operatorname(X), V(X), or \mathbb(X). An advantage of variance as a measure of dispersion is that it is more amenable to algebraic manipulation than other measures of dispersion such as the expected absolute deviation; for e ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
