|
Semi-Markov Process
In probability and statistics, a Markov renewal process (MRP) is a random process that generalizes the notion of Markov jump processes. Other random processes like Markov chains, Poisson processes and renewal processes can be derived as special cases of MRP's. Definition Consider a state space \mathrm. Consider a set of random variables (X_n,T_n), where T_n are the jump times and X_n are the associated states in the Markov chain (see Figure). Let the inter-arrival time, \tau_n=T_n-T_. Then the sequence (X_n,T_n) is called a Markov renewal process if : \begin & \Pr(\tau_\le t, X_=j\mid(X_0, T_0), (X_1, T_1),\ldots, (X_n=i, T_n)) \\ pt= & \Pr(\tau_\le t, X_=j\mid X_n=i)\, \forall n \ge1,t\ge0, i,j \in \mathrm \end Relation to other stochastic processes # If we define a new stochastic process Y_t:=X_n for t \in _n,T_), then the process Y_t is called a semi-Markov process. Note the main difference between an MRP and a semi-Markov process is that the former is defined as a tw ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Stochastic Process
In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that appear to vary in a random manner. Examples include the growth of a bacterial population, an electrical current fluctuating due to thermal noise, or the movement of a gas molecule. Stochastic processes have applications in many disciplines such as biology, chemistry, ecology, neuroscience, physics, image processing, signal processing, control theory, information theory, computer science, cryptography and telecommunications. Furthermore, seemingly random changes in financial markets have motivated the extensive use of stochastic processes in finance. Applications and the study of phenomena have in turn inspired the proposal of new stochastic processes. Examples of such stochastic processes include the Wiener process or Brownia ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Hidden Semi-Markov Model
A hidden semi-Markov model (HSMM) is a statistical model with the same structure as a hidden Markov model except that the unobservable process is semi-Markov rather than Markov Markov ( Bulgarian, russian: Марков), Markova, and Markoff are common surnames used in Russia and Bulgaria. Notable people with the name include: Academics *Ivana Markova (born 1938), Czechoslovak-British emeritus professor of psychology at .... This means that the probability of there being a change in the hidden state depends on the amount of time that has elapsed since entry into the current state. This is in contrast to hidden Markov models where there is a constant probability of changing state given survival in the state up to that time. For instance modelled daily rainfall using a hidden semi-Markov model. If the underlying process (e.g. weather system) does not have a geometrically distributed duration, an HSMM may be more appropriate. Hidden semi-Markov models can be used in implementati ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Variable-order Markov Model
In the mathematical theory of stochastic processes, variable-order Markov (VOM) models are an important class of models that extend the well known Markov chain models. In contrast to the Markov chain models, where each random variable in a sequence with a Markov property depends on a fixed number of random variables, in VOM models this number of conditioning random variables may vary based on the specific observed realization. This realization sequence is often called the ''context''; therefore the VOM models are also called ''context trees''. VOM models are nicely rendered by colorized probabilistic suffix trees (PST). The flexibility in the number of conditioning random variables turns out to be of real advantage for many applications, such as statistical analysis, classification and prediction. Example Consider for example a sequence of random variables, each of which takes a value from the ternary alphabet . Specifically, consider the string ' constructed from infinite concate ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Renewal Theory
Renewal theory is the branch of probability theory that generalizes the Poisson process for arbitrary holding times. Instead of exponentially distributed holding times, a renewal process may have any independent and identically distributed (IID) holding times that have finite mean. A renewal-reward process additionally has a random sequence of rewards incurred at each holding time, which are IID but need not be independent of the holding times. A renewal process has asymptotic properties analogous to the strong law of large numbers and central limit theorem. The renewal function m(t) (expected number of arrivals) and reward function g(t) (expected reward value) are of key importance in renewal theory. The renewal function satisfies a recursive integral equation, the renewal equation. The key renewal equation gives the limiting value of the convolution of m'(t) with a suitable non-negative function. The superposition of renewal processes can be studied as a special case of Markov re ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Markov Process
A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs ''now''." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC). It is named after the Russian mathematician Andrey Markov. Markov chains have many applications as statistical models of real-world processes, such as studying cruise control systems in motor vehicles, queues or lines of customers arriving at an airport, currency exchange rates and animal population dynamics. Markov processes are the basis for general stochastic simulation methods known as Markov chain Monte Carlo, which are used for simulating sampling from complex probabilit ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
DTMC
In probability, a discrete-time Markov chain (DTMC) is a sequence of random variables, known as a stochastic process, in which the value of the next variable depends only on the value of the current variable, and not any variables in the past. For instance, a machine may have two states, ''A'' and ''E''. When it is in state ''A'', there is a 40% chance of it moving to state ''E'' and a 60% chance of it remaining in state ''A''. When it is in state ''E'', there is a 70% chance of it moving to ''A'' and a 30% chance of it staying in ''E''. The sequence of states of the machine is a Markov chain. If we denote the chain by X_0, X_1, X_2, ... then X_0 is the state which the machine starts in and X_ is the random variable describing its state after 10 transitions. The process continues forever, indexed by the natural numbers. An example of a stochastic process which is not a Markov chain is the model of a machine which has states ''A'' and ''E'' and moves to ''A'' from either state w ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
CTMC
A continuous-time Markov chain (CTMC) is a continuous stochastic process in which, for each state, the process will change state according to an exponential random variable and then move to a different state as specified by the probabilities of a stochastic matrix. An equivalent formulation describes the process as changing state according to the least value of a set of exponential random variables, one for each possible state it can move to, with the parameters determined by the current state. An example of a CTMC with three states \ is as follows: the process makes a transition after the amount of time specified by the holding time—an exponential random variable E_i, where ''i'' is its current state. Each random variable is independent and such that E_0\sim \text(6), E_1\sim \text(12) and E_2\sim \text(18). When a transition is to be made, the process moves according to the jump chain, a discrete-time Markov chain with stochastic matrix: :\begin 0 & \frac & \frac \\ \frac & ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Exponential Distribution
In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average rate. It is a particular case of the gamma distribution. It is the continuous analogue of the geometric distribution, and it has the key property of being memoryless. In addition to being used for the analysis of Poisson point processes it is found in various other contexts. The exponential distribution is not the same as the class of exponential families of distributions. This is a large class of probability distributions that includes the exponential distribution as one of its members, but also includes many other distributions, like the normal, binomial, gamma, and Poisson distributions. Definitions Probability density function The probability density function (pdf) of an exponential distribution is : f(x;\lambda) = \begin \lam ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Continuous-time Markov Process
A continuous-time Markov chain (CTMC) is a continuous stochastic process in which, for each state, the process will change state according to an exponential random variable and then move to a different state as specified by the probabilities of a stochastic matrix. An equivalent formulation describes the process as changing state according to the least value of a set of exponential random variables, one for each possible state it can move to, with the parameters determined by the current state. An example of a CTMC with three states \ is as follows: the process makes a transition after the amount of time specified by the holding time—an exponential random variable E_i, where ''i'' is its current state. Each random variable is independent and such that E_0\sim \text(6), E_1\sim \text(12) and E_2\sim \text(18). When a transition is to be made, the process moves according to the jump chain, a discrete-time Markov chain with stochastic matrix: :\begin 0 & \frac & \frac \\ \frac & ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Markov Property
In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process. It is named after the Russian mathematician Andrey Markov. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time. The term Markov assumption is used to describe a model where the Markov assumption is assumed to hold, such as a hidden Markov model. A Markov random field extends this property to two or more dimensions or to random variables defined for an interconnected network of items. An example of a model for such a field is the Ising model. A discrete-time stochastic process satisfying the Markov property is known as a Markov chain. Introduction A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the presen ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Tuple
In mathematics, a tuple is a finite ordered list (sequence) of elements. An -tuple is a sequence (or ordered list) of elements, where is a non-negative integer. There is only one 0-tuple, referred to as ''the empty tuple''. An -tuple is defined inductively using the construction of an ordered pair. Mathematicians usually write tuples by listing the elements within parentheses "" and separated by a comma and a space; for example, denotes a 5-tuple. Sometimes other symbols are used to surround the elements, such as square brackets " nbsp; or angle brackets "⟨ ⟩". Braces "" are used to specify arrays in some programming languages but not in mathematical expressions, as they are the standard notation for sets. The term ''tuple'' can often occur when discussing other mathematical objects, such as vectors. In computer science, tuples come in many forms. Most typed functional programming languages implement tuples directly as product types, tightly associated with alge ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Marked Point Process
In linguistics and social sciences, markedness is the state of standing out as nontypical or divergent as opposed to regular or common. In a marked–unmarked relation, one term of an opposition is the broader, dominant one. The dominant default or minimum-effort form is known as ''unmarked''; the other, secondary one is ''marked''. In other words, markedness involves the characterization of a "normal" linguistic unit against one or more of its possible "irregular" forms. In linguistics, markedness can apply to, among others, phonological, grammatical, and semantic oppositions, defining them in terms of marked and unmarked oppositions, such as ''honest'' (unmarked) vs. ''dishonest'' (marked). Marking may be purely semantic, or may be realized as extra morphology. The term derives from the marking of a grammatical role with a suffix or another element, and has been extended to situations where there is no morphological distinction. In social sciences more broadly, markedness ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
