Pachinko allocation
   HOME

TheInfoList



OR:

In
machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine ...
and natural language processing, the pachinko allocation model (PAM) is a
topic model In statistics and natural language processing, a topic model is a type of statistical model for discovering the abstract "topics" that occur in a collection of documents. Topic modeling is a frequently used text-mining tool for discovery of hidden ...
. Topic models are a suite of algorithms to uncover the hidden thematic structure of a collection of documents. The algorithm improves upon earlier topic models such as
latent Dirichlet allocation In natural language processing, Latent Dirichlet Allocation (LDA) is a generative statistical model that explains a set of observations through unobserved groups, and each group explains why some parts of the data are similar. The LDA is an ex ...
(LDA) by modeling correlations between topics in addition to the word correlations which constitute topics. PAM provides more flexibility and greater expressive power than latent Dirichlet allocation. While first described and implemented in the context of natural language processing, the algorithm may have applications in other fields such as bioinformatics. The model is named for
pachinko is a mechanical game originating in Japan that is used as an arcade game, and much more frequently for gambling. Pachinko fills a niche in Japanese gambling comparable to that of the slot machine in the West as a form of low-stakes, low-st ...
machines—a game popular in Japan, in which metal balls bounce down around a complex collection of pins until they land in various bins at the bottom.


History

Pachinko allocation was first described by Wei Li and
Andrew McCallum Andrew McCallum is a professor in the computer science department at University of Massachusetts Amherst. His primary specialties are in machine learning, natural language processing, information extraction, information integration, and social ...
in 2006. The idea was extended with hierarchical Pachinko allocation by Li, McCallum, and David Mimno in 2007. In 2007, McCallum and his colleagues proposed a nonparametric Bayesian prior for PAM based on a variant of the hierarchical Dirichlet process (HDP). The algorithm has been implemented in the MALLET software package published by McCallum's group at the
University of Massachusetts Amherst The University of Massachusetts Amherst (UMass Amherst, UMass) is a public research university in Amherst, Massachusetts and the sole public land-grant university in Commonwealth of Massachusetts. Founded in 1863 as an agricultural college, ...
.


Model

PAM connects words in V and topics in T with an arbitrary
directed acyclic graph In mathematics, particularly graph theory, and computer science, a directed acyclic graph (DAG) is a directed graph with no directed cycles. That is, it consists of vertices and edges (also called ''arcs''), with each edge directed from one v ...
(DAG), where topic nodes occupy the interior levels and the leaves are words. The probability of generating a whole corpus is the product of the probabilities for every document: P(\mathbf, \alpha) = \prod_d P(d, \alpha)


See also

*
Probabilistic latent semantic indexing Probabilistic latent semantic analysis (PLSA), also known as probabilistic latent semantic indexing (PLSI, especially in information retrieval circles) is a statistical technique for the analysis of two-mode and co-occurrence data. In effect, one ca ...
(PLSI), an early topic model from Thomas Hofmann in 1999. *
Latent Dirichlet allocation In natural language processing, Latent Dirichlet Allocation (LDA) is a generative statistical model that explains a set of observations through unobserved groups, and each group explains why some parts of the data are similar. The LDA is an ex ...
, a generalization of PLSI developed by
David Blei David Meir Blei is a professor in the Statistics and Computer Science departments at Columbia University. Prior to fall 2014 he was an associate professor in the Department of Computer Science at Princeton University. His work is primarily in mach ...
,
Andrew Ng Andrew Yan-Tak Ng (; born 1976) is a British-born American computer scientist and technology entrepreneur focusing on machine learning and AI. Ng was a co-founder and head of Google Brain and was the former Chief Scientist at Baidu, building ...
, and Michael Jordan in 2002, allowing documents to have a mixture of topics. * MALLET, an open-source Java library that implements Pachinko allocation.


References


External links


Mixtures of Hierarchical Topics with Pachinko Allocation
a video recording of David Mimno presenting HPAM in 2007. Statistical natural language processing Latent variable models {{comp-sci-stub