Probabilistic programming (PP) is a

programming paradigm Programming paradigms are a way to classify programming languages based on their features. Languages can be classified into multiple paradigms. Some paradigms are concerned mainly with implications for the execution model of the language, suc ...

in which

probabilistic model A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, ...

s are specified and inference for these models is performed automatically. It represents an attempt to unify probabilistic modeling and traditional general purpose programming in order to make the former easier and more widely applicable.Pfeffer, Avrom (2014), ''Practical Probabilistic Programming'', Manning Publications. p.28. It can be used to create systems that help make decisions in the face of uncertainty. Programming languages used for probabilistic programming are referred to as "probabilistic programming languages" (PPLs).

Applications

Probabilistic reasoning has been used for a wide variety of tasks such as predicting stock prices, recommending movies, diagnosing computers, detecting cyber intrusions and image detection. However, until recently (partially due to limited computing power), probabilistic programming was limited in scope, and most inference algorithms had to be written manually for each task. Nevertheless, in 2015, a 50-line probabilistic computer vision program was used to generate 3D models of human faces based on 2D images of those faces. The program used inverse graphics as the basis of its inference method, and was built using the Picture package in

Julia Julia is usually a feminine given name. It is a Latinate feminine form of the name Julio and Julius. (For further details on etymology, see the Wiktionary entry "Julius".) The given name ''Julia'' had been in use throughout Late Antiquity (e.g ...

. This made possible "in 50 lines of code what used to take thousands". The

Gen Gen may refer to: * ''Gen'' (film), 2006 Turkish horror film directed by Togan Gökbakar * Gen (Street Fighter), a video game character from the ''Street Fighter'' series * Gen Fu, a video game character from the ''Dead or Alive'' series * Gen l ...

probabilistic programming library (also written in Julia) has been applied to vision and robotics tasks. More recently, the probabilistic programming system Turing.jl has been applied in various pharmaceutical and economics applications. Probabilistic programming in Julia has also been combined with

differentiable programming Differentiable programming is a programming paradigm in which a numeric computer program can be differentiated throughout via automatic differentiation. This allows for gradient-based optimization of parameters in the program, often via grad ...

by combining the Julia package Zygote.jl with Turing.jl.

Probabilistic programming languages

PPLs often extend from a basic language. The choice of underlying basic language depends on the similarity of the model to the basic language's

ontology In metaphysics, ontology is the philosophical study of being, as well as related concepts such as existence, becoming, and reality. Ontology addresses questions like how entities are grouped into categories and which of these entities exi ...

, as well as commercial considerations and personal preference. For instance, Dimple and Chimple are based on

Java Java (; id, Jawa, ; jv, ꦗꦮ; su, ) is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea to the north. With a population of 151.6 million people, Java is the world's mos ...

Infer.NET Infer.NET is a free and open source .NET software library for machine learning. It supports running Bayesian inference in graphical models and can also be used for probabilistic programming. Overview Infer.NET follows a model-based approach an ...

is based on

.NET Framework The .NET Framework (pronounced as "''dot net"'') is a proprietary software framework developed by Microsoft that runs primarily on Microsoft Windows. It was the predominant implementation of the Common Language Infrastructure (CLI) until bein ...

, while PRISM extends from

Prolog Prolog is a logic programming language associated with artificial intelligence and computational linguistics. Prolog has its roots in first-order logic, a formal logic, and unlike many other programming languages, Prolog is intended primarily ...

. However, some PPLs such as

WinBUGS WinBUGS is statistical software for Bayesian analysis using Markov chain Monte Carlo (MCMC) methods. It is based on the BUGS ( Bayesian inference Using Gibbs Sampling) project started in 1989. It runs under Microsoft Windows, though it can als ...

offer a self-contained language, that maps closely to the mathematical representation of the statistical models, with no obvious origin in another programming language. The language for winBUGS was implemented to perform Bayesian computation using Gibbs Sampling (and related algorithms). Although implemented in a relatively old programming language (Pascal), this language permits Bayesian inference for a wide variety of statistical models using a flexible computational approach. The same BUGS language may be used to specify Bayesian models for inference via different computational choices ("samplers") and conventions or defaults, using a standalone package winBUGS (or related R packages, rbugs and r2winbugs) and JAGS (Just Another Gibbs Sampler, another R package). More recently, other languages to support Bayesian model specification and inference allow different or more efficient choices for the underlying Bayesian computation, and are accessible from the R data analysis and programming environment, e.g.: Stan, NIMBLE and NUTS. The influence of the BUGS language is evident in these later languages, which even use the same syntax for some aspects of model specification. Several PPLs are in active development, including some in beta test. Two popular tools are Stan and

PyMC PyMC (formerly known as PyMC3) is a Python package for Bayesian statistical modeling and probabilistic machine learning which focuses on advanced Markov chain Monte Carlo and variational fitting algorithms. It is a rewrite from scratch of the prev ...

Relational

A probabilistic relational programming language (PRPL) is a PPL specially designed to describe and infer with probabilistic relational models (PRMs). A PRM is usually developed with a set of algorithms for reducing, inference about and discovery of concerned distributions, which are embedded into the corresponding PRPL.

List of probabilistic programming languages

This list summarises the variety of PPLs that are currently available, and clarifies their origins.

Difficulty

Reasoning about variables as probability distributions causes difficulties for novice programmers, but these difficulties can be addressed through use of Bayesian network visualisations and graphs of variable distributions embedded within the source code editor.

Notes

{{Reflist, 30em, refs= {{cite book, url=https://dl.acm.org/citation.cfm?id=2627375, title=NOVA: A Functional Language for Data Parallelism, work=acm.org, series=Array'14, date=June 9, 2014, pages=8–13, doi=10.1145/2627373.2627375, isbn=9781450329378, s2cid=6748967 {{cite web, url=https://popl19.sigplan.org/event/lafi-2019-probabilistic-programming-with-cuppl, title=Probabilistic Programming with CuPPL, work=popl19.sigplan.org {{cite web, url=https://github.com/probcomp/bayeslite, title=BayesDB on SQLite. A Bayesian database table for querying the probable implications of data as easily as SQL databases query the data itself, work=GitHub, date=December 26, 2021 {{cite web, url=http://bayesloop.com/, title=bayesloop: Probabilistic programming framework that facilitates objective model selection for time-varying parameter models {{cite web, url=https://github.com/christophmark/bayesloop, title=GitHub -- bayesloop, website=

GitHub GitHub, Inc. () is an Internet hosting service for software development and version control using Git. It provides the distributed version control of Git plus access control, bug tracking, software feature requests, task management, continu ...

, date=December 7, 2021 {{cite web, url=http://www.analytica.com, title=Analytica-- A Probabilistic Modeling Language, work=lumina.com {{cite web, url=http://probcomp.csail.mit.edu/venture/, title=Venture -- a general-purpose probabilistic programming platform, work=mit.edu, access-date=September 20, 2014, archive-url=https://web.archive.org/web/20160125130827/http://probcomp.csail.mit.edu/venture/, archive-date=January 25, 2016, url-status=dead {{cite web, url=https://github.com/probprog/anglican-infcomp, title=The Anglican Probabilistic Programming System, work=ox.ac.uk, date=January 6, 2021 {{cite web, url=http://www.robots.ox.ac.uk/~brooks/probabilistic-c/, title=Probabilistic C, work=ox.ac.uk, access-date=March 24, 2015, archive-url=https://web.archive.org/web/20160104201746/http://www.robots.ox.ac.uk/~brooks/probabilistic-c/, archive-date=January 4, 2016, url-status=dead {{cite web, url=http://www.eecs.harvard.edu/~avi/IBAL/, title=IBAL Home Page, url-status=dead, archive-url=https://web.archive.org/web/20101226131239/http://www.eecs.harvard.edu/~avi/IBAL/, archive-date=December 26, 2010, df=mdy-all {{cite web, url=http://rjida.meijo-u.ac.jp/prism/, title=PRISM: PRogramming In Statistical Modeling, website=rjida.meijo-u.ac.jp, access-date=July 8, 2015, archive-url=https://web.archive.org/web/20150301155729/http://rjida.meijo-u.ac.jp/prism/, archive-date=March 1, 2015, url-status=dead {{cite web, url=http://research.microsoft.com/en-us/um/cambridge/projects/infernet/, title=Infer.NET, publisher=Microsoft, work=microsoft.com {{cite web, url=https://github.com/analog-garage/dimple, title=Dimple Home Page, work=analog.com, date=July 2, 2021 {{cite web, url=https://github.com/analog-garage/chimple, title=Chimple Home Page, work=analog.com, date=April 16, 2021 {{cite web, url=http://people.csail.mit.edu/milch/blog/, title=Bayesian Logic (BLOG), work=mit.edu, url-status=dead, archive-url=https://web.archive.org/web/20110616214423/http://people.csail.mit.edu/milch/blog/, archive-date=June 16, 2011, df=mdy-all {{cite web, url=https://github.com/MatthiasNickles/diff-SAT/, title=diff-SAT (probabilistic SAT/ASP), website=

, date=October 8, 2021 {{cite journal, title=PSQL: A query language for probabilistic relational data, doi=10.1016/S0169-023X(98)00015-9 , volume=28, journal=Data & Knowledge Engineering, pages=107–120, year = 1998, last1 = Dey, first1 = Debabrata, last2=Sarkar , first2=Sumit {{cite web, url=http://www.mrc-bsu.cam.ac.uk/bugs/, title=The BUGS Project - MRC Biostatistics Unit, work=cam.ac.uk, access-date=January 12, 2011, archive-url=https://web.archive.org/web/20140314080841/http://www.mrc-bsu.cam.ac.uk/bugs/, archive-date=March 14, 2014, url-status=dead {{cite web, url=http://code.google.com/p/factorie/, title=Factorie - Probabilistic programming with imperatively-defined factor graphs - Google Project Hosting, work=google.com {{cite web, url=http://code.google.com/p/pmtk3/, title=PMTK3 - probabilistic modeling toolkit for Matlab/Octave, version 3 - Google Project Hosting, work=google.com {{cite web, url=https://github.com/rmculpepper/gamble, title=gamble: Probabilistic Programming, first=Ryan, last=Culpepper, date=January 17, 2017, via=GitHub {{cite web, url=http://alchemy.cs.washington.edu/, title=Alchemy - Open Source AI, work=washington.edu {{cite web, url=http://www.dyna.org/, title=Dyna, website=www.dyna.org, access-date=January 12, 2011, archive-url=https://web.archive.org/web/20160117155947/http://dyna.org/, archive-date=January 17, 2016, url-status=dead {{cite web, url=http://www.cra.com/figaro, title=Charles River Analytics - Probabilistic Modeling Services, work=cra.com, date=February 9, 2017 {{cite web, url=http://projects.csail.mit.edu/church/wiki/Church, title=Church, work=mit.edu, access-date=April 8, 2013, archive-url=https://web.archive.org/web/20160114182510/http://projects.csail.mit.edu/church/wiki/Church, archive-date=January 14, 2016, url-status=dead {{cite web, url=http://dtai.cs.kuleuven.be/problog, title=ProbLog: Probabilistic Programming, website=dtai.cs.kuleuven.be {{cite web, url=http://www.probayes.com/fr/Bayesian-Programming-Book/downloads/, title=ProbaYes - Ensemble, nous valorisations vos données, author=ProbaYes, work=probayes.com, access-date=November 26, 2013, archive-url=https://web.archive.org/web/20160305000751/http://www.probayes.com/fr/Bayesian-Programming-Book/downloads/, archive-date=March 5, 2016, url-status=dead {{cite web, url=http://mc-stan.org/, archive-url=https://web.archive.org/web/20120903133321/http://mc-stan.org/, url-status=dead, archive-date=2012-09-03, title=Stan, work=mc-stan.org {{cite web, url=https://hakaru-dev.github.io/, title=Hakaru Home Page, work=hakaru-dev.github.io/ {{cite web, url=http://www.bali-phy.org/, title=BAli-Phy Home Page, work=bali-phy.org {{cite web, url=https://github.com/opcode81/ProbCog/wiki/Features, title=ProbCog, work=GitHub {{cite web, url=http://i.stanford.edu/hazy/tuffy/home, title=Tuffy: A Scalable Markov Logic Inference Engine, work=stanford.edu {{cite web, url=https://docs.pymc.io/en/v3/, title=PyMC, author=PyMC devs, work=pymc-devs.github.io {{cite web, url=https://bitbucket.org/piedenis/lea, title=Lea Home Page, work=bitbucket.org {{cite web, url=http://dippl.org/, title=WebPPL Home Page, work=github.com/probmods/webppl {{cite web, url=https://github.com/yebai/Turing.jl, title=The Turing language for probabilistic programming, website=

, date=December 28, 2021 {{cite web, url=http://topps.diku.dk/torbenm/troll.msp, title=Troll dice roller and probability calculator {{cite web, url=https://github.com/zz5013/pwCompiler, title=PWhile Compiler, work=GitHub, date=May 25, 2020 {{cite web, url=https://github.com/bradleygramhansen/PyLFPPL, title=LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models, work=ox.ac.uk, date=November 2, 2019 {{cite web, url=https://beanmachine.org, title=Bean Machine - A universal probabilistic programming language to enable fast and accurate Bayesian analysis, work=beanmachine.org

External links

List of Probabilistic Model Mini Language ToolkitsProbabilistic programming wiki
Probabilistic models Probabilistic software Programming paradigms