The decentralized partially observable Markov decision process (Dec-POMDP) is a model for coordination and
decision-making
In psychology, decision-making (also spelled decision making and decisionmaking) is regarded as the cognitive process resulting in the selection of a belief or a course of action among several possible alternative options. It could be either r ...
among multiple agents. It is a
probabilistic
Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speaking, ...
model that can consider
uncertainty
Uncertainty refers to Epistemology, epistemic situations involving imperfect or unknown information. It applies to predictions of future events, to physical measurements that are already made, or to the unknown. Uncertainty arises in partially ...
in outcomes, sensors and communication (i.e., costly, delayed, noisy or nonexistent communication).
It is a generalization of a
Markov decision process (MDP) and a
partially observable Markov decision process
A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process (MDP). A POMDP models an agent decision process in which it is assumed that the system dynamics are determined by an MDP, but the agent cannot ...
(POMDP) to consider multiple decentralized agents.
Definition
Formal definition
A Dec-POMDP is a 7-tuple
, where
*
is a set of states,
*
is a set of actions for agent ''i'', with
is the set of joint actions,
*
is a set of conditional transition probabilities between states,
,
*
is the reward function.
*
is a set of observations for agent ''i'', with
is the set of joint observations,
*
is a set of conditional observation probabilities
, and
*