Probabilistic Reasoning in AI
Understanding Uncertainty, Bayes' Theorem, Bayesian Networks, and Markov Decision Processes (MDPs).
Probabilistic Reasoning in AI
Artificial Intelligence often operates in environments that are partially observable, non-deterministic, or dynamic. In such scenarios, absolute certainty is impossible. Probabilistic reasoning provides a formal mathematical framework for managing uncertainty and making rational decisions based on available evidence.
Uncertainty in AI
Uncertainty arises from various sources, including noisy sensor data, incomplete information, and inherent stochasticity in the environment.
Probabilistic models allow AI agents to express their degree of belief in various states of the world, represented as values between 0 and 1.
Bayes' Theorem
At the core of probabilistic reasoning is Bayes' Theorem, which describes how to update the probability of a hypothesis as more evidence or information becomes available.
Where:
- is the posterior probability: the probability of hypothesis A given evidence B.
- is the likelihood: the probability of evidence B given that hypothesis A is true.
- is the prior probability: the initial probability of hypothesis A before observing evidence.
- is the marginal likelihood: the total probability of observing the evidence.
Bayesian Networks
Bayesian Networks, also known as belief networks, are directed acyclic graphs (DAGs) that represent a set of variables and their conditional dependencies.
- Nodes: Represent random variables (e.g., Burglary, Earthquake).
- Edges: Represent conditional dependencies.
Bayesian Networks dramatically reduce the computational complexity of representing joint probability distributions by exploiting conditional independence between variables.
Markov Decision Processes (MDPs)
While Bayesian Networks model beliefs, Markov Decision Processes (MDPs) provide a framework for decision-making under uncertainty.
An MDP is defined by a tuple \{S, A, P, R, \gamma\}:
| Component | Symbol | Description |
|---|---|---|
| States | S | A finite set of states in the environment. |
| Actions | A | A set of available actions the agent can take. |
| Transitions | P | is the probability of moving to state after taking action in state . |
| Rewards | R | is the immediate reward received after transitioning. |
| Discount | A factor between 0 and 1 that discounts future rewards. |
The goal of solving an MDP is to find an optimal policy, , which maps states to actions in a way that maximizes the expected cumulative reward.
The Markov Property states that the future is independent of the past given the present. This means the transition probabilities depend only on the current state and action.