Markov Chain

tldr

“If you know the probability of something going from state A to state B and you follow that and do it enough times, the result will be a distribution similar to the real-world result.”

A Markov chain is a type of mathematical model created by Andrey Markov, a Russian mathematician, that describes a system that transitions from one state to another, where the probability of moving to the next state depends only on the current state, not on the sequence of events that preceded it. This is known as the Markov property, or memorylessness.

What is a Markov Chain?

At its core, a Markov chain consists of:

• A finite (or countable) set of states: e.g., {Sunny, Rainy}, or {Happy, Sad, Angry}.
• Transition probabilities between those states: e.g., P(Sunny → Rainy) = 0.3.
• A transition matrix (if discrete), where each entry P(i,j) tells you the probability of going from state i to state j.

Example: Weather System

Let’s say you want to model weather changes:

• If today is Sunny, there’s an 80% chance tomorrow will also be Sunny, and 20% chance it’ll be Rainy.
• If today is Rainy, there’s a 60% chance tomorrow will also be Rainy, and 40% chance it’ll be Sunny.

This system can be modeled with a Markov chain because each day’s weather depends only on the previous day, not the week before.