Markov Chain

tldr

“If you know the probability of something going from state A to state B and you follow that and do it enough times, the result will be a distribution similar to the real-world result.”

A Markov chain is a type of mathematical model created by Andrey Markov, a Russian mathematician, that describes a system that transitions from one state to another, where the probability of moving to the next state depends only on the current state — not on the sequence of events that preceded it. This is known as the Markov property, or memorylessness.

What is a Markov Chain?

At its core, a Markov chain consists of:

• A finite (or countable) set of states — e.g., {Sunny, Rainy}, or {Happy, Sad, Angry}.

• Transition probabilities between those states — e.g., P(Sunny → Rainy) = 0.3.

• A transition matrix (if discrete), where each entry P(i,j) tells you the probability of going from state i to state j.

Example: Weather System

Let’s say you want to model weather changes:

• If today is Sunny, there’s an 80% chance tomorrow will also be Sunny, and 20% chance it’ll be Rainy.

• If today is Rainy, there’s a 60% chance tomorrow will also be Rainy, and 40% chance it’ll be Sunny.

This system can be modeled with a Markov chain because each day’s weather depends only on the previous day — not the week before.