March 2026

Modeling Math Stochastic Processes

An Introduction to Master Equations

This post references material from TAME course materials by Laurent Hébert-Dufresne and Guillaume St-Onge.

1. What is $P_n(t)$ ?

Consider a system whose condition can be summarized by a single integer $n$ — the count of particles, infected agents, voters holding a given opinion, and so on. Define $P_n(t)$ as the probability that the system is in state $n$ at time $t$ .

A useful mental image: imagine running infinitely many identical but independent copies of the system in parallel. At any moment, $P_n(t)$ is simply the fraction of those copies that find themselves in state $n$ . We are not following a single trajectory — we are following the entire distribution of trajectories.

Master equations vs. compartmental models Compartmental models (SIR, SIS, etc.) split the parts of a system into boxes. Master equations split a multiverse of whole systems into boxes. Compartmental models lose information about correlations between parts; master equations track the exact state of the system and so lose nothing — at the cost of requiring many more equations.

To model any system with master equations, three questions must be answered before writing a single equation:

Q1
What are the available states? Define the full set of values $n$ can take.
Q2
What transitions are possible? Which pairs of states are connected by an event?
Q3
At what rates do they occur? Assign a transition rate to every arrow.

2. Conservation of probability

The distribution $P_n(t)$ must satisfy normalization at every instant:

\sum_n P_n(t) = 1

Because this holds for all time, the sum is a constant. The derivative of a constant is zero, so differentiating both sides gives:

\sum_n \frac{dP_n}{dt} = 0

This is a powerful consistency check. In a master equation, every probability current $J_{n \to m}$ appears exactly twice across the full system — once as a loss from state $n$ , and once as a gain to state $n$ . The two contributions cancel in the sum, so the constraint is satisfied automatically by any correctly written master equation.

Practical use If you sum all your master equations and the result is not zero, you have a missing transition somewhere — probability is leaking out of (or flooding into) your state space. This sum-to-zero check is the first thing to run when debugging a new model.

3. The bookkeeping principle

The master equation for state $n$ is built entirely from probability currents in and out of that state:

ddtPn(t)=∑m≠nJn→m(t)+Jm→n(t)\frac{d}{dt}P_n(t) = \sum_{m \neq n} J_{n \to m}(t) + J_{m \to n}(t)dtd​Pn​(t)=m=n∑​Jn→m​(t)+Jm→n​(t)

In practice this reduces to a simple rule that applies to every state in every model:

01
Leaving $P_n$ in any direction → loss. Subtract the transition rate multiplied by $P_n$ .
02
Entering $P_n$ from any direction → gain. Add the transition rate (evaluated at the originating state) multiplied by the neighboring $P_{n+1}$ or $P_{n-1}$ .
03
Respect boundaries. If a neighboring state does not exist (e.g., $P_{-1}$ when $n=0$ ), that term simply drops out. The boundary is enforced by the physics, not by adding extra rules.

In general notation:

dPndt=∑mR(m→n) Pm  −  ∑mR(n→m) Pn\frac{dP_n}{dt}
    = \sum_m R(m \to n)\,P_m
    \;-\; \sum_m R(n \to m)\,P_ndtdPn​​=m∑​R(m→n)Pm​−m∑​R(n→m)Pn​

The pattern for every term Every term in a master equation follows the same three-part structure:

(±) × rate(originating state) × P(originating state)

The sign is determined by direction of flow relative to the state you are writing the equation for — negative if probability is leaving, positive if it is arriving. The rate and the distribution are both evaluated at the state the probability is departing from, not where it is going.

4. Worked example: Birth-death process

Particles are created at a constant rate $\mu$ from a reservoir and each existing particle disappears independently at rate $\nu$ . This is the simplest non-trivial system that exhibits both boundaries and state-dependent rates, making it an ideal first example.

Answering the three questions:

Question	Answer
States	$n \in \{0, 1, 2, \ldots\}$
Transitions	$n \to n+1$ (birth), $n \to n-1$ (death, only if $n > 0$ )
Rates	Birth: $\mu$ (constant, from reservoir). Death: $n\nu$ (each of the $n$ particles dies at rate $\nu$ ).

The death rate $n\nu$ deserves a moment's attention. Each particle dies independently, so with $n$ particles present there are $n$ independent chances for a death event. The total rate is $\nu$ summed $n$ times. Birth, by contrast, comes from an external reservoir rather than from the particles themselves, so its rate is a flat $\mu$ regardless of $n$ .

J_{n \to n+1} = \mu, \qquad J_{n > 0 \to n-1} = n\nu

Applying the bookkeeping principle to each state gives two cases. When $n=0$ the state below does not exist — there are no particles to kill — so the $\mu P_{n-1}$ gain term has nothing to attach to and drops out entirely:

\frac{d}{dt}P_0 = -\mu P_0 + \nu P_1

Read term by term: $-\mu P_0$ is the loss from a birth event pushing the system up to state 1; $+\nu P_1$ is the gain from the system being at state 1 and its single particle dying. The $P_1$ term is a gain to state 0 — it originates elsewhere and arrives here. For all $n > 0$ :

ddtPn=−(μ+nν) Pn+(n+1)ν Pn+1+μ Pn−1n>0\frac{d}{dt}P_n = -(\mu + n\nu)\,P_n + (n+1)\nu\,P_{n+1} + \mu\,P_{n-1} \qquad n > 0dtd​Pn​=−(μ+nν)Pn​+(n+1)νPn+1​+μPn−1​n>0

The boundary enforces itself. Notice the death rate

n\nu

naturally becomes zero at

n=0

, so no special case would even be needed for that term. The separate

n=0

equation is only necessary because the

\mu P_{n-1}

gain term references a state that does not exist, not because the death terms misbehave.

5. Example: The Voter Model

$N$ agents each hold one of two opinions: For (F) or Against (A). At each step, a randomly chosen agent copies a randomly chosen neighbor. Let $n$ = number of F-agents.

To move from $n$ to $n-1$ , an F-agent must be chosen and select an A-neighbor. The rates are:

R(n \to n-1) = n\cdot\frac{N-n}{N}, \qquad R(n \to n+1) = (N-n)\cdot\frac{n}{N}

Both rates are symmetric in $n$ and $N-n$ — the dynamics treat F and A identically. The full master equation:

dPndt=−n(N−n)NPn−(N−n)nNPn+(n+1)(N−n−1)NPn+1+(N−n+1)(n−1)NPn−1\frac{dP_n}{dt}
    = -\frac{n(N-n)}{N}P_n
    - \frac{(N-n)n}{N}P_n
    + \frac{(n+1)(N-n-1)}{N}P_{n+1}
    + \frac{(N-n+1)(n-1)}{N}P_{n-1}dtdPn​​=−Nn(N−n)​Pn​−N(N−n)n​Pn​+N(n+1)(N−n−1)​Pn+1​+N(N−n+1)(n−1)​Pn−1​

Why divide by $N$ ? The division converts a neighbor count into a probability. Out of all

N

possible neighbors, a fraction

n/N

are A-type. This is the approximate in AME — the exact version uses

n/(N-1)

(excluding self), valid for large

N

6. Example: SIS Epidemic Dynamics

$N$ agents are either Susceptible (S) or Infectious (I). Infection spreads at per-contact rate $\beta$ ; infected agents recover at rate $\alpha$ . Let $n$ = number of infectious agents. The system is bounded: $n=0$ is a valid absorbing state (extinction) and $n$ cannot exceed $N$ .

The key structural difference from the Voter Model: $\beta$ already encodes the per-contact rate, so we count all S–I pairs directly (mass-action) rather than sampling one neighbor. There is no division by $N$ . The rates are:

R(n \to n+1) = \beta(N-n)n, \qquad R(n \to n-1) = n\alpha

Applying the bookkeeping principle with boundary conditions $P_{-1} = P_{N+1} = 0$ :

dPndt=−nα Pn−β(N−n)n Pn+(n+1)α Pn+1+β(N−n+1)(n−1) Pn−1\frac{dP_n}{dt}
    = -n\alpha\, P_n
    - \beta(N-n)n\, P_n
    + (n+1)\alpha\, P_{n+1}
    + \beta(N-n+1)(n-1)\,P_{n-1}dtdPn​​=−nαPn​−β(N−n)nPn​+(n+1)αPn+1​+β(N−n+1)(n−1)Pn−1​

Keep the directions straight. Infection (rate

\beta

) increases

n

— it appears in transitions that move the system up. Recovery (rate

\alpha

) decreases

n

— it appears in transitions that move the system down. Track which process moves the count in which direction and the relabeling follows mechanically.

The mean-field limit recovers the familiar ODE by replacing the full distribution with its average:

\frac{dI}{dt} = \beta(N - I)I - \alpha I

The threshold $\mathcal{R}_0 = \beta/\alpha$ separates extinction from endemic persistence — a result the master equation approach can derive more rigorously by examining the absorbing boundary at $n = 0$ .

1. What is Pn(t)P_n(t)Pn​(t)?

2. Conservation of probability

3. The bookkeeping principle

4. Worked example: Birth-death process

5. Example: The Voter Model

6. Example: SIS Epidemic Dynamics

1. What is $P_n(t)$ ?