Markov Decision Process

---

A state $S_t$ is Markov if and only if

$$\mathbb P[S_{t+1} | S_t] = \mathbb P [S_{t+1} | S_1, ...,S_t]$$