Key idea: the past and future data are independent, conditioned on the present.

The Markovian assumption can be violated in many ways:

  1. Unmodelled dynamics in the environment
  2. Inaccuracies in probabilistic models
  3. Approximation errors when using approximate representations of belief functions