Understanding Markov Processes and their Representations

Markov processes, named after the Russian mathematician Andrey Markov, represent a class of stochastic processes in which the future state of a process depends solely on the present state, irrespective of how we arrived at this state. In other words, the key characteristic of a Markov process is the property of "memorylessness", which emphasizes the path-dependency feature of the process.

One of the most prominent examples of the application of Markov processes is in the game of Go, as used by DeepMind. The game of Go exemplifies a discrete-time Markov process. Each move in the game is step-like and countable, with no concepts such as 1.5 or 1.3 steps. The vast number of possible states in Go requires substantial computational resources, but in theory, all possible moves occur on a fixed board, making it countable and terminal.

Markov processes do not require the presence of a terminal or absorbing state, nor do they require the state space to be countable. However, beginning our understanding of Markov processes with countable states and discrete time simplifies and focuses our comprehension of the fundamental properties of the process. As we encounter continuous time or uncountable states, we can adjust and extend our understanding accordingly.

Markov processes can be represented in three different ways:

1. Functional Representation: This method employs a transition function which, when given a non-terminal state, returns a probability distribution of the next states. This representation is beneficial for simulations involving sampling of the next state from the probability distribution. This method applies to both finite and infinite state spaces in Markov processes.

2. Sparse Data Structure Representation: This representation utilizes a transition map, which is compact for storage and handy for visualization purposes. However, this method is only applicable to finite state spaces in Markov processes.

3. Dense Data Structure Representation: This representation utilizes a 2D numpy array derived from the get_transition_matrix method. This representation is useful for conducting linear algebra operations, which are often needed to compute the mathematical properties of the process, such as the stationary distribution. Like the sparse data structure representation, this method only applies to finite state spaces in Markov processes.

Finite Markov processes, a subset of Markov processes with a finite state space, have several unique properties. They have a discrete state space, a finite transition matrix, and a unique stationary distribution that remains constant over time. This stationary distribution, which can be computed using linear algebra techniques, provides important information about the long-term behavior of the process.

In programming, "currying" is a technique used to simplify the handling of functions with multiple arguments. It breaks down such functions into a sequence of functions, each accepting a single argument. This method is especially useful when dealing with the transition matrices in finite Markov processes, as it allows us to focus only on non-zero transitions.

Setting up a Markov process typically involves specifying a transition probability function and an initial state. The transition probability function acts as the dynamics or rules of the process, determining the evolution of the process once the initial state is set. As such, Markov processes offer a powerful set of tools for modeling and solving complex, stochastic problems.