Mdpsolver

MDPSolver is a Python package for Markov Decision Processes (MDPs) with discounted rewards and infinite-horizon.

Features

* Fast solver: Our C++-based solver is substantially faster than other MDP packages available for Python. See details in the documentation.
* Three optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.
* Three value-update methods: *Standard*, *Gauss–Seidel*, *Successive over-relaxation*.
* Supports sparse matrices.
* Employs parallel computing.

Initial pre-release of our C++based solver for Markov Decision Process optimization problems. The solver is based on a Modified Policy Iteration (MPI) algorithm, which derives an epsilon-optimal policy that maximizes the expected total discounted reward, where epsilon is a tolerance parameter given to the algorithm. We further provide the user with the option to choose between three different value update methods as well as switching to an epsilon-optimal Value Iteration or Policy Iteration algorithm. See the Readme-file for further information.