Open-spiel

This release adds several games and algorithms, improvements, bug fixes, and documentation updates.

Support and Process changes

- Added Python 3.11 support
- Added Roshambo bot population to wheels
- Removed Python 3.6 support
- Upgraded versions of supported extra packages (OR-Tools, abseil, Jax, TF, Pytorch, etc.)

Games

- Bach or Stravisnky matrix game
- Block Dominoes (python)
- Crazy Eights
- Dhu Dizhu
- Liar's poker (python)
- MAEDN (Mensch Ärgere Dich Nicht)
- Nine Men's morris

Game Transforms

- Add Noisy utility to leaves game transform
- Add Zero-sum game transform

Other environments

- Atari Learning Environment (ALE)

Algorithms

- Boltzmann Policy Iteration (for mean-field games)
- Correlated Q-learning
- Information State MCTS, Cowling et al. '12 (Python)
- LOLA and LOLA-DiCE (Foerster, Chen, Al-Shedivat, et al. '18) and Opponent Shaping (JAX)
- MIP Nash solver (Sandholm, Gilpin, and Conitzer '05)
- Proximal Policy Optimization (PPO); adapted from CleanRL. Supports single-agent use case, tested on ALE.
- Regret-matching (Hart & Mas-Colell '00) for normal-form games and as a PSROv2 meta-solver
- Regularized Nash Dynamics (R-NaD), Perolat & de Vylder et. al '22, Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning

Bots

- Simple heuristic Gin Rummy bot
- Roshambo bot population (see python/examples/roshambo_bot_population.py)

Examples

- Opponent shaping on iterated matrix games example
- Roshambo population example
- Using Nash bargaining solution for negotiation example

Improvements and other additions

- Add `Bot::Clone()` method for cloning bots
- Avoid relying on C++ exceptions for playthrough tests
- Add support Agent-vs-Task case in Nash averaging
- Add scoring variants to the game Oh Hell
- Add eligibility traces in C++ Q-learning and SARSA
- Allow creation of per-player random policies
- Support simultaneous move games in policy aggregator and exploitability
- Support UCIBot via pybind11
- Add single_tensor observer for all games
- Add used_indices for non-marginal solvers in PSROv2
- Add Flat Dirichlet random policy sampling
- Add several options to bargaining game: probabilistic ending, max turns, discounted utilities
- Add lambda returns support to JAX policy gradient
- Several improvements to Gambit EFG parser / support
- Add support for softmax policies in fictitious play
- Add temperature parameter to fixed point MFG algorithms
- Add information state tensor to battleship
- Add option to tabular BR to return maximum entropy BR

Fixes

- Fix UCIBot compilation in Windows
- Misc fixes to Nash averaging
- RNaD: fix MLP torso in final layer
- Dark hex observation (max length)
- Fix max game length in abstracted poker games
- Fix legal moves in some ACPC(poker) game cases
- Fix joint policy aggregator
- Fix non-uniform chance outcome sampling in Deep CFR (TF2 & Pytorch)
- Fix randomization bug in alpha_zero_torch

Several other miscellaneous fixes and improvements.

Known issues

There are a few known issues that will be fixed in the coming months.

- Collision with pybind11 and version in C++ LibTorch AlphaZero. See 966.
- PyTorch NFSP convergence issue. See 1008.

Acknowledgments

Thanks to Google DeepMind for continued support of development and maintenance of OpenSpiel.

Thanks to all of our contributors:

- Core Team: https://github.com/deepmind/open_spiel/blob/master/docs/authors.md
- All Contributors: https://github.com/deepmind/open_spiel/graphs/contributors

This release adds several games and algorithms, improvements, bug fixes, and documentation updates.

Support and Process changes

- Upgrade support for newer versions of dependencies
- Add dependency to pybind11_abseil

Games

- 2048
- Checkers
- Dynamic routing game
- Euchre
- Mancala
- Nim
- Phantom Go

Algorithms

- Asymmetric Q-learning
- Magnetic Mirror Descent (MMD)
- NeuRD (PyTorch)
- Policy gradients (JAX)
- Sample-based NeuRD loss (PyTorch)
- Stackelberg solver
- WoLF-PHC

Improvements and other additions

- Blackjack: add observation tensor
- C++ DQN: in-memory target net, saving + loading of model
- Core API reference
- Remove hard-coded inclusion of Hanabi and ACPC in setup.py

Fixes

- Colored Trails: fix max utility
- MCTS handling of chance nodes: properly handle them not just at the root
- Nash averaging optimization fix
- Othello: fix the max game length
- Policy aggregator, surface copy -> deep copy
- pybind11: change game references to shared pointers

Several other miscellaneous fixes and improvements.

Acknowledgments

Thanks to DeepMind for continued support of development and maintenance of OpenSpiel.

Thanks to all of our contributors:

- Core Team: https://github.com/deepmind/open_spiel/blob/master/docs/authors.md
- All Contributors: https://github.com/deepmind/open_spiel/graphs/contributors

This release adds several algorithms and games, and several process changes.

Support, APIs, and Process changes

- Removed support for Python 3.6
- Added support for Python 3.10
- Upgrade support for newer versions of dependencies
- Rust API: add support for loading bots
- CI tests: added MacOS-11, MacOS-12, and Ubuntu 22.04. Removed CI tests for Ubuntu 18.04.

Games

- Colored Trails
- Dynamic Routing Game: added Sioux Falls network
- Mancala (Kalah)
- Multi-issue Bargaining
- Repeated game transform: add info state strings & tensors, utility sum, finite recall
- Sheriff: add info state tensor

Algorithms

- Boltzmann DQN
- Boltzmann Q-learning
- Correlated Q-learning (Greenwald & Hall)
- Deep average network for FP (mean-field games)
- Nash averaging (Balduzzi et al.)
- Nash Q-learning (Hu & Wellman)

Improvements and other additions

- Example: support mean-field games
- File wrapper: expose to Python and add WriteContents
- Nash bargaining score example

Fixes

- VR-MCCFR with nonzero baselines
- PyTorch policy gradient clipping
- Promote pawn to queen in RBC
- PyTorch and LibTorch DQN: fix for illegal moves

Many other fixes to docs and code quality.

Acknowledgments

Thanks to DeepMind for continued support of development and maintenance of OpenSpiel.

Thanks to all of our contributors:

- Core Team: https://github.com/deepmind/open_spiel/blob/master/docs/authors.md
- All Contributors: https://github.com/deepmind/open_spiel/graphs/contributors

This release adds some major functionality: new games, new algorithm, several fixes and new features.

Support and APIs

- Windows: native build via Microsoft Visual Studio (experimental)
- Rust API

Games

- Amazons
- Morpion Solitaire
- Gridworld pathfinding (single-agent and multiagent)
- Linear-Quadratic games (mean-field game)
- Pig: Piglet variant added
- Quoridor: 3-player and 4-player support added
- Utlimate Tic-Tac-Toe

Algorithms

- AlphaZero support for games with chance nodes (Python and C++)
- ADIDAS approximate Nash equilibrium solver by [Gemp et al. '21](https://arxiv.org/abs/2106.01285)
- Boltzmann DQN
- Deep Online Mirror Descent (for mean-field games)
- Expectiminimax (C++)

Mean-field Games

- Deep Online Mirror Descent
- Best response value function (instead of only exact)
- Allow specifying learning rate in fictitious play
- Routing game experiment data
- Softmax policy

Bots

- WBridge5 external bot
- Roshambo bots: expose to Python

Fixes

- Chess SAN notation
- `get_all_states`: support added for games with loops
- Hex and DarkHex bug fixes for even-sized boards
- MCTS sampling from the prior when 0-1 visits specified (Python and C++)
- Pig: 2D observation tensor, `ActionString`, `MaxChanceNodesInHistory`
- Stones n' Gems serialization fix

Miscellaneous

- Added `SpielFatalErrorWithStateInfo` debug helper
- Refactored policies computed by RL into a shared `JointRLAgentPolicy`
- Custom info state resampling function for IS-MCTS
- Hidden Information Games Competition tournament code: make optional dependency
- Upgrade versions of abseil and OR-Tools and versions in python extra deps
- Python dependency on scipy
- Poker chump policies

Acknowledgments

Thanks to DeepMind for continued support of development and maintenance of OpenSpiel.

Thanks to all of our contributors:

- Core Team: https://github.com/deepmind/open_spiel/blob/master/docs/authors.md
- All Contributors: https://github.com/deepmind/open_spiel/graphs/contributors

This is a minor release: mainly for bug fixes, and also some new functionality and updates to core functionality.


New games and modifications

- Dynamic routing game: change to explicit stochastic (Python MFG), or deterministic (Python)
- New garnet game (randomized MDPs) for mean-field games (C++)

New algorithms and other functionality

- Restricted Nash Response (C++), Johanson et al. '08
- Update mean-field game algorithms to use value functions
- Enable Python best response to work for simultaneous-move games

Bug fixes

- Allow observation tensors for turn-based simultaneous move games
- Fixes to HIGC tournament code, add synchronous mode, explicit calls to bots
- Fix game type in built-in observer
- Fix information type for iterated prisoner's dilemma
- Fix to wheels CI testing: always use python3

Misc

- Add missing algorithms and games to algorithms page
- Add patch to our version of absl to compile with newer compilers (Ubuntu 21.10)
- Add python games to API test (now fully supported alongside all C++ games)
- Enable noisy_policy to work for simultaneous move games
- Added Common Loop Utils (CLU) to python extra deps

Acknowledgments

Thanks to DeepMind for continued support of development and maintenance of OpenSpiel.

Thanks to all of our contributors:

- Core Team: https://github.com/deepmind/open_spiel/blob/master/docs/authors.md
- All Contributors: https://github.com/deepmind/open_spiel/graphs/contributors

This is a minor release: mainly for bug fixes, and also some new functionality and updates to core functionality.


New game

- Dynamic routing (python game and its mean-field limit game)

New functionality

- Allow TabularBestResponseMDP to be computed for a specific player
- Add Hidden Information Game Competition (HIGC) tournament code
- Add expected game score for simultaneous move games

Bug fixes

- Fix to blackjack to use standard policy for dealer
- Several fixes to Reconnaissance Blind Chess (see 695 696 and 697)
- Update dependency to newer version of Hanabi
- Fix imperfect recall state string in Phantom Tic-Tac-Toe and Dark Hex
- Fix noisy policy (see https://github.com/deepmind/open_spiel/commit/2703b208068169fb45ebc5bee25dafc0bcb76cfc)
- Fix UndoAction for a number of games, add test for it (also remove UndoAction from some games)

Acknowledgments

Thanks to DeepMind for continued support of development and maintenance of OpenSpiel.

Thanks to all of our contributors:

- Core Team: https://github.com/deepmind/open_spiel/blob/master/docs/authors.md
- All Contributors: https://github.com/deepmind/open_spiel/graphs/contributors