Compiler-gym

includes numerous updates and bug fixes.

Summary of Changes

- [llvm] Added two new observation spaces, `LexedIr` and `LexedIrTuple`,
providing access to a sequence of IR tokens
([742](https://github.com/facebookresearch/CompilerGym/pull/742), thanks
fivosts!).
- [llvm] Added the "Jotaibench" benchmark suite, providing 18,761 new executable
C programs extracted from handwritten code on GitHub
([705](https://github.com/facebookresearch/CompilerGym/pull/705), thanks
canesche!).
- Added support for Python 3.10.
- [llvm] Fixed a bug with non-terminating subprocesses
([741](https://github.com/facebookresearch/CompilerGym/pull/741), thanks
thecoblack!).
- [llvm] Fixed a bug where the incorrect number of runtimes were reported by
`reset()` ([761](https://github.com/facebookresearch/CompilerGym/pull/761)),
and an incorrect number of warm up runs were being performed
([717](https://github.com/facebookresearch/CompilerGym/pull/717), thanks
lqwk!).
- [llvm] New leaderboard submission using GATv2 and DD-PPO
([728](https://github.com/facebookresearch/CompilerGym/pull/728), thanks
anthony0727!).
- Added the ability to set timeout on each of the individual environment
operations ([716](https://github.com/facebookresearch/CompilerGym/pull/716),
thanks ricardoprins!).
* Added support for loading URLs in `CompilerEnvStateReader.read_paths()`
([692](https://github.com/facebookresearch/CompilerGym/pull/692), thanks
thecoblack!).
- Simplified Makefile rules: renamed `install-test` to `test` and deprecated
bazel test rules.
- Fixed a bug where the `TimeLimit` wrapper would interfere with benchmark
iterator wrappers
([739](https://github.com/facebookresearch/CompilerGym/pull/739), thanks
nluu175!).
- [ci] Added CI test coverage of example services
([695](https://github.com/facebookresearch/CompilerGym/pull/695),
[642](https://github.com/facebookresearch/CompilerGym/pull/642),
[699](https://github.com/facebookresearch/CompilerGym/pull/699), thanks
mostafaelhoushi!).
- [ci] Updated Github actions to use Node v16.
- Reduced verbosity and wall time of CMake builds.
- Updates and fixes dependent package conflicts (fixes 771, 768).

Credits

A huge thank you to all code contributors!

* anthony0727
* canesche made their first contribution in
[705](https://github.com/facebookresearch/CompilerGym/pull/705)
* fivosts made their first contribution in
[742](https://github.com/facebookresearch/CompilerGym/pull/742)
* jaopaulolc made their first contribution in
[738](https://github.com/facebookresearch/CompilerGym/pull/738)
* lqwk made their first contribution in
[717](https://github.com/facebookresearch/CompilerGym/pull/717)
* mostafaelhoushi
* nluu175 made their first contribution in
[739](https://github.com/facebookresearch/CompilerGym/pull/739)
* ricardoprins made their first contribution in
[716](https://github.com/facebookresearch/CompilerGym/pull/716)
* ryanrussell made their first contribution in
[755](https://github.com/facebookresearch/CompilerGym/pull/755)
* sahirgomez1
* thecoblack
* youweiliang made their first contribution in
[751](https://github.com/facebookresearch/CompilerGym/pull/751)

**Full Changelog**:
[v0.2.4...v0.2.5](https://github.com/facebookresearch/CompilerGym/compare/v0.2.4...v0.2.5)

This release adds a new compiler environment, new APIs, and a suite of backend
improvements to improve the flexibility of CompilerGym environments. Many thanks
to code contributors: sogartar, KyleHerndon, SoumyajitKarmakar, uduse, and
anthony0727!

Highlights of this release include:

- [mlir] Began work on a new environment for matrix multiplication using MLIR
([652](https://github.com/facebookresearch/CompilerGym/pull/652), thanks
KyleHerndon and sogartar!). Note this environment is not yet included in the
pypi package and must be [compiled from
source](https://github.com/facebookresearch/CompilerGym/blob/development/INSTALL.md#building-from-source-with-cmake).
- [llvm] Added a new `env.benchmark_from_clang_invocation()` method
([577](https://github.com/facebookresearch/CompilerGym/pull/577)) that can be
used for constructing LLVM environment automatically from C/C++ compiler
invocations. This makes it much easier to integrate CompilerGym with your
existing build scripts.
- Added three new wrapper classes: `Counter`, that provides op counts for
analysis ([683](https://github.com/facebookresearch/CompilerGym/pull/683));
`SynchronousSqliteLogger`, that provides logging of environment interactions
to a relational database
([679](https://github.com/facebookresearch/CompilerGym/pull/679)), and
`ForkOnStep` that provides an `undo()` operation
([682](https://github.com/facebookresearch/CompilerGym/pull/682)).
- Added `reward_space` and `observation_space` parameters to `env.reset()`
([659](https://github.com/facebookresearch/CompilerGym/pull/659), thanks
SoumyajitKarmakar!)

This release includes a number of improvements to the backend APIs that make it
easier to write new CompilerGym environments:

- Refactored the backend to make `CompilerEnv` an abstract interface, and
`ClientServiceCompilerEnv` the concrete implementation of this interface. This
enables new environments to be implemented without using gRPC
([633](https://github.com/facebookresearch/CompilerGym/pull/633), thanks
sogartar!).
- Extended the support for different types of action and observation spaces
([641](https://github.com/facebookresearch/CompilerGym/pull/641),
[643](https://github.com/facebookresearch/CompilerGym/pull/643), thanks
sogartar!), including new `Permutation` and `SpaceSequence` spaces
([645](https://github.com/facebookresearch/CompilerGym/pull/645), thanks
sogartar!)..
- Added a new `disk/` subdirectory to compiler service's working directories,
which is symlinked to an on-disk location for devices which support in-memory
working directories. This fixes a bug with leftover temporary directories from
LLVM ([672](https://github.com/facebookresearch/CompilerGym/pull/672)).

This release also includes numerous bug fixes and improvements, many of which
were reported or fixed by the community. For example, fixing a bug in cache file
locations ([656](https://github.com/facebookresearch/CompilerGym/pull/656),
thanks uduse!), and a missing flag definition in example code
([684](https://github.com/facebookresearch/CompilerGym/pull/684), thanks
anthony0727!).

**Full Changelog**:
https://github.com/facebookresearch/CompilerGym/compare/v0.2.3...v0.2.4

This release brings in deprecating changes to the core `env.step()` routine, and
lays the groundwork for enabling new types of compiler optimizations to be
exposed through CompilerGym. Many thanks to code contributors: mostafaelhoushi,
sogartar, KyleHerndon, uduse, parthchadha, and xtremey!

Highlights of this release include:

- Added a new `TextSizeInBytes` observation space for LLVM
([575](https://github.com/facebookresearch/CompilerGym/pull/575)).
* Added a new PPO leaderboard entry
([580](https://github.com/facebookresearch/CompilerGym/pull/580). Thanks
xtremey!
- Fixed a bug in which temporary directories created by the LLVM environment
were not cleaned up
([592](https://github.com/facebookresearch/CompilerGym/pull/592)).
- **[Backend]** The function `createAndRunCompilerGymService` now returns an
int, which is the exit return code
([592](https://github.com/facebookresearch/CompilerGym/pull/592)).
- Improvements to the examples documentation
([548](https://github.com/facebookresearch/CompilerGym/pull/548)) and FAQ
([586](https://github.com/facebookresearch/CompilerGym/pull/586))

Deprecations and breaking changes:

- `CompilerEnv.step` no longer accepts a list of actions
([627](https://github.com/facebookresearch/CompilerGym/pull/627)). A new
method, `CompilerEnv.multistep` provides this functionality. This is to
provide compatibility with environments whose action spaces are lists. To
update your code, replace any calls to `env.step()` which take a list of
actions to use `env.multistep()`. Thanks sogartar!
- The arguments `observations` and `rewards` to `step()` have been renamed
`observation_spaces` and `reward_spaces`, respectively
([627](https://github.com/facebookresearch/CompilerGym/pull/627)).
- `Reward.id` has been renamed `Reward.name`
([565](https://github.com/facebookresearch/CompilerGym/pull/565),
[612](https://github.com/facebookresearch/CompilerGym/pull/612)). Thanks
parthchadha!
* The backend protocol buffer schema has been updated to natively support more
types of observation and action, and to support nested spaces
([531](https://github.com/facebookresearch/CompilerGym/pull/531)). Thanks
sogartar!

**Full Changelog**:
https://github.com/facebookresearch/CompilerGym/compare/v0.2.2...v0.2.3

Amongst the highlights of this release are support for building with CMake and a
new compiler environment based on loop unrolling. Many thanks to sogartar,
mostafaelhoushi, KyleHerndon, and yqtianust for code contributions!

- Added support for building CompilerGym from source on Linux using **CMake**
([498](https://github.com/facebookresearch/CompilerGym/pull/498),
[478](https://github.com/facebookresearch/CompilerGym/pull/478)). The new
build system coexists with the bazel build and enables customization over the
CMake configuration used to build the LLVM environment. See
[INSTALL.md](https://github.com/facebookresearch/CompilerGym/blob/development/INSTALL.md#building-from-source-with-cmake)
for details. Credit: sogartar, KyleHerndon.
- Added an environment for loop optimizations in LLVM
([530](https://github.com/facebookresearch/CompilerGym/pull/530),
[529](https://github.com/facebookresearch/CompilerGym/pull/529),
[517](https://github.com/facebookresearch/CompilerGym/pull/517)). This new
example environment provides control over loop unrolling factors and
demonstrates how to build a standalone LLVM binary using the new CMake build
system. Credit: mostafaelhoushi.
- Added a new `BenchmarkUri` class and API for parsing URIs
([525](https://github.com/facebookresearch/CompilerGym/pull/525)). This
enables benchmarks to have optional parameters that can be used by the backend
services to modify their behavior.
- **[llvm]** Enabled runtime reward to be calculated on systems where `/dev/shm`
does not permit executables
([510](https://github.com/facebookresearch/CompilerGym/pull/510)).
- **[llvm]** Added a new `benchmark://mibench-v1` dataset and deprecated
`benchmark://mibench-v0`
([511](https://github.com/facebookresearch/CompilerGym/pull/511)). If you are
using `mibench-v0`, please update to the new version.
- **[llvm]** Enabled all 20 of the cBench runtime datasets to be used by the
`benchmark://cbench-v1` dataset
([525](https://github.com/facebookresearch/CompilerGym/pull/525)).
- Made the `site_data_base` argument of the `Dataset` class constructor optional
([518](https://github.com/facebookresearch/CompilerGym/pull/518)).
- Added support for building CompilerGym from source on macOS Monterey
([494](https://github.com/facebookresearch/CompilerGym/issues/494)).
- Removed the legacy dataset scripts and APIs that were deprecated in v0.1.8.
Please use the [new dataset
API](https://compilergym.com/compiler_gym/datasets.html#datasets). The
following has been removed:
- The `compiler_gym.bin.datasets` script.
- The properties: `CompilerEnv.available_datasets`, and
`CompilerEnv.benchmarks`.
- The `CompilerEnv.require_dataset()`, `CompilerEnv.require_datasets()`,
`CompilerEnv.register_dataset()`, and
`CompilerEnv.get_benchmark_validation_callback()` methods.
- Numerous other bug fixes and improvements.

**Full Change Log**:
[v0.2.1...v0.2.2](https://github.com/facebookresearch/CompilerGym/compare/v0.2.1...v0.2.2)

Highlights of this release include:

- **[Complex and composite action spaces]** Added a new schema for describing
action spaces
([369](https://github.com/facebookresearch/CompilerGym/pull/369)). This
complete overhaul enables a much richer set of actions to be exposed, such as
composite actions spaces, dictionaries, and continuous actions.
- **[State Transition Dataset]** We have released the first iteration of the
state transition dataset, a large collection of (state,action,reward) tuples
for the LLVM environments, suitable for large-scale supervised learning. We
have added an example learned cost model using a graph neural network in
`examples/gnn_cost_model`
([484](https://github.com/facebookresearch/CompilerGym/pull/484), thanks
bcui19!).
- **[New examples]** We have added several new examples to the `examples/`
directory, including a new loop unrolling demo based on LLVM
([477](https://github.com/facebookresearch/CompilerGym/pull/477), thanks
mostafaelhoushi!), a loop tool demo
([457](https://github.com/facebookresearch/CompilerGym/pull/457), thanks
bwasti!), micro-benchmarks for operations, and example reinforcement learning
scripts ([484](https://github.com/facebookresearch/CompilerGym/pull/484)).
See `examples/README.md` for details. We also overhauled the example compiler
gym service
([467](https://github.com/facebookresearch/CompilerGym/pull/467)).
- **[New logo]** Thanks Christy for designing a great new logo for CompilerGym!
([471](https://github.com/facebookresearch/CompilerGym/pull/471))
- **[llvm]** Added a new `Bitcode` observation space
([442](https://github.com/facebookresearch/CompilerGym/pull/442)).
- Numerous bug fixes and improvements.

Deprecations and breaking changes:

- **[Backend API change]** Out-of-tree compiler services will require updating
to the new action space API
([369](https://github.com/facebookresearch/CompilerGym/pull/369)).
- The `env.observation.add_derived_space()` method has been deprecated and will
be removed in a future release. Please use the new
`derived_observation_spaces` argument to the `CompilerEnv` constructor
([463](https://github.com/facebookresearch/CompilerGym/pull/463)).
- The `compiler_gym.utils.logs` module has been deprecated. Use
`compiler_gym.utils.runfiles_path` instead
([453](https://github.com/facebookresearch/CompilerGym/pull/453)).
- The `compiler_gym.replay_search` module has been deprecated and merged into
the `compiler_gym.random_search`
([453](https://github.com/facebookresearch/CompilerGym/pull/453)).

This release adds two new compiler optimization problems to CompilerGym: GCC
command line flag optimization and CUDA loop nest optimization.

- **[GCC]** A new `gcc-v0` environment, authored by hughleat, exposes the
command line flags of [GCC](https://gcc.gnu.org/) as a reinforcement learning
environment. GCC is a production-grade compiler for C and C++ used throughout
industry. The environment provides several datasets and a large, high
dimensional action space that works on several GCC versions. For further
details check out the [reference
documentation](https://facebookresearch.github.io/CompilerGym/envs/gcc.html).
- **[loop_tool]** A new `loop_tool-v0` environment, authored by bwasti,
provides an experimental intermediate representation of *n*-dimensional data
computation that can be lowered to both CPU and GPU backends. This provides a
reinforcement learning environment for manipulating nests of loop computations
to maximize throughput. For further details check out the [reference
documentation](https://facebookresearch.github.io/CompilerGym/envs/loop_tool.html).

Other highlights of this release include:

- **[Docker]** Published a
[chriscummins/compiler_gym](https://hub.docker.com/repository/docker/chriscummins/compiler_gym)
docker image that can be used to run CompilerGym services in standalone
isolated containers
([424](https://github.com/facebookresearch/CompilerGym/pull/424)).
- **[LLVM]** Fixed a bug in the experimental `Runtime` observation space that
caused observations to slow down over time
([398](https://github.com/facebookresearch/CompilerGym/pull/398)).
- **[LLVM]** Added a new utility module to compute observations from bitcodes
([405](https://github.com/facebookresearch/CompilerGym/pull/405)).
- Overhauled the continuous integration services to reduce computational
requirements by 59.4% while increasing test coverage
([392](https://github.com/facebookresearch/CompilerGym/pull/392)).
- Improved error reporting if computing an observation fails
([380](https://github.com/facebookresearch/CompilerGym/pull/380)).
- Changed the return type of `compiler_gym.random_search()` to a `CompilerEnv`
([387](https://github.com/facebookresearch/CompilerGym/pull/387)).
- Numerous other bug fixes and improvements.

Many thanks to code contributors: thecoblack, bwasti, hughleat, and
sahirgomez1!