Rl4co

Add FlashAttention2 support ⚡

- Add FlashAttention2 support as mentioned [here](https://github.com/kaist-silab/rl4co/issues/85)
- Remove old wrapper for `half()` precision since Lightning already deals with this
- Fix `scaled_dot_product_attention` implementation in PyTorch < 2.0
- Minor fixes

QoL: New Baseline, Testing Search Methods, Downloader, Miscellanea 🚀

Changelog
- Add mean baseline hyeok9855
- Add testing for search methods
- Move downloader to external repo, extra URL as backup for DPP
- Small bug fix for duplicate args
- Add more modular data generation
- Suppress extra warning in `automatic_optimization`
- Minor doc cleaning

QoL, Better documentation, Bug Fixes 🚀

- Add `RandomPolicy` class
- Control `max_steps` for debugging purposes during decoding
- Better documentation, add tutorials, and references 88 bokveizen
- Set bound to < Python 3.11 for the time being 90 hyeok9855
- Log more info by default in PPO
- `precompute_cache` method can now accept `td` as well
- If `Trainer` is supplied with `gradient_clip_val` and `manual_optimization=False`, then remove gradient clipping (e.g. for PPO)
- Fix test data size following training and not test by default

Search Methods, Flexible Embeddings, New Graph Encoders and more 🚀

Search methods
- New flexible and extensible abstract class
- Active Search (Bello et al, 2016)
- Efficient Active Search (Hottung et al, 2022)

Flexible embeddings
- Support for changing any environment embedding (`init`, `context` and `dynamic`)
- Add [new notebook](https://github.com/kaist-silab/rl4co/blob/main/notebooks/tutorials/2-solving-new-problem.ipynb) showcasing how to solve new complex problems (example of multi-depot multi-agent pickup and delivery problem - MDPDP)

Support for `torch-geometric`
- Added new template graph neural networks (MPNN, GCN)
- Example Notebook [here](https://github.com/kaist-silab/rl4co/blob/main/notebooks/tutorials/3-change-encoder.ipynb)

Miscellaneous
- Separate loggers
- Better imports
- Bugfix compatibility with Mac
- Update configs
- ... and more!

Better training, Bug fixes, and more 🚀

- Better automatic training with DDP 87
- Bug Fix `RL4COTrainer`
- Avoid broadcasting error warning in critic baselines
- Fix rollout baseline bug
- New experiment config structure: interpolate with environment name (we won't need anymore to have separate folders for each environment name such as TSP, CVRP etc, simply use one config to rule them all!

Major release: refactoring of models, trainer and pipelines, and more! 🚀

- Refactored the old `task` class into a base class (`RL4COLitModule`) that is the base for RL algorithms (such as REINFORCE and PPO), following the discussion in 67
- New base class for construction methods: now encoder, decoder, policy, and model can be based on common parent classes to make implementation much more modular
- Added native loading from the checkpoint, which used to be buggy
- Nice new logo (we like it, but we are obviously biased, so feel free to give us your opinion ;) )
- Added mPDP environment (and added some WIP for EquityTransformer)
- New `RL4COTrainer` that automatically includes training tricks for RL
- Added Codecov coverage
- Better testing: now we thoroughly test most of the library, including training (the Hydra part as well!)
- Documentation overhaul: add Sphinx plugins for modularized, automatic docs
- ... and more!