This release is fully compatible with SynapseAI 1.7.0, which is the latest version. Check out Habana's [documentation](https://docs.habana.ai/en/v1.7.0/) for more information about the new features.
Memory stats
Memory stats are now logged every `logging_steps` steps to give more information about the memory consumption of HPUs.
- Memory stats 89
DeepSpeed demo notebook with GPT2-XL
This repository now has a notebook displaying how to use DeepSpeed to pre-train/fine-tune GPT2-XL on GAUDI. You can find it [here](https://github.com/huggingface/optimum-habana/blob/main/notebooks/AI_HW_Summit_2022.ipynb).
- Add DeepSpeed demo notebook 112
Fix gradient checkpointing for BERT/RoBERTa/ALBERT
An error used to be raised by PyTorch when running BERT-like models with gradient checkpointing. This has been fixed.
- Fix gradient checkpointing for BERT/RoBERTa/ALBERT 118