Optimum-habana

Upgrade to Transformers v4.32.0 to fix a bug with Llama.

- Upgrade to Transformers v4.32 354 regisss

**Full Changelog**: https://github.com/huggingface/optimum-habana/compare/v1.7.0...v1.7.1

This release is fully compatible with SynapseAI 1.7.0, which is the latest version. Check out Habana's [documentation](https://docs.habana.ai/en/v1.7.0/) for more information about the new features.


Memory stats

Memory stats are now logged every `logging_steps` steps to give more information about the memory consumption of HPUs.

- Memory stats 89


DeepSpeed demo notebook with GPT2-XL

This repository now has a notebook displaying how to use DeepSpeed to pre-train/fine-tune GPT2-XL on GAUDI. You can find it [here](https://github.com/huggingface/optimum-habana/blob/main/notebooks/AI_HW_Summit_2022.ipynb).

- Add DeepSpeed demo notebook 112


Fix gradient checkpointing for BERT/RoBERTa/ALBERT

An error used to be raised by PyTorch when running BERT-like models with gradient checkpointing. This has been fixed.

- Fix gradient checkpointing for BERT/RoBERTa/ALBERT 118

This release is fully compatible with SynapseAI 1.6.0.
- Update to SynapseAI 1.6.0 91

*It is recommended to use SynapseAI 1.6.0 for optimal performance.*


Documentation

Optimum Habana now has a dedicated documentation. you can find it [here](https://huggingface.co/docs/optimum/habana_index).

It shows how to quickly make a Transformers-based script work with the library. It also contains guides explaining how to do distributed training, how to use DeepSpeed or how to make the most of HPUs to accelerate training.


Masked Language Modeling

[A new example script](https://github.com/huggingface/optimum-habana/blob/main/examples/language-modeling/run_mlm.py) has been added to perform masked language modeling. This is especially useful if you want to pretrain models such as BERT or RoBERTa.
- Add run_mlm.py in language-modeling examples 83

BLOOM(Z)

BLOOM is introduced in this release with HPU-optimized tweaks to perform fast inference using DeepSpeed. A text-generation example is provided [here](https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation) so that you can easily try it.

- Add text-generation example for BLOOM/BLOOMZ with DeepSpeed-inference 190 regisss

Check out [the blog post](https://huggingface.co/blog/habana-gaudi-2-bloom) we recently released for a benchmark comparing BLOOMZ performance on Gaudi2 and A100.

Multi-node training

This release adds support for multi-node training through DeepSpeed. This enables you to scale out up to thousands of nodes to speed up your trainings even more!

- Add support for multi-node training 116

Check out the [documentation](https://huggingface.co/docs/optimum/habana/usage_guides/multi_node_training) to get started.


Inference through HPU graphs

You can now perform inference faster on Gaudi with [HPU graphs](https://docs.habana.ai/en/v1.8.0/PyTorch/Inference_on_PyTorch/Inference_Using_HPU_Graphs.html).

- Add support for inference through HPU graphs in GaudiTrainer 151

HPU graphs are currently only supported for single-device runs. Check out the [documentation](https://huggingface.co/docs/optimum/habana/usage_guides/accelerate_inference) for more information.