Fms-hf-tuning

What's Changed
* fix: Added correct link in main readme for the trainer-controller readme by seshapad in https://github.com/foundation-model-stack/fms-hf-tuning/pull/254
* trainer controller doc updates by alex-jw-brooks in https://github.com/foundation-model-stack/fms-hf-tuning/pull/244
* docs: fix the instructions for running with LORA by HarikrishnanBalagopal in https://github.com/foundation-model-stack/fms-hf-tuning/pull/265
* refactor code to preprocess datasets by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/259
* Replace shutil.copytree() to fix permission error by olson-ibm in https://github.com/foundation-model-stack/fms-hf-tuning/pull/251
* fix: logic for getting tracker config by HarikrishnanBalagopal in https://github.com/foundation-model-stack/fms-hf-tuning/pull/267
* fix: remove lm_head for granite with llama arch models by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/258


**Full Changelog**: https://github.com/foundation-model-stack/fms-hf-tuning/compare/v1.0.0-rc.1...v1.1.0-rc.1

New major features:
1. Trainer Controller and ability to stop training earlier using metrics and thresholds. See https://github.com/foundation-model-stack/fms-hf-tuning?tab=readme-ov-file#trainer-controller-framework
2. Experiment Tracker while tuning for more insights (choice of frameworks)

Dependency upgrades:
1. Transformers to minimum version 4.42.0 needed for tuning of granite with llama models

API /interface changes:
1. Introduction of tokenizer field in ModelArguments https://github.com/foundation-model-stack/fms-hf-tuning/blob/main/tuning/config/configs.py#L51 to allow custom tokenizer while tuning
2. Deprecation of tokenizer field from PromptTuningConfig and moved to ModelArguments

Full List of what's Changed:
* minor refactor to allow modular functions by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/224
* feat: support some metrics being 'None' without stopping training by HarikrishnanBalagopal in https://github.com/foundation-model-stack/fms-hf-tuning/pull/169
* Support custom embedding layer resizing to the desired multiple by kmehant in https://github.com/foundation-model-stack/fms-hf-tuning/pull/227
* ReIntroduce Package for FMS Accel by fabianlim in https://github.com/foundation-model-stack/fms-hf-tuning/pull/223
* feat: History based loss metric by seshapad in https://github.com/foundation-model-stack/fms-hf-tuning/pull/156
* docs: instructions for using the trainer controller framework by HarikrishnanBalagopal in https://github.com/foundation-model-stack/fms-hf-tuning/pull/214
* feat: support custom tokenizer by kmehant in https://github.com/foundation-model-stack/fms-hf-tuning/pull/229
* tests: tracker unit tests by dushyantbehl in https://github.com/foundation-model-stack/fms-hf-tuning/pull/172
* feat: Aim runid export by dushyantbehl in https://github.com/foundation-model-stack/fms-hf-tuning/pull/90
* Move default operations and metrics to variables by alex-jw-brooks in https://github.com/foundation-model-stack/fms-hf-tuning/pull/249
* Update README.md by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/240
* deps: Update transformers to latest and skip broken prompt tuning tests by Abhishek-TAMU in https://github.com/foundation-model-stack/fms-hf-tuning/pull/246
* deps: Update transformers lower bound version by Abhishek-TAMU in https://github.com/foundation-model-stack/fms-hf-tuning/pull/250
* V100rc1 release by olson-ibm in https://github.com/foundation-model-stack/fms-hf-tuning/pull/252

New Contributors
* Abhishek-TAMU made their first contribution in https://github.com/foundation-model-stack/fms-hf-tuning/pull/246
* olson-ibm made their first contribution in https://github.com/foundation-model-stack/fms-hf-tuning/pull/252

**Full Changelog**: https://github.com/foundation-model-stack/fms-hf-tuning/compare/v0.4.0...v1.0.0

What's Changed
* minor refactor to allow modular functions by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/224
* feat: support some metrics being 'None' without stopping training by HarikrishnanBalagopal in https://github.com/foundation-model-stack/fms-hf-tuning/pull/169
* Support custom embedding layer resizing to the desired multiple by kmehant in https://github.com/foundation-model-stack/fms-hf-tuning/pull/227
* ReIntroduce Package for FMS Accel by fabianlim in https://github.com/foundation-model-stack/fms-hf-tuning/pull/223
* feat: History based loss metric by seshapad in https://github.com/foundation-model-stack/fms-hf-tuning/pull/156
* docs: instructions for using the trainer controller framework by HarikrishnanBalagopal in https://github.com/foundation-model-stack/fms-hf-tuning/pull/214
* feat: support custom tokenizer by kmehant in https://github.com/foundation-model-stack/fms-hf-tuning/pull/229
* tests: tracker unit tests by dushyantbehl in https://github.com/foundation-model-stack/fms-hf-tuning/pull/172
* feat: Aim runid export by dushyantbehl in https://github.com/foundation-model-stack/fms-hf-tuning/pull/90
* Move default operations and metrics to variables by alex-jw-brooks in https://github.com/foundation-model-stack/fms-hf-tuning/pull/249
* Update README.md by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/240
* deps: Update transformers to latest and skip broken prompt tuning tests by Abhishek-TAMU in https://github.com/foundation-model-stack/fms-hf-tuning/pull/246
* deps: Update transformers lower bound version by Abhishek-TAMU in https://github.com/foundation-model-stack/fms-hf-tuning/pull/250

New Contributors
* Abhishek-TAMU made their first contribution in https://github.com/foundation-model-stack/fms-hf-tuning/pull/246

**Full Changelog**: https://github.com/foundation-model-stack/fms-hf-tuning/compare/v0.4.0-rc.3...v1.0.0-rc.1

Summary of Changes
* Support for LoRA tuning for llama3 and granite (with GPTBigCode) architectures
* Dependencies versions adjustment

What's Changed
* remove merge model for lora tuned adapters by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/197
* Add test coverage by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/171
* Install Acceleration Framework into Training Script by fabianlim in https://github.com/foundation-model-stack/fms-hf-tuning/pull/157
* deps: limit dependency ranges by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/54
* Delete dependabot.yml by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/207
* add dependabot.yml by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/208
* Fix additional callbacks by VassilisVassiliadis in https://github.com/foundation-model-stack/fms-hf-tuning/pull/199
* Update trl by alex-jw-brooks in https://github.com/foundation-model-stack/fms-hf-tuning/pull/213
* deps: cap transformers at 4.40.2 by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/218
* Formatting consolidation main by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/216
* Fix PyPi publish error caused by direct url reference by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/219


**Full Changelog**: https://github.com/foundation-model-stack/fms-hf-tuning/compare/v0.3.0...v0.4.0

What's Changed
* remove merge model for lora tuned adapters by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/197
* Add test coverage by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/171
* Install Acceleration Framework into Training Script by fabianlim in https://github.com/foundation-model-stack/fms-hf-tuning/pull/157
* deps: limit dependency ranges by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/54
* Delete dependabot.yml by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/207
* add dependabot.yml by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/208
* Fix additional callbacks by VassilisVassiliadis in https://github.com/foundation-model-stack/fms-hf-tuning/pull/199
* Update trl by alex-jw-brooks in https://github.com/foundation-model-stack/fms-hf-tuning/pull/213
* deps: cap transformers at 4.40.2 by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/218
* Formatting consolidation main by Ssukriti in https://github.com/foundation-model-stack/fms-hf-tuning/pull/216
* Fix PyPi publish error caused by direct url reference by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/219


**Full Changelog**: https://github.com/foundation-model-stack/fms-hf-tuning/compare/v0.3.0...v0.4.0-rc.3

Summary of Changes
* Support for LoRA tuning for llama3 and granite (with GPTBigCode) architectures
* Various dependencies versions adjustment
What's Changed
* remove merge model for lora tuned adapters by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/197
* Add test coverage by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/171
* Install Acceleration Framework into Training Script by fabianlim in https://github.com/foundation-model-stack/fms-hf-tuning/pull/157
* deps: limit dependency ranges by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/54
* Delete dependabot.yml by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/207
* add dependabot.yml by tedhtchang in https://github.com/foundation-model-stack/fms-hf-tuning/pull/208
* Fix additional callbacks by VassilisVassiliadis in https://github.com/foundation-model-stack/fms-hf-tuning/pull/199
* Update trl by alex-jw-brooks in https://github.com/foundation-model-stack/fms-hf-tuning/pull/213
* deps: cap transformers at 4.40.2 by anhuong in https://github.com/foundation-model-stack/fms-hf-tuning/pull/218


**Full Changelog**: https://github.com/foundation-model-stack/fms-hf-tuning/compare/v0.3.0...v0.4.0-rc.2