Mosaicml

What's Changed
* Update mcli examples to use 0.16.0 by irenedea in https://github.com/mosaicml/llm-foundry/pull/1713
* Refactor HF checkpointer by milocress in https://github.com/mosaicml/llm-foundry/pull/1690
Previously, MlFlow required PEFT models to be specified as a special "flavor" distinct from Transformers models. This workaround is no longer necessary, allowing us to simplify the codepath and cleanly abstract uploading the HuggingFace checkpoints from registering trained models.
* Bump version to 0.18.0.dev by milocress in https://github.com/mosaicml/llm-foundry/pull/1717
Removes the deprecated `sample_weighing_factor` argument from `mpt` loss calculations.

**Full Changelog**: https://github.com/mosaicml/llm-foundry/compare/v0.16.0...v0.17.0

What's New

**1. Torch 2.1 Support**

Composer officially supports PyTorch 2.1! We support several new features from 2.1, including CustomPolicy which supports granular wrapping with FSDP.

What's Changed
* Add 0.16 checkpoint to backwards compatibility tests by eracah in https://github.com/mosaicml/composer/pull/2567
* Updating FSDP monkeypatch by mvpatel2000 in https://github.com/mosaicml/composer/pull/2571
* Add Databricks UC Volume Object Store by panchalhp-db in https://github.com/mosaicml/composer/pull/2548
* Fix pytest disk space OOM issue by adding tmp_path_retention_policy=None by j316chuck in https://github.com/mosaicml/composer/pull/2583
* Change daily nightly test version by j316chuck in https://github.com/mosaicml/composer/pull/2596
* Add save and register wrappers to mlflow logger by dakinggg in https://github.com/mosaicml/composer/pull/2579
* Missing () fo or in auto microbatching gate by mvpatel2000 in https://github.com/mosaicml/composer/pull/2574
* Simplify FSDP Gradient Clipping by mvpatel2000 in https://github.com/mosaicml/composer/pull/2586
* Use FSDP CustomPolicy to support custom kwargs passed to different wrapped modules by cli99 in https://github.com/mosaicml/composer/pull/2585
* Free outputs callback by mvpatel2000 in https://github.com/mosaicml/composer/pull/2598
* Merge branch 'dev' into spr/dev/458c4e36 by b-chu in https://github.com/mosaicml/composer/pull/2595
* Fix a bug when batch type is dict and one of the values is the list by mvpatel2000 in https://github.com/mosaicml/composer/pull/2599
* Readme update by ejyuen in https://github.com/mosaicml/composer/pull/2581
* Add chain of thought eval by bmosaicml in https://github.com/mosaicml/composer/pull/2466
* Add torch 2.1.0 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2602
* Change pr cpu and pr gpu test docker images by j316chuck in https://github.com/mosaicml/composer/pull/2611
* Change the tokenizer json file to read binary by dakinggg in https://github.com/mosaicml/composer/pull/2608
* [Docs] MLflow casing by aspfohl in https://github.com/mosaicml/composer/pull/2609
* Call generate callback at end of training by aspfohl in https://github.com/mosaicml/composer/pull/2607
* Refactor save interval and eval interval to share code by dakinggg in https://github.com/mosaicml/composer/pull/2600
* Deprecate many datasets and models by mvpatel2000 in https://github.com/mosaicml/composer/pull/2605
* Clean up gpu tests by mvpatel2000 in https://github.com/mosaicml/composer/pull/2612
* Remove apex test by j316chuck in https://github.com/mosaicml/composer/pull/2616
* Patch default precision by mvpatel2000 in https://github.com/mosaicml/composer/pull/2628
* Add logging for generate callbacks by aspfohl in https://github.com/mosaicml/composer/pull/2630
* Expose input_names and output_names when exporting to ONNX by antoinebrl in https://github.com/mosaicml/composer/pull/2601
* Bump version to 0.16.4 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2627

New Contributors
* panchalhp-db made their first contribution in https://github.com/mosaicml/composer/pull/2548
* cli99 made their first contribution in https://github.com/mosaicml/composer/pull/2585

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.16.3...v0.16.4

What's New

**1. Add passk for HumanEval**

HumanEval now supports passk. We also support first-class integration with the MosaicML platform for secure code evaluation.

**2. `log_model` with MLFlow**

The MLFlow integration now supports `log_model` at the end of the run.

What's Changed
* Update checkpoint.py by b-chu in https://github.com/mosaicml/composer/pull/2540
* Add log image to mlflow by eracah in https://github.com/mosaicml/composer/pull/2416
* Log runtime estimator units by mvpatel2000 in https://github.com/mosaicml/composer/pull/2542
* Bump traitlets from 5.9.0 to 5.10.0 by dependabot in https://github.com/mosaicml/composer/pull/2547
* Bump gitpython from 3.1.35 to 3.1.36 by dependabot in https://github.com/mosaicml/composer/pull/2546
* Bump ipykernel from 6.25.1 to 6.25.2 by dependabot in https://github.com/mosaicml/composer/pull/2544
* Add providers param to ONNX Session in tests by nik-mosaic in https://github.com/mosaicml/composer/pull/2553
* Bump flash attn by mvpatel2000 in https://github.com/mosaicml/composer/pull/2551
* Remove pin by mvpatel2000 in https://github.com/mosaicml/composer/pull/2554
* Change filter to include pull_request_target by mvpatel2000 in https://github.com/mosaicml/composer/pull/2557
* Downgrade nightly to previous version by mvpatel2000 in https://github.com/mosaicml/composer/pull/2556
* MCLI Code Eval by rishab-partha in https://github.com/mosaicml/composer/pull/2479
* Bump cryptography from 41.0.3 to 41.0.4 by dependabot in https://github.com/mosaicml/composer/pull/2559
* Bump gitpython from 3.1.36 to 3.1.37 by dependabot in https://github.com/mosaicml/composer/pull/2560
* Update numpy requirement from <1.26.0,>=1.21.5 to >=1.21.5,<1.27.0 by dependabot in https://github.com/mosaicml/composer/pull/2561
* Update support for HumanEval by mcarbin in https://github.com/mosaicml/composer/pull/2550
* Add log_model to MLFlowLogger by dakinggg in https://github.com/mosaicml/composer/pull/2541
* Bump version to 0.16.3 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2566

New Contributors
* mcarbin made their first contribution in https://github.com/mosaicml/composer/pull/2550

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.16.2...v0.16.3

What's New

**1. PyTorch Nightly Support**

Composer now supports PyTorch Nightly and Cuda 12! Along with new docker images based on nightly PyTorch versions and release candidates, we've updated our PyTorch monkeypatches to support the latest version of PyTorch. These monkeypatches add additional functionality in finer-grain FSDP wrapping and patch bugs related to sharded checkpoints. We are in the process of upstreaming these changes into PyTorch.

Bug Fixes

**1. MosaicML Logger Robustness**

MosaicML logger now is robust to platform timeouts and other errors. Additionally, it can now be disabled by setting the environment variable `MOSAICML_PLATFORM` to `'False'` when training on the MosaicML platform.

**2. GCS Integration**

GCS authentication is now supported with HMAC keys, patching a bug in the previous implementation.

**3. Optimizer Monitor Norm Calculation (https://github.com/mosaicml/composer/pull/2531)**

Previously, the optimizer monitor incorrectly reduced norms across GPUs. It now correctly computes norms in a distributed setting.

What's Changed
* fix: when there is no train_metrics, do not checkpoint by furkanbiten in https://github.com/mosaicml/composer/pull/2502
* Remove metric saving by mvpatel2000 in https://github.com/mosaicml/composer/pull/2514
* Fix daily tests by removing gpu marker by j316chuck in https://github.com/mosaicml/composer/pull/2515
* Refactor mosaic_fsdp.py by b-chu in https://github.com/mosaicml/composer/pull/2506
* Disable slack notifications for PRs by mvpatel2000 in https://github.com/mosaicml/composer/pull/2517
* Add custom sharding to ChunkShardingSpec by b-chu in https://github.com/mosaicml/composer/pull/2507
* Update nightly docker image to torch nightly 09-03-23 by j316chuck in https://github.com/mosaicml/composer/pull/2518
* Update pre-commit in setup.py by b-chu in https://github.com/mosaicml/composer/pull/2522
* Add FSDP custom wrap with torch 2.1 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2460
* Fix GCSObjectStore bug where hmac keys auth doesn't work by eracah in https://github.com/mosaicml/composer/pull/2519
* Bump gitpython from 3.1.34 to 3.1.35 by dependabot in https://github.com/mosaicml/composer/pull/2525
* Bump pytest from 7.4.0 to 7.4.2 by dependabot in https://github.com/mosaicml/composer/pull/2523
* Upgrade to MLFlow version 2.5.0 by ngcgarcia in https://github.com/mosaicml/composer/pull/2528
* Disable cifar daily test by mvpatel2000 in https://github.com/mosaicml/composer/pull/2527
* Mosaicml logger robustness improvements by mvpatel2000 in https://github.com/mosaicml/composer/pull/2530
* Fix metrics keys sort in DecoupledAdamW for OptimizerMonitor FSDP metric agreggation by m1kol in https://github.com/mosaicml/composer/pull/2531
* Fix github actions for GCS integration testing by mvpatel2000 in https://github.com/mosaicml/composer/pull/2532
* Fix GCS tests by mvpatel2000 in https://github.com/mosaicml/composer/pull/2535
* Change cast for mosaicml logger by mvpatel2000 in https://github.com/mosaicml/composer/pull/2538
* Bump Version to 0.16.2 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2537
* Bump transformers version by dakinggg in https://github.com/mosaicml/composer/pull/2539

New Contributors
* ngcgarcia made their first contribution in https://github.com/mosaicml/composer/pull/2528
* m1kol made their first contribution in https://github.com/mosaicml/composer/pull/2531

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.16.1...v0.16.2

New Features

**1. HPU (Habana Gaudi) Support (https://github.com/mosaicml/composer/pull/2444)**

Composer now supports Habana Gaudi chips! To enable HPUs, `device` needs to be specified as `'hpu'`:


composer_model = MyComposerModel(n_layers=3)

trainer = Trainer(
model=composer_model,
device='hpu',
...
)


**2. Generate Callback (https://github.com/mosaicml/composer/pull/2449)**

We've added a new callback which runs generate on a language model at a given frequency to visualize outputs:


from composer.callbacks import Generate

composer_model = MyComposerModel(n_layers=3)
generate_callback = Generate(prompts=['How good is my model?'], interval='5ba')

trainer = Trainer(
model=composer_model,
callbacks = generate_callback,
...
)


Bug Fixes

**1. Checkpoint Fixes**

Elastic sharded checkpointing now disables torchmetric saving to avoid issues with torchmetrics tensors being sharded. Additionally, checkpointing now falls back on the old path which does not convert torchmetrics tensors to `numpy`. Checkpointing also no longer materializes optimizer state when saving weights only.

**2. MLFlow Performance Improvements**

MLFlow integration has significant performance improvements in logging frequency and system metrics collected.


What's Changed
* Hpu support by vivekgoe in https://github.com/mosaicml/composer/pull/2444
* Change `input_ids` to a kwarg in `HuggingFaceModel.generate` by dakinggg in https://github.com/mosaicml/composer/pull/2459
* Add log_table by irenedea in https://github.com/mosaicml/composer/pull/2437
* Enable composer to work with torch nightly builds, torch 2.1.0, and cuda 12.1. by j316chuck in https://github.com/mosaicml/composer/pull/2463
* Materialize only model state_dict in memory for `save_weights_only` by eracah in https://github.com/mosaicml/composer/pull/2450
* Improve performance of MLflow logging by dbczumar in https://github.com/mosaicml/composer/pull/2442
* Fail fast if scheduler warmup and max duration are incompatible by dakinggg in https://github.com/mosaicml/composer/pull/2458
* Add nightly docker image by j316chuck in https://github.com/mosaicml/composer/pull/2452
* Fix local eval by rishab-partha in https://github.com/mosaicml/composer/pull/2465
* Add torch 2.1.0 args for github release-docker workflow by j316chuck in https://github.com/mosaicml/composer/pull/2470
* Log system metrics on each event by prithvikannan in https://github.com/mosaicml/composer/pull/2412
* Fix torch 2.1.0 docker tag by j316chuck in https://github.com/mosaicml/composer/pull/2472
* Upstream Generate Callback by irenedea in https://github.com/mosaicml/composer/pull/2449
* Bump torch nightly docker image by j316chuck in https://github.com/mosaicml/composer/pull/2476
* Test pytorch 2.1.0 docker images on ci/cd by j316chuck in https://github.com/mosaicml/composer/pull/2469
* Fix huggingface tokenizer loading for slow tokenizers by dakinggg in https://github.com/mosaicml/composer/pull/2483
* Deprecate Fused LayerNorm by nik-mosaic in https://github.com/mosaicml/composer/pull/2475
* Transformers upgrade by dakinggg in https://github.com/mosaicml/composer/pull/2489
* Update RTD build config with build.os by bandish-shah in https://github.com/mosaicml/composer/pull/2490
* Upgrade torch docker version and tests by j316chuck in https://github.com/mosaicml/composer/pull/2488
* upgrade node by j316chuck in https://github.com/mosaicml/composer/pull/2492
* Gating tying modules w/ FSDP for torch 2.0 by bcui19 in https://github.com/mosaicml/composer/pull/2467
* Removing min_params by bcui19 in https://github.com/mosaicml/composer/pull/2494
* Fix torchmetrics backwards compatibility issue by eracah in https://github.com/mosaicml/composer/pull/2468
* Adding some fixes to FSDP tests by bcui19 in https://github.com/mosaicml/composer/pull/2495
* Fail count on mosaicml logger by mvpatel2000 in https://github.com/mosaicml/composer/pull/2496
* Remove PR curve metrics from backward compatibility test and skip torch 1.13 by eracah in https://github.com/mosaicml/composer/pull/2497
* filter warning by mvpatel2000 in https://github.com/mosaicml/composer/pull/2500
* Bump version to 0.16.1 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2498
* Skip metrics in state dict by mvpatel2000 in https://github.com/mosaicml/composer/pull/2501
* Add peak memory stats by mvpatel2000 in https://github.com/mosaicml/composer/pull/2504
* Fix sharded ckpt by mvpatel2000 in https://github.com/mosaicml/composer/pull/2505
* Bump gitpython from 3.1.31 to 3.1.34 by dependabot in https://github.com/mosaicml/composer/pull/2509
* Annotate `torch_prof_remote_file_name` as Optional by srstevenson in https://github.com/mosaicml/composer/pull/2512

New Contributors
* vivekgoe made their first contribution in https://github.com/mosaicml/composer/pull/2444
* irenedea made their first contribution in https://github.com/mosaicml/composer/pull/2437
* j316chuck made their first contribution in https://github.com/mosaicml/composer/pull/2463
* dbczumar made their first contribution in https://github.com/mosaicml/composer/pull/2442

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.16.0...v0.16.1

What's New
Streaming 0.11.0 🚀 (1711)
We've upgraded [streaming](https://github.com/mosaicml/streaming) to 0.11.0. StreamingDataset can now be used with custom Stream implementations via a registry. See [the documentation page](https://docs.mosaicml.com/projects/streaming/en/stable/dataset_configuration/mixing_data_sources.html) for example usage.

What's Changed
* Fix llama3 example yamls by j316chuck in https://github.com/mosaicml/llm-foundry/pull/1688
* Update example yamls to use newest foundry version by snarayan21 in https://github.com/mosaicml/llm-foundry/pull/1689
* Update datasets requirement from <2.21,>=2.20.0 to >=2.20.0,<3.2 by dependabot in https://github.com/mosaicml/llm-foundry/pull/1670
* Catch multiple slashes in source dataset into one slash by KuuCi in https://github.com/mosaicml/llm-foundry/pull/1697
* Make loaded peft adapters optionally trainable by snarayan21 in https://github.com/mosaicml/llm-foundry/pull/1701
* Adding preprocessors for QA and messages datasets by ShashankMosaicML in https://github.com/mosaicml/llm-foundry/pull/1700
* Update pycln by b-chu in https://github.com/mosaicml/llm-foundry/pull/1704
* Add permission error by b-chu in https://github.com/mosaicml/llm-foundry/pull/1703
* Update datasets requirement from <3.2,>=2.20.0 to >=2.20.0,<3.3 by dependabot in https://github.com/mosaicml/llm-foundry/pull/1698
* Bump coverage[toml] from 7.6.4 to 7.6.10 by dependabot in https://github.com/mosaicml/llm-foundry/pull/1702
* Update mosaicml-streaming to 0.11.0 by es94129 in https://github.com/mosaicml/llm-foundry/pull/1711
* Bump version to 0.17.0.dev0 by irenedea in https://github.com/mosaicml/llm-foundry/pull/1712


**Full Changelog**: https://github.com/mosaicml/llm-foundry/compare/v0.15.1...v0.16.0