Composer

Bug Fixes

**1. Increased Robustness to Checkpoint Loading**

We've patched several edge cases in loading sharded checkpoints, especially with DTensors, which should decrease memory usage when loading checkpoints. We've also hardened retry logic against object cloud failure, ensuring higher robustness to transient network issues.

What's Changed
* Raise daily test timeout by mvpatel2000 in https://github.com/mosaicml/composer/pull/3172
* fix remote file naming by cli99 in https://github.com/mosaicml/composer/pull/3173
* [fix] DTensor + SHARD_GRAD_OP + use_orig_params by bigning in https://github.com/mosaicml/composer/pull/3175
* Bump db sdk by dakinggg in https://github.com/mosaicml/composer/pull/3176
* Build latest pytorch nightly images by dakinggg in https://github.com/mosaicml/composer/pull/3179
* Add FP8 TransformerEngine activation checkpointing by cli99 in https://github.com/mosaicml/composer/pull/3156
* Enabling the computation of validation loss and other metrics when using sequence parallelism by ShashankMosaicML in https://github.com/mosaicml/composer/pull/3183
* Update mosaic_fsdp_utils.py by vchiley in https://github.com/mosaicml/composer/pull/3185
* Fix the FSDP.optim_state_dict_to_load OOM by bigning in https://github.com/mosaicml/composer/pull/3184
* Revert "Update mosaic_fsdp_utils.py" by vchiley in https://github.com/mosaicml/composer/pull/3187
* Bump databricks-sdk from 0.24.0 to 0.25.1 by dependabot in https://github.com/mosaicml/composer/pull/3190
* Add version tag to local builds by mvpatel2000 in https://github.com/mosaicml/composer/pull/3188
* Update `NeptuneLogger` by AleksanderWWW in https://github.com/mosaicml/composer/pull/3165
* Filter neptune warning in doctests by mvpatel2000 in https://github.com/mosaicml/composer/pull/3195
* Removal of metrics deepcopy before computing the metrics by gregjauvion in https://github.com/mosaicml/composer/pull/3180
* Fix MLFlow Tag Name for Resumption by KuuCi in https://github.com/mosaicml/composer/pull/3194
* Fix mistral gating by dakinggg in https://github.com/mosaicml/composer/pull/3199
* Bump version to 0.21.3 by mvpatel2000 in https://github.com/mosaicml/composer/pull/3198

New Contributors
* gregjauvion made their first contribution in https://github.com/mosaicml/composer/pull/3180

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.21.2...v0.21.3

Bug Fixes

**1. Enable torch 2.2.2 (3161)**

Composer currently monkeypatches PyTorch for nightly versions in order to fix upstream bugs. With the release of torch 2.2.2, these monkeypatches were mistakenly applied to the stable release due to incorrect gating on imports. This release fixes the gating, enabling torch 2.2.2.

**2. MPS Metric Computation on CPU (3105)**

Due to bugs in computing torchmetrics on Mac devices, we move metric computation onto CPU. This previously had issues with data not properly moving to CPU.

Thank you to hyenal for this contribution!

**3. Batch Sampler Support (3105)**

Composer now supports batch sampler, which previously resulted in an error if specified in the dataloader.

Thank you to Ghelfi for this contribution!

What's Changed
* Make codequality callable by mvpatel2000 in https://github.com/mosaicml/composer/pull/3133
* Explicitly print checkpoint downloading exception by bigning in https://github.com/mosaicml/composer/pull/3131
* Change release actions by mvpatel2000 in https://github.com/mosaicml/composer/pull/3136
* Passing rank and num_replicas to dist.get_sampler by ShashankMosaicML in https://github.com/mosaicml/composer/pull/3137
* Fix broadcast by mvpatel2000 in https://github.com/mosaicml/composer/pull/3138
* Compressor fixes by mbway in https://github.com/mosaicml/composer/pull/3142
* In case of MPS device also copy batch to CPU by hyenal in https://github.com/mosaicml/composer/pull/3105
* Composer object store download retry by bigning in https://github.com/mosaicml/composer/pull/3140
* Bump databricks-sdk from 0.22.0 to 0.23.0 by dependabot in https://github.com/mosaicml/composer/pull/3144
* Update transformers requirement from !=4.34.0,<4.39,>=4.11 to >=4.11,!=4.34.0,<4.40 by dependabot in https://github.com/mosaicml/composer/pull/3148
* Update protobuf requirement from <3.21 to <5.27 by dependabot in https://github.com/mosaicml/composer/pull/3147
* Bump traitlets from 5.14.1 to 5.14.2 by dependabot in https://github.com/mosaicml/composer/pull/3145
* Bump to 0.21 by mvpatel2000 in https://github.com/mosaicml/composer/pull/3150
* Fixing sequence parallel error conditions and adding type float for microbatch_size in typehints by ShashankMosaicML in https://github.com/mosaicml/composer/pull/3139
* Fix torch monkeypatch version check by dakinggg in https://github.com/mosaicml/composer/pull/3155
* Update torchmetrics requirement from <1.3.2,>=0.10.0 to >=0.10.0,<1.3.3 by dependabot in https://github.com/mosaicml/composer/pull/3157
* Bump gitpython from 3.1.42 to 3.1.43 by dependabot in https://github.com/mosaicml/composer/pull/3160
* Prevent crash if signal handler cannot be set by mbway in https://github.com/mosaicml/composer/pull/3152
* Pin pillow for code quality workflow by dakinggg in https://github.com/mosaicml/composer/pull/3162
* Fix torch version check by dakinggg in https://github.com/mosaicml/composer/pull/3161
* add more retry to checkpoint downloading by bigning in https://github.com/mosaicml/composer/pull/3164
* Append to gpu rank log files instead of throwing error by jjanezhang in https://github.com/mosaicml/composer/pull/3166
* Call `set_epoch` on `Dataloader.batch_sampler` if defined by Ghelfi in https://github.com/mosaicml/composer/pull/3124
* Bump version to 0.21.2 by mvpatel2000 in https://github.com/mosaicml/composer/pull/3168

New Contributors
* hyenal made their first contribution in https://github.com/mosaicml/composer/pull/3105
* Ghelfi made their first contribution in https://github.com/mosaicml/composer/pull/3124

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.21.1...v0.21.2

**Bug Fixes**

**1. Fix to HSDP checkpoint loading**

The previous release broke checkpoint loading when using HSDP with mutliple replicas. This patch release fixes checkpoint loading.

What's Changed
* Fix broadcast by mvpatel2000 in https://github.com/mosaicml/composer/pull/3138

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.21.0...v0.21.1

What's New

**1. Aggregate Memory Monitoring (3042)**

The Memory Monitor callback now supports aggregating memory statistics across nodes. Getting summary stats for a run's memory usage across the cluster can dramatically help debug straggler nodes or non-homogenous workloads. The memory monitor can now aggregate and log combined values at a user specified frequency.

Example:

from composer import Trainer
from composer.callbacks import MemoryMonitor

trainer = Trainer(
model=model,
train_dataloader=train_dataloader,
optimizers=optimizer,
max_duration="1ep",
callbacks=[
MemoryMonitor(
dist_aggregate_batch_interval=10, aggregate every 10 batches
)
],
)


**2. Advanced Compression Options (3118)**

Large model checkpoints can be expensive to store and transfer. In this release, we've upgraded our compression support to accept several new formats which result in better compression-time tradeoffs using CLI tools. In order to use compression, you can post-fix your checkpoint name with a compression path. We know support the following extensions:
- bz2
- gz
- lz4
- lzma
- lzo
- xz
- zst

Example:

from composer import Trainer
from composer.callbacks import MemoryMonitor

trainer = Trainer(
model=model,
train_dataloader=train_dataloader,
optimizers=optimizer,
max_duration="1ep",
save_filename='ep{epoch}-ba{batch}-rank{rank}.pt.lz4',
)


Thank you to mbway for adding this support!

What's Changed
* Rename composer_run_name tag to run_name when logging to MLflow by jerrychen109 in https://github.com/mosaicml/composer/pull/3040
* enable aggregate mem monitoring by vchiley in https://github.com/mosaicml/composer/pull/3042
* Bump junitparser from 3.1.1 to 3.1.2 by dependabot in https://github.com/mosaicml/composer/pull/3056
* Add SHARD_GRAD_OP to device mesh error check by mvpatel2000 in https://github.com/mosaicml/composer/pull/3058
* Add torch 2.2.1 support by mvpatel2000 in https://github.com/mosaicml/composer/pull/3059
* Use testing repo actions for linting by b-chu in https://github.com/mosaicml/composer/pull/3060
* Link autoresume docs back to watchdog by aspfohl in https://github.com/mosaicml/composer/pull/3052
* Deprecate get_state and remove deprecations by b-chu in https://github.com/mosaicml/composer/pull/3017
* Bump version to 0.20.1 by mvpatel2000 in https://github.com/mosaicml/composer/pull/3061
* Remove s3_bucket pytest cli flag by b-chu in https://github.com/mosaicml/composer/pull/3064
* Remove s3_bucket flag from gpu test by b-chu in https://github.com/mosaicml/composer/pull/3065
* Clean Up OOM Observer Remote Uploader Download path by j316chuck in https://github.com/mosaicml/composer/pull/3070
* Fix daily test for iteration by b-chu in https://github.com/mosaicml/composer/pull/3068
* Remove "generation_length" in favor of "generation_kwargs" by maxisawesome in https://github.com/mosaicml/composer/pull/3014
* Bump packaging by mvpatel2000 in https://github.com/mosaicml/composer/pull/3072
* Use ci-testing repo for CPU and GPU tests by b-chu in https://github.com/mosaicml/composer/pull/3062
* Add new torch monkeypatches to Composer by mvpatel2000 in https://github.com/mosaicml/composer/pull/3063
* Add initial support for neuron devices by bfontain in https://github.com/mosaicml/composer/pull/3049
* Stripping whitespaces as default for QATask ICL eval by ksreenivasan in https://github.com/mosaicml/composer/pull/3073
* Add ICL base class to __all__ by mvpatel2000 in https://github.com/mosaicml/composer/pull/3079
* pass prelimiter into ALL ICL datasets by eitanturok in https://github.com/mosaicml/composer/pull/3069
* Bump sentencepiece from 0.1.99 to 0.2.0 by dependabot in https://github.com/mosaicml/composer/pull/3083
* Add Iteration related Events to callbacks by b-chu in https://github.com/mosaicml/composer/pull/3077
* Add Iteration related Events by b-chu in https://github.com/mosaicml/composer/pull/3076
* Bump CI/CD to v3 by mvpatel2000 in https://github.com/mosaicml/composer/pull/3086
* Add docstring to _iteration_length by b-chu in https://github.com/mosaicml/composer/pull/3088
* Check FSDP module has _device_mesh before getting it by eracah in https://github.com/mosaicml/composer/pull/3091
* Bump minor version in base image by mvpatel2000 in https://github.com/mosaicml/composer/pull/3092
* Enforce async logging flush in mlflow logger at `post_close` call by chenmoneygithub in https://github.com/mosaicml/composer/pull/3093
* Warning log to info log by aspfohl in https://github.com/mosaicml/composer/pull/3096
* Bump transformers by dakinggg in https://github.com/mosaicml/composer/pull/3095
* Change style for splitting on commas by b-chu in https://github.com/mosaicml/composer/pull/3078
* Remove slash by b-chu in https://github.com/mosaicml/composer/pull/3098
* Allowing for fractional number of samples per rank by ShashankMosaicML in https://github.com/mosaicml/composer/pull/3075
* Output eval logging (batch level) by maxisawesome in https://github.com/mosaicml/composer/pull/2977
* Replace errors with warnings for eval args by mvpatel2000 in https://github.com/mosaicml/composer/pull/3100
* Ability to load sharded checkpoints with remote symlink load_path by eracah in https://github.com/mosaicml/composer/pull/3097
* Improvements to `NeptuneLogger` by AleksanderWWW in https://github.com/mosaicml/composer/pull/3085
* Revert "Improvements to `NeptuneLogger`" by mvpatel2000 in https://github.com/mosaicml/composer/pull/3111
* Bump mlflow min pin by dakinggg in https://github.com/mosaicml/composer/pull/3110
* Fix rounding issue in interval calculation by dakinggg in https://github.com/mosaicml/composer/pull/3109
* Bump coverage[toml] from 7.4.1 to 7.4.3 by dependabot in https://github.com/mosaicml/composer/pull/3102
* Uses v0.0.4 of ci-testing by b-chu in https://github.com/mosaicml/composer/pull/3112
* Add versioned deprecation warning by irenedea in https://github.com/mosaicml/composer/pull/2984
* Update Flash Attention to 2.5.5 by Skylion007 in https://github.com/mosaicml/composer/pull/3113
* Setting the max duration to current timestamp in the same units as cu… by ShashankMosaicML in https://github.com/mosaicml/composer/pull/3090
* Making default_split_batch public by ShashankMosaicML in https://github.com/mosaicml/composer/pull/3116
* Adding log exception to Mosaic Logger by jjanezhang in https://github.com/mosaicml/composer/pull/3089
* Add checks to schedulers by b-chu in https://github.com/mosaicml/composer/pull/3115
* Removed default attrs from exception class in the attrs dict by jjanezhang in https://github.com/mosaicml/composer/pull/3126
* Bump coverage[toml] from 7.4.3 to 7.4.4 by dependabot in https://github.com/mosaicml/composer/pull/3121
* Refactor initialization by Practicinginhell in https://github.com/mosaicml/composer/pull/3127
* Bump databricks sdk version by dakinggg in https://github.com/mosaicml/composer/pull/3128
* Update packaging requirement from <23.3,>=21.3.0 to >=21.3.0,<24.1 by dependabot in https://github.com/mosaicml/composer/pull/3122
* Remove rng from save_weights_only ckpt by eracah in https://github.com/mosaicml/composer/pull/3129
* More compression options by mbway in https://github.com/mosaicml/composer/pull/3118
* Only broadcast distcp files by mvpatel2000 in https://github.com/mosaicml/composer/pull/3130
* Bump version to 0.21 by mvpatel2000 in https://github.com/mosaicml/composer/pull/3132

New Contributors
* ksreenivasan made their first contribution in https://github.com/mosaicml/composer/pull/3073
* eitanturok made their first contribution in https://github.com/mosaicml/composer/pull/3069
* Practicinginhell made their first contribution in https://github.com/mosaicml/composer/pull/3127
* mbway made their first contribution in https://github.com/mosaicml/composer/pull/3118

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.20.1...v0.21.0

What's New

1. Torch 2.2.1 Support

Composer now supports torch 2.2.1! We've raised the pin to allow the latest torch, and we've upstreamed all torch monkeypatches so Composer can run out of the box with the latest and greatest torch features.

What's Changed
* Add torch 2.2.1 support by mvpatel2000 in https://github.com/mosaicml/composer/pull/3059
* Bump version to 0.20.1 by mvpatel2000 in https://github.com/mosaicml/composer/pull/3061

What's New

1. New Neptune Logger
Composer now supports logging training data to [neptune.ai](https://neptune.ai/) using the `NeptuneLogger`. To get started:

python
neptune_project = 'test_project'
neptune_api_token = 'test_token'

neptune_logger = NeptuneLogger(
project=neptune_project,
api_token=neptune_api_token,
rank_zero_only=False,
mode='debug',
upload_artifacts=True,
)


We also have an [example project](https://app.neptune.ai/o/showcase/org/mosaicml-composer/runs/details?viewId=standard-view&detailsTab=dashboard&dashboardId=9b1f1fae-f543-41d1-a778-8604c9b6503d&shortId=MMLCOMP-3) demonstrating all the awesome things you can do with this integration!


Additional information on the `NeptuneLogger` can be found in the [docs](https://docs.mosaicml.com/projects/composer/en/latest/api_reference/generated/composer.loggers.NeptuneLogger.html).


2. OOM observer callback with memory visualizations

Composer now has an OOM observer callback. When a model runs out of memory, this callback helps produce a trace which identifies memory allocations, which can be critical to designing strategies to mitigate memory usage.

Example:
python
from composer import Trainer
from composer.callbacks import OOMObserver
constructing trainer object with this callback
trainer = Trainer(
model=model,
train_dataloader=train_dataloader,
eval_dataloader=eval_dataloader,
optimizers=optimizer,
max_duration="1ep",
callbacks=[
OOMObserver(
folder="traces",
overwrite=true,
filename="rank{rank}_oom",
remote_filename="oci://bucket_name/{run_name}/oom_traces/rank{rank}_oom",
)
],
)


OOM Visualization:



3. Log all gpu rank stdout/err to MosaicML platform

Composer has expanded it's integration with the MosaicML platform.. Now, we can view all gpu rank stdout/stderrs with MCLI logs to enable more comprehensive analysis of jobs.

Example:


mcli logs <run-name> --node x --gpu x

Note, this defaults to node rank 0 if `--node` is not provided.

Also, we can find the logs of any global gpu rank with the command:

mcli logs <run-name> --global-gpu-rank x


Bug Fixes
* Only save RNG on rank 0 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2998
* [Auto-microbatch fix] FSDP reshard and cleanup after OOM to fix the cuda memory leak by bigning in https://github.com/mosaicml/composer/pull/3030
* Fix skip_first for profiler during resumption by bigning in https://github.com/mosaicml/composer/pull/2986
* Race condition fix in checkpoint loading util by jessechancy in https://github.com/mosaicml/composer/pull/3001

What's Changed
* Remove .ci folder and move FILE_HEADER and CODEOWNERS by irenedea in https://github.com/mosaicml/composer/pull/2957
* Modify UCObjectStore.list_objects to lists all files recursively by irenedea in https://github.com/mosaicml/composer/pull/2959
* Refactor MemorySnapshot by cli99 in https://github.com/mosaicml/composer/pull/2960
* Log all gpu rank stdout/err to MosaicML platform by jjanezhang in https://github.com/mosaicml/composer/pull/2839
* Add Torch 2.2 tests by mvpatel2000 in https://github.com/mosaicml/composer/pull/2970
* Memory snapshot dump pickle by cli99 in https://github.com/mosaicml/composer/pull/2968
* Neptune logger by AleksanderWWW in https://github.com/mosaicml/composer/pull/2447
* Fix torch pins in tests by mvpatel2000 in https://github.com/mosaicml/composer/pull/2973
* Add a register_model_with_run_id api to MLflowLogger by dakinggg in https://github.com/mosaicml/composer/pull/2967
* Remove bespoke codeowners by mvpatel2000 in https://github.com/mosaicml/composer/pull/2971
* Add a BEFORE_LOAD event by snarayan21 in https://github.com/mosaicml/composer/pull/2974
* More torch 2.2 fixes by mvpatel2000 in https://github.com/mosaicml/composer/pull/2975
* Adding the step argument to logger.log_table by ShashankMosaicML in https://github.com/mosaicml/composer/pull/2961
* Fix daily tests for torch 2.2 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2980
* Format load_path with name by mvpatel2000 in https://github.com/mosaicml/composer/pull/2978
* Bump to 0.19.1 by mvpatel2000 in https://github.com/mosaicml/composer/pull/2979
* Fix UC object store bugfix by nancyhung in https://github.com/mosaicml/composer/pull/2982
* [Bugfix][UC] Add back the full object path by nancyhung in https://github.com/mosaicml/composer/pull/2988
* Minor cleanup of UC get_object_size by dakinggg in https://github.com/mosaicml/composer/pull/2989
* Pin UC to earlier version by dakinggg in https://github.com/mosaicml/composer/pull/2990
* Revert "fix skip_first for resumption" by bigning in https://github.com/mosaicml/composer/pull/2991
* Broadcast files for HSDP by mvpatel2000 in https://github.com/mosaicml/composer/pull/2914
* Bump ipykernel from 6.29.0 to 6.29.2 by dependabot in https://github.com/mosaicml/composer/pull/2994
* Bump yamllint from 1.33.0 to 1.34.0 by dependabot in https://github.com/mosaicml/composer/pull/2995
* Refactor `update_metric` by maxisawesome in https://github.com/mosaicml/composer/pull/2965
* Add azure integration test by mvpatel2000 in https://github.com/mosaicml/composer/pull/2996
* Fix Profiler schedule skip_first by bigning in https://github.com/mosaicml/composer/pull/2992
* Remove planner validation by mvpatel2000 in https://github.com/mosaicml/composer/pull/2985
* Fix load for non-HSDP device mesh by mvpatel2000 in https://github.com/mosaicml/composer/pull/2997
* Update NCCL arg since torch deprecated old one by mvpatel2000 in https://github.com/mosaicml/composer/pull/3000
* Add bias argument to LPLN by mvpatel2000 in https://github.com/mosaicml/composer/pull/2999
* Revert "Add bias argument to LPLN" by mvpatel2000 in https://github.com/mosaicml/composer/pull/3003
* Revert "Update NCCL arg since torch deprecated old one" by mvpatel2000 in https://github.com/mosaicml/composer/pull/3004
* Add torch 2.3 image for aws cluster by j316chuck in https://github.com/mosaicml/composer/pull/3002
* Patch torch 2.3 aws naming by j316chuck in https://github.com/mosaicml/composer/pull/3006
* Add debug log before training loop starts by mvpatel2000 in https://github.com/mosaicml/composer/pull/3005
* Deprecate ffcv code by j316chuck in https://github.com/mosaicml/composer/pull/3007
* Remove log for mosaicml logger by mvpatel2000 in https://github.com/mosaicml/composer/pull/3008
* [EASY] Always log 1st batch when resuming training by bigning in https://github.com/mosaicml/composer/pull/3009
* Use reusable actions for linting by b-chu in https://github.com/mosaicml/composer/pull/2948
* Make CodeEval respect device_eval_batch_size by josejg in https://github.com/mosaicml/composer/pull/2969
* Use Mosaic constant for GPU file prefix by jjanezhang in https://github.com/mosaicml/composer/pull/3018
* Fall back to normal logging when gpu prefix is not present by jjanezhang in https://github.com/mosaicml/composer/pull/3020
* Revert "Use reusable actions for linting" to fix CI/CD by mvpatel2000 in https://github.com/mosaicml/composer/pull/3023
* Change to pull_request_target by b-chu in https://github.com/mosaicml/composer/pull/3025
* Bump gitpython from 3.1.41 to 3.1.42 by dependabot in https://github.com/mosaicml/composer/pull/3031
* Bump yamllint from 1.34.0 to 1.35.1 by dependabot in https://github.com/mosaicml/composer/pull/3034
* Update torchmetrics requirement from <1.3.1,>=0.10.0 to >=0.10.0,<1.3.2 by dependabot in https://github.com/mosaicml/composer/pull/3035
* Bump pypandoc from 1.12 to 1.13 by dependabot in https://github.com/mosaicml/composer/pull/3033
* Add tensorboard images support by Menduist in https://github.com/mosaicml/composer/pull/3021
* Add sorted to logs for checkpoint broadcast by mvpatel2000 in https://github.com/mosaicml/composer/pull/3036
* Friendlier device mesh error by mvpatel2000 in https://github.com/mosaicml/composer/pull/3039
* Upgrade to python3.11 for torch nightly by j316chuck in https://github.com/mosaicml/composer/pull/3038
* Download symlink once by mvpatel2000 in https://github.com/mosaicml/composer/pull/3043
* Add min size to OCI download by mvpatel2000 in https://github.com/mosaicml/composer/pull/3044
* Lint fix by mvpatel2000 in https://github.com/mosaicml/composer/pull/3045
* Revert "Change to pull_request_target " by mvpatel2000 in https://github.com/mosaicml/composer/pull/3047
* Bump composer version 0.19.2 by j316chuck in https://github.com/mosaicml/composer/pull/3048
* Update XLA support by bfontain in https://github.com/mosaicml/composer/pull/2964
* Bump composer version 0.20.0 by j316chuck in https://github.com/mosaicml/composer/pull/3051
* Update ruff. Fix PLE & LOG lints by Skylion007 in https://github.com/mosaicml/composer/pull/3050

New Contributors
* AleksanderWWW made their first contribution in https://github.com/mosaicml/composer/pull/2447
* ShashankMosaicML made their first contribution in https://github.com/mosaicml/composer/pull/2961
* nancyhung made their first contribution in https://github.com/mosaicml/composer/pull/2982
* bigning made their first contribution in https://github.com/mosaicml/composer/pull/2986
* jessechancy made their first contribution in https://github.com/mosaicml/composer/pull/3001
* josejg made their first contribution in https://github.com/mosaicml/composer/pull/2969
* Menduist made their first contribution in https://github.com/mosaicml/composer/pull/3021
* bfontain made their first contribution in https://github.com/mosaicml/composer/pull/2964

**Full Changelog**: https://github.com/mosaicml/composer/compare/v0.19.1...v0.20.0