Litdata

What's Changed
* Add register function for downloader by ouj in https://github.com/Lightning-AI/litdata/pull/496
* Allow for more lenient state resume. by JackUrb in https://github.com/Lightning-AI/litdata/pull/497
* Slighy faster speed by tchaton in https://github.com/Lightning-AI/litdata/pull/503
* Add s5cmd by tchaton in https://github.com/Lightning-AI/litdata/pull/506
* Feat: add support for gcp by deependujha in https://github.com/Lightning-AI/litdata/pull/504
* Bump version 0.2.42 by tchaton in https://github.com/Lightning-AI/litdata/pull/508

New Contributors
* ouj made their first contribution in https://github.com/Lightning-AI/litdata/pull/496

**Full Changelog**: https://github.com/Lightning-AI/litdata/compare/v0.2.41...v0.2.42

What's Changed
* doc: improve dev doc by deependujha in https://github.com/Lightning-AI/litdata/pull/488
* Expose optimize dns by tchaton in https://github.com/Lightning-AI/litdata/pull/498
* Update `_get_folder_size`: Reduce Logs noise and switch to `os.scandir` by bhimrazy in https://github.com/Lightning-AI/litdata/pull/499
* Bump: version 0.2.41 by deependujha in https://github.com/Lightning-AI/litdata/pull/500


**Full Changelog**: https://github.com/Lightning-AI/litdata/compare/v0.2.40...v0.2.41

What's Changed
* fix: `clean parquet dir cache` fixture by deependujha in https://github.com/Lightning-AI/litdata/pull/474
* Fix: Allow using `Machine` types in `map` by ethanwharris in https://github.com/Lightning-AI/litdata/pull/473
* 🛠️ Fix: Ensure `chunk_bytes` in `index.json` matches actual chunk file size by bhimrazy in https://github.com/Lightning-AI/litdata/pull/478
* fix: _get_folder_size fn by deependujha in https://github.com/Lightning-AI/litdata/pull/471
* Added boolean serialiser called by litdata.optimise() by DominiquePaul in https://github.com/Lightning-AI/litdata/pull/481
* Doc: improve dev doc & add ToDos by deependujha in https://github.com/Lightning-AI/litdata/pull/479
* upd: Add hf file download progress and update local file path by bhimrazy in https://github.com/Lightning-AI/litdata/pull/484
* fix: segmentation fault error in streaming tokens by bhimrazy in https://github.com/Lightning-AI/litdata/pull/485
* Warn user if `max_cache_size` is less than 25GB in StreamingDataset by bhimrazy in https://github.com/Lightning-AI/litdata/pull/489
* fix: Properly assign the chunks to the right worker by tchaton in https://github.com/Lightning-AI/litdata/pull/449
* Bump version to 0.2.40 by bhimrazy in https://github.com/Lightning-AI/litdata/pull/491

New Contributors
* ethanwharris made their first contribution in https://github.com/Lightning-AI/litdata/pull/473
* DominiquePaul made their first contribution in https://github.com/Lightning-AI/litdata/pull/481

**Full Changelog**: https://github.com/Lightning-AI/litdata/compare/v0.2.39...v0.2.40

What's Changed
* Feat: add support for HuggingFace datasets by deependujha in https://github.com/Lightning-AI/litdata/pull/462
* Using count-locks for multi-node-single-cache support by JackUrb in https://github.com/Lightning-AI/litdata/pull/468
* Bump version to 0.2.39 by tchaton in https://github.com/Lightning-AI/litdata/pull/470


**Full Changelog**: https://github.com/Lightning-AI/litdata/compare/v0.2.38...v0.2.39

What's Changed
* Adding upsampling support for StreamingDataset by JackUrb in https://github.com/Lightning-AI/litdata/pull/453
* ci: bump utils to latest / `main` by Borda in https://github.com/Lightning-AI/litdata/pull/457
* [WIP] Improve debugging of DDP hanging by tchaton in https://github.com/Lightning-AI/litdata/pull/456
* Feat: Add support for parquet files by deependujha in https://github.com/Lightning-AI/litdata/pull/443
* Nit: specify testpaths for pytest by deependujha in https://github.com/Lightning-AI/litdata/pull/464
* Update the S3 client initialization to explicitly use `boto3.Session()` by bhimrazy in https://github.com/Lightning-AI/litdata/pull/461
* Add more code owners by tchaton in https://github.com/Lightning-AI/litdata/pull/465
* Refactor: Parquet support PR by deependujha in https://github.com/Lightning-AI/litdata/pull/460
* Bump version 0.2.38 by tchaton in https://github.com/Lightning-AI/litdata/pull/466

New Contributors
* JackUrb made their first contribution in https://github.com/Lightning-AI/litdata/pull/453

**Full Changelog**: https://github.com/Lightning-AI/litdata/compare/v0.2.37...v0.2.38

What's Changed
* Fix renaming from cluster_id to cloud_account by tchaton in https://github.com/Lightning-AI/litdata/pull/455


**Full Changelog**: https://github.com/Lightning-AI/litdata/compare/v0.2.36...v0.2.37