Ray

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:
- Allow user to configure timeout for actor pool (45508)
- Add override_num_blocks to from_pandas and perform auto-partition (44937)
- Upgrade Arrow version to 16 in CI (45565)

💫 Enhancements:
- Clarify that num_rows_per_file isn't strict (45529)
- Record more telemetry for newly added datasources (45647)
- Avoid pickling LanceFragment when creating read tasks for Lance (45392)

Ray Train<a id="ray-train"></a>

📖 Documentation:
- [HPU] Add example of Stable Diffusion fine-tuning and serving on Intel Gaudi (45217)
- [HPU] Add example of Llama-2 fine-tuning on Intel Gaudi (44667)


Ray Tune<a id="ray-tune"></a>
🏗 Architecture refactoring:
- Improve excessive syncing warning and deprecate TUNE_RESULT_DIR, RAY_AIR_LOCAL_CACHE_DIR, local_dir (45210)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:
- Clean up Serve proxy files ([45486](https://github.com/ray-project/ray/pull/45486))

📖 Documentation:
- vllm example to serve llm models ([45430](https://github.com/ray-project/ray/pull/45430))

RLLib<a id="rllib"></a>

💫 Enhancements:
- DreamerV3 on tf: Bug fix, so it can run again with tf==2.11.1 (2.11.0 is not available anymore) (45419); Added weekly release test for DreamerV3.
- Added support for multi-agent off-policy algorithms (DQN and SAC) in the new (45182)
- Config option for APPO/IMPALA to change number of GPU-loader threads (45467)

🔨 Fixes:
- Various MetricsLogger bug fixes (45543, 45585, 45575)
- Other fixes: 45588, 45617, 45517, 45465

📖 Documentation:
- Example script for new API stack: How-to restore 1 of n agents from a checkpoint. (45462)
- Example script for new API stack: Autoregressive action module. 45525

Ray Core<a id="ray-core"></a>

💫 Enhancements:
- Improve node death observability (45320, 45357, 45533, 45644, 45497)
- Ray c++ backend structured logging (44468)

🔨 Fixes:
- Fix worker crash when getting actor name from runtime context (45194)
- log dedup should not dedup number only lines (45385)

📖 Documentation:
- Improve doc for `--object-store-memory` to describe how the default value is set ([45301](https://github.com/ray-project/ray/pull/45301))

Dashboard<a id="ray-dashboard"></a>
🔨 Fixes:
- Move Job package uploading to another thread to unblock the event loop. ([45282](https://github.com/ray-project/ray/pull/45282))

Many thanks to all those who contributed to this release: maxliuofficial, simonsays1980, GeneDer, dudeperf3ct, khluu, justinvyu, andrewsykim, Catch-Bull, zcin, bveeramani, rynewang, angelinalg, matthewdeng, jjyao, kira-lin, harborn, hongchaodeng, peytondmurray, aslonnie, timkpaine, 982945902, maxpumperla, stephanie-wang, ruisearch42, alanwguo, can-anyscale, c21, Atry, KamenShah, sven1977, raulchen

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:
- Add support for using GPUs with map_groups (45305)
- Add support for using actors with map_groups (45310)

💫 Enhancements:
- Refine exception handling from arrow data conversion (45294)

🔨 Fixes:
- Fix Ray databricks UC reader with dynamic Databricks notebook scope token (45153)
- Fix bug where you can't return objects and array from UDF (45287 )
- Fix bug where map_groups triggers execution during input validation (45314)

Ray Tune<a id="ray-tune"></a>
🔨 Fixes:
- [tune] Fix PB2 scheduler error resulting from trying to sort by Trial objects (45161)

Ray Serve<a id="ray-serve"></a>
🔨 Fixes:
- Log application unhealthy errors at error level instead of warning level ([45211](https://github.com/ray-project/ray/pull/45211))

RLLib<a id="rllib"></a>
💫 Enhancements:
- Examples and `tuned_examples` learning test for new API stack are now “self-executable” (don’t require a third-party script anymore to run them). + WandB support. ([45023](https://github.com/ray-project/ray/pull/45023))

🔨 Fixes:
- Fix result dict “spam” (duplicate, deprecated keys, e.g. “sampler_results” dumped into top level). ([45330](https://github.com/ray-project/ray/pull/45330))

📖 Documentation:
- Add example for training with fractional GPUs on new API stack. ([45379](https://github.com/ray-project/ray/pull/45379))
- Cleanup examples folder and remove deprecated sub directories. ([45327](https://github.com/ray-project/ray/pull/45327))

Ray Core<a id="ray-core"></a>
💫 Enhancements:
- [Logs] Add runtime env started logs to job driver ([45255](https://github.com/ray-project/ray/pull/45255))
- `ray.util.collective` support `torch.bfloat16` ([39845](https://github.com/ray-project/ray/pull/39845))
- [Core] Better propagate node death information ([45128](https://github.com/ray-project/ray/pull/45128))

🔨 Fixes:
- [Core] Fix worker process leaks after job finishes ([44214](https://github.com/ray-project/ray/pull/44214))


Many thanks to all those who contributed to this release: hongchaodeng, khluu, antoni-jamiolkowski, ameroyer, bveeramani, can-anyscale, WeichenXu123, peytondmurray, jackhumphries, kevin85421, jjyao, robcaulk, rynewang, scottsun94, swang, GeneDer, zcin, ruisearch42, aslonnie, angelinalg, raulchen, ArthurBook, sven1977, wuxibin89

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:
- Add function to dynamically generate `ray_remote_args` for Map APIs (45143)
- Allow manually setting resource limits for training jobs (45188)

💫 Enhancements:
- Introduce abstract interface for data autoscaling (45002)
- Add debugging info for `SplitCoordinator` (45226)

🔨 Fixes:
- Don’t show `AllToAllOperator` progress bar if the disable flag is set (45136)
- Don't load Arrow `PyExtensionType` by default (45084)
- Don't raise batch size error if `num_gpus=0` (45202)

Ray Train<a id="ray-train"></a>

💫 Enhancements:
- [XGBoost][LightGBM] Update RayTrainReportCallback to only save checkpoints on rank 0 (45083)

Ray Core<a id="ray-core"></a>

🔨 Fixes:
- Fix the cpu percentage metrics for dashboard process (45124)

Dashboard<a id="dashboard"></a>

💫 Enhancements:
- Improvements to log viewer so line numbers do not get selected when copying text.
- Improvements to the log viewer to avoid unnecessary re-rendering which causes text selection to clear.


Many thanks to all those who contributed to this release: justinvyu, simonsays1980, chris-ray-zhang, kevin85421, angelinalg, rynewang, brycehuang30, alanwguo, jjyao, shaikhismail, khluu, can-anyscale, bveeramani, jrosti, WeichenXu123, MortalHappiness, raulchen, scottjlee, ruisearch42, aslonnie, alexeykudinkin

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New features:
- Add `read_lance` API to read Lance Dataset (45106)

🔨 Fixes:
- Retry RaySystemError application errors (45079)

📖 Documentation:
- Fix broken references in data documentation (44956)

Ray Train<a id="ray-train"></a>

📖 Documentation:
- Fix broken links in Train documentation (44953)

Ray Tune<a id="ray-tune"></a>

📖 Documentation:
- Update Hugging Face example to add reference (42771)

🏗 Architecture refactoring:
- Remove deprecated `ray.air.callbacks` modules (45104)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:
- Allow methods to pass type serve.batch type hint (45004)
- Allow configuring Serve control loop interval (45063)

🔨 Fixes:
- Fix bug with controller failing to recover for autoscaling deployments (45118)
- Fix control+c after serve run doesn't shutdown serve components (45087)
- Fix lightweight update max ongoing requests (45006)



RLlib<a id="rllib"></a>

🎉 New Features:
- New MetricsLogger API now fully functional on the new API stack (working now also inside Learner classes, i.e. loss functions). ([44995](https://github.com/ray-project/ray/pull/44995), [#45109](https://github.com/ray-project/ray/pull/45109))

💫 Enhancements:
- Renamings and cleanups (toward new API stack and more consistent naming schemata): WorkerSet -> EnvRunnerGroup, DEFAULT_POLICY_ID -> DEFAULT_MODULE_ID, config.rollouts() -> config.env_runners(), etc.. ([45022](https://github.com/ray-project/ray/pull/45022), [#44920](https://github.com/ray-project/ray/pull/44920))
- Changed behavior of `EnvRunnerGroup.foreach_worker…` methods to new defaults: `mark_healthy=True` (used to be False) and `healthy_only=True` (used to be False). ([44993](https://github.com/ray-project/ray/pull/44993))
- Fix `get_state()/from_state()` methods in SingleAgent- and MultiAgentEpisodes. ([45012](https://github.com/ray-project/ray/pull/45012))

🔨 Fixes:
- Bug fix for (torch) global_norm clipping overflow problem: ([45055](https://github.com/ray-project/ray/pull/45055))
- Various bug- and test case fixes: [45030](https://github.com/ray-project/ray/pull/45030), [#45031](https://github.com/ray-project/ray/pull/45031), [#45070](https://github.com/ray-project/ray/pull/45070), [#45053](https://github.com/ray-project/ray/pull/45053), [#45110](https://github.com/ray-project/ray/pull/45110), [#45111](https://github.com/ray-project/ray/pull/45111)

📖 Documentation:
- Example scripts using the MetricsLogger for env rendering and recording w/ WandB: [45073](https://github.com/ray-project/ray/pull/45073), [#45107](https://github.com/ray-project/ray/pull/45107)

Ray Core<a id="ray-core"></a>

🔨 Fixes:
- Fix `ray.init(logging_format)` argument is ignored (45037)
- Handle unserializable user exception (44878)
- Fix dashboard process event loop blocking issues (45048, 45047)

Dashboard<a id="dashboard"></a>
🔨 Fixes:
- Fix Nodes page sorting not working correctly.
- Add back “actors per page” UI control in the actors page.

Many thanks to all those who contributed to this release: rynewang, can-anyscale, scottsun94, bveeramani, ceddy4395, GeneDer, zcin, JoshKarpel, nikitavemuri, stephanie-wang, jackhumphries, matthewdeng, yash97, simonsays1980, peytondmurray, evalaiyc98, c21, alanwguo, shrekris-anyscale, kevin85421, hongchaodeng, sven1977, st--, khluu

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

💫 Enhancements:
- Dedupe repeated schema during `ParquetDatasource` metadata prefetching (44750)
- Update `map_groups` implementation to better handle large outputs (44862)
- Deprecate `prefetch_batches` arg of `iter_rows` and change default value (44982)
- Adding in default behavior to false for creating dirs on s3 writes (44972)
- Make internal UDF names more descriptive (44985)
- Make `name` a required argument for `AggregateFn` (44880)

📖 Documentation:
- Add key concepts to and revise "Data Internals" page (44751)


Ray Train<a id="ray-train"></a>

💫 Enhancements:
- Setup XGBoost `CommunicatorContext` automatically (44883)
- Track Train Run Info with `TrainStateActor` (44585)

📖 Documentation:
- Add documentation for `accelerator_type` (44882)
- Update Ray Train example titles (44369)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:
- Remove trial table when running Ray Train in a Jupyter notebook (44858)
- Clean up temporary checkpoint directories for class Trainables (ex: RLlib) (44366)

📖 Documentation:
- Fix minor doc format issues (44865)
- Remove outdated ScalingConfig references (44918)

Ray Serve<a id="ray-serve"></a>

💫 Enhancements:
- Handle push metric interval is now configurable with environment variable RAY_SERVE_HANDLE_METRIC_PUSH_INTERVAL_S (32920)
- Improve performance of developer API serve.get_app_handle (44812)

🔨 Fixes:
- Fix memory leak in handles for autoscaling deployments (the leak happens when
- RAY_SERVE_COLLECT_AUTOSCALING_METRICS_ON_HANDLE=1) (44877)


RLlib<a id="rllib"></a>

🎉 New Features:
- Introduce `MetricsLogger`, a unified API for users of RLlib to log custom metrics and stats in all of RLlib’s components (Algorithm, EnvRunners, and Learners). Rolled out for new API stack for Algorithm (`training_step`) and EnvRunners (custom callbacks). `Learner` (custom loss functions) support in progress. [44888](https://github.com/ray-project/ray/pull/44888), [#44442](https://github.com/ray-project/ray/pull/44442)
- Introduce “inference-only” (slim) mode for RLModules that run inside an EnvRunner (and thus don’t require value-functions or target networks): [44797](https://github.com/ray-project/ray/pull/44797)

💫 Enhancements:
- MultiAgentEpisodeReplayBuffer for new API stack (preparation for multi-agent support of SAC and DQN): [44450](https://github.com/ray-project/ray/pull/44450)
- AlgorithmConfig cleanup and renaming of properties and methods for better consistency/transparency: [44896](https://github.com/ray-project/ray/pull/44896)

🔨 Fixes:
- Various minor bug fixes: [44989](https://github.com/ray-project/ray/pull/44989), [#44988](https://github.com/ray-project/ray/pull/44988), [#44891](https://github.com/ray-project/ray/pull/44891), [#44898](https://github.com/ray-project/ray/pull/44898), [#44868](https://github.com/ray-project/ray/pull/44868), [#44867](https://github.com/ray-project/ray/pull/44867), [#44845](https://github.com/ray-project/ray/pull/44845)

Ray Core and Ray Clusters<a id="ray-core"></a>

💫 Enhancements:
- Report GCS internal pubsub buffer metrics and cap message size (44749)

🔨 Fixes:
- Fix task submission never return when network partition happens (44692)
- Fix incorrect use of ssh port forward option. (44973)
- Make sure dashboard will exit if grpc server fails (44928)
- Make sure dashboard agent will exit if grpc server fails (44899)

Thanks can-anyscale, hongchaodeng, zcin, marwan116, khluu, bewestphal, scottjlee, andrewsykim, anyscalesam, MortalHappiness, justinvyu, JoshKarpel, woshiyyya, rynewang, Abirdcfly, omatthew98, sven1977, marcelocarmona, rueian, mattip, angelinalg, aslonnie, matthewdeng, abizjakpro, simonsays1980, jjyao, terraflops1048576, hongpeng-guo, stephanie-wang, bw-matthew, bveeramani, ruisearch42, kevin85421, Tongruizhe

Many thanks to all those who contributed to this release!

Ray Libraries<a id="ray-libraries"></a>

Ray Data<a id="ray-data"></a>

🎉 New Features:

- Store Ray Data logs in special subdirectory (44743)

💫 Enhancements:
- Add in `local_read` option to `from_torch` (44752)

🔨 Fixes:
- Fix the config to disable progress bar (44342)

📖 Documentation:
- Clarify deprecated Datasource docstrings (44790)

Ray Train<a id="ray-train"></a>

🔨 Fixes:
- Disable gathering the full state dict in `RayFSDPStrategy` for `lightning>2.1` (44569)

Ray Tune<a id="ray-tune"></a>

💫 Enhancements:

- Remove spammy log for "new output engine" (44824)
- Enable isort (44693)

Ray Serve<a id="ray-serve"></a>

🔨 Fixes:
- [Serve] fix getting attributes on stdout during Serve logging redirect ([44787](https://github.com/ray-project/ray/pull/44787))

RLlib<a id="rllib"></a>

🎉 New Features:

- Support of images and video logging in WandB (env rendering example script for the new API stack coming up). ([43356](https://github.com/ray-project/ray/pull/43356))

💫 Enhancements:

- Better support and separation-of-concerns for `model_config_dict` in new API stack. ([44263](https://github.com/ray-project/ray/pull/44263))
- Added example script to pre-train an `RLModule` in single-agent fashion, then bring checkpoint into multi-agent setup and continue training. ([44674](https://github.com/ray-project/ray/pull/44674))
- More `examples` scripts got translated from the old- to the new API stack: Curriculum learning, custom-gym-env, etc..: ([44706](https://github.com/ray-project/ray/pull/44706), [#44707](https://github.com/ray-project/ray/pull/44707), [#44735](https://github.com/ray-project/ray/pull/44735), [#44841](https://github.com/ray-project/ray/pull/44841))

Ray Core and Ray Clusters<a id="ray-core"></a>

🔨 Fixes:
- Fix GetAllJobInfo `is_running_tasks` is not returning the correct value when driver starts ray (44459)

Thanks

Many thanks to all those who contributed to this release!
can-anyscale, hongpeng-guo, sven1977, zcin, shrekris-anyscale, liuxsh9, jackhumphries, GeneDer, woshiyyya, simonsays1980, omatthew98, andrewsykim, n30111, architkulkarni, bveeramani, aslonnie, alexeykudinkin, WeichenXu123, rynewang, matthewdeng, angelinalg, c21