Distributed

Changes

- Don't stop Adaptive on error hendrikmakait (8871)
- Update gpuCI `RAPIDS_VER` to `24.12` github-actions (8879)
- Don't consider scheduler idle while executing `Scheduler.update_graph` hendrikmakait (8877)
- Bump jacobtomlinson/gha-anaconda-package-version from 0.1.3 to 0.1.4 dependabot (8878)
- Support P2P rechunking datetime arrays jrbourbeau (8875)

See the [Changelog](https://docs.dask.org/en/stable/changelog.html) for more information.

Changes

- Homogeneously schedule P2P's unpack tasks hendrikmakait (8873)
- Work/fix firewall for localhost maldag (8868)
- Bump ``bokeh`` minimum version to 3.1.0 jrbourbeau (8861)
- Use new ``tokenize`` module jrbourbeau (8858)
- Point to user code with ``idempotent`` plugin warning jrbourbeau (8856)
- Fix test nanny timeout fjetter (8847)
- Bump JamesIves/github-pages-deploy-action from 4.5.0 to 4.6.4 dependabot (8853)
- Speed up ``Client.map`` by computing ``token`` only once for ``func`` and ``kwargs`` fjetter (8855)
- Update precommit fjetter (8852)

See the [Changelog](https://docs.dask.org/en/stable/changelog.html) for more information.

Changes

- Avoid capturing code of xdist fjetter (8846)
- Reduce memory footprint of culling P2P rechunking hendrikmakait (8845)
- Add tests for choosing default rechunking method hendrikmakait (8843)
- Increase visibility of GPU CI updates charlesbluca (8841)
- Bump test\_pause\_while\_idle timeout fjetter (8844)
- Concatenate small input chunks before P2P rechunking hendrikmakait (8832)
- Remove dump cluster from gen\_cluster fjetter (8823)
- Bump `numpy>=1.24` and `pyarrow>=14.0.1` minimum versions jrbourbeau (8837)
- Fix PipInstall plugin on Worker hendrikmakait (8839)
- Remove more Python 3.10 compatibility code jrbourbeau (8824)
- Use task-based rechunking to prechunk along partial boundaries hendrikmakait (8831)
- Ensure client\_desires\_keys does not corrupt Scheduler state fjetter (8827)
- Bump minimum ``cloudpickle`` to 3 jrbourbeau (8836)

See the [Changelog](https://docs.dask.org/en/stable/changelog.html) for more information.

Changes

- Reduce frequency of unmanaged memory use warning phofl (8834)
- Update gpuCI `RAPIDS_VER` to `24.10` github-actions (8786)
- Avoid `RuntimeError: dictionary changed size during iteration` in `Server._shift_counters()` hendrikmakait (8828)
- Improve concurrent close for scheduler hendrikmakait (8829)
- MINOR: Extract truncation logic out of partial concatenation in P2P rechunking hendrikmakait (8826)
- Avoid excessive attribute access overhead for remove\_from\_task\_prefix\_count fjetter (8821)
- Avoid key validation if validation is disabled fjetter (8822)
- Log ``worker\_client`` event jrbourbeau (8819)
- Drop support for Python 3.9 phofl (8793)

See the [Changelog](https://docs.dask.org/en/stable/changelog.html) for more information.

Changes

- Run graph normalisation after dask order phofl (8818)
- Update large graph size warning to remove scatter recommendation phofl (8815)
- Fail tasks exceeding `no-workers-timeout` hendrikmakait (8806)
- Fix exception handling for ``NannyPlugin.setup`` and ``NannyPlugin.teardown`` hendrikmakait (8811)
- Fix exception handling for ``WorkerPlugin.setup`` and ``WorkerPlugin.teardown`` hendrikmakait (8810)
- Fix typo in transitions event logging alex-rakowski (8812)
- Fix if-else for send\_recv\_from\_rpc phofl (8809)
- Ensure that adaptive only stops once hendrikmakait (8807)
- Reduce noise from GC-related logging hendrikmakait (8804)
- Remove unused `delete_interval` and `synchronize_worker_interval` from `Scheduler` hendrikmakait (8801)
- Change log level for Compute Failed log message phofl (8802)
- Add Prometheus metric for time spent on GC hendrikmakait (8803)
- Add Prometheus metrics for `dask_scheduler_workers_{added|removed}_total` hendrikmakait (8798)
- Add log event for `worker-ttl-timeout` hendrikmakait (8800)
- Add Prometheus metrics for `dask_scheduler_client_connections_{added|removed}_total` hendrikmakait (8799)
- Fix restarting workers with `PackageInstall` plugin hendrikmakait (8794)
- Make stealing more robust hendrikmakait (8788)
- leave a warning about future instantiation fjetter (8782)

See the [Changelog](https://docs.dask.org/en/stable/changelog.html) for more information.

Changes

- Ensure Locks always register with scheduler fjetter (8781)
- Temporarily pin ``setuptools \< 71`` jrbourbeau (8785)
- Restore len() on TaskPrefix hendrikmakait (8783)
- Avoid false positives for `p2p-failed` log event hendrikmakait (8777)
- Expose paused and retired workers separately in prometheus phofl (8613)
- Creating transitions-failures log event alex-rakowski (8776)
- Implement HLG layer for P2P rechunking hendrikmakait (8751)
- Ensure Lock will not lock up in case of worker failures fjetter (8770)
- Add another test for a possible deadlock scenario caused by 8703 hendrikmakait (8769)
- Raise an error if compute on persisted collection with released futures fjetter (8764)
- Re-raise `P2PConsistencyError` from failed P2P tasks. hendrikmakait (8748)
- Robuster faster tests memory sampler fjetter (8758)
- fix scheduler\_bokeh::test\_shuffling fjetter (8766)
- increase timeouts for pubsub::test\_client\_worker fjetter (8765)
- Factor out async taskgroup fjetter (8756)
- Don't sort keys lexicographically in worker table fjetter (8753)
- Use functools.cache instead of functools.lru\_cache for url\_escape \& \_expects\_comm jonded94 (8762)
- Robuster deeply nested structures fjetter (8730)
- Adding HLG to MAP alex-rakowski (8740)
- Add close worker button to worker info page jrbourbeau (8742)

See the [Changelog](https://docs.dask.org/en/stable/changelog.html) for more information.