Unidist

This release contains a few important improvements for MPI backend.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-402: Fix MPICH version check (403)
* FIX-404: Add psutil as a required dependency for unidist[mpi] (406)
* FIX-407: Make data put into MPI object store immutable (409)
* Performance enhancements
* PERF-360: Re - Implement asyncronous get request with a fix (405)
* Refactor Codebase
* REFACTOR-000: Refactor remote function class for MPI backend (410)

Contributors
------------
YarShev
Retribution98
arunjose696

This release contains an important bugfix for MPI backend and
adds psutil as a required dependency for unidist[mpi] package.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-https://github.com/modin-project/unidist/issues/402: Fix MPICH version check (https://github.com/modin-project/unidist/pull/403)
* FIX-https://github.com/modin-project/unidist/issues/404: Add psutil as a required dependency for unidist[mpi] (https://github.com/modin-project/unidist/pull/406)

Contributors
------------
YarShev
Retribution98

This release adds support for a shared object store for MPI backend,
which can be enabled with UNIDIST_MPI_SHARED_OBJECT_STORE environment variable,
contains a lot of performance enhancements, includes several features,
important documentation updates and a bunch of bugfixes.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-329: Allow remaining communication requests to complete (330)
* FIX-343: Get rid of the hacks in the MPI backend (344)
* FIX-345: Test both modes of MPI execution on Linux only (346)
* FIX-349: Remove the warning regarding MpiPickleThreshold (350)
* FIX-354: Fix unidist.wait for MPI backend (355)
* FIX-363: Fix DataID usage between different MPI processes (364)
* FIX-371: Fix check for out of bounds reserved memory in shared storage (372)
* FIX-375: Fix running an MPI cluster without using shared memory (377)
* FIX-340: Fix hangs at low UNIDIST_MPI_SHARED_OBJECT_STORE_THRESHOLD (380)
* FIX-376: Partition buffers of size larger than the _bigmpi.blocksize into blocks while sending and receiving with MPI (383)
* FIX-337: Fix MpiHosts parameter usage (378)
* FIX-000: Revert PR 387 (388)
* FEAT-392: Raise a warning if shared object storage is enabled but the MPI library does not support it (393)
* FIX-389: Revert async get changes made in 359 (390)
* FIX-394: Fix CI running on Windows (395)
* FIX-396: Fix Split_type call for Unidist on MSMPI (397)
* Performance enhancements
* PERF-365: Cache info about sent metadata of shared data (366)
* PERF-360: Implement asyncronous get request (359)
* PERF-367: Use std::fill to fill service shared buffer with a given value (373)
* New Features
* FEAT-285: Implement shared object storage for MPI backend (286)
* FEAT-341: Add the ability to specify environment variables for MPI workers (352)
* FEAT-347: Get rid of mpi4py-mpich dependency (358)
* FEAT-386: Make MPI shared object store default option (387)
* Refactor Codebase
* REFACTOR-000: Remove unnecessary build step in CI (339)
* REFACTOR-000: Introduce task_info metadata package (348)
* REFACTOR-000: Remove unncessary else branch in mpi init (351)
* REFACTOR-000: Change mpi_state.comm to mpi_state.global_comm (379)
* REFACTOR-000: Change IsMpiSpawnWorkers to MpiSpawn to make it more concise (382)
* Documentation improvements
* DOCS-000: Update the workflow status badge (332)
* DOCS-000: Add the downloads badge in readme (333)
* DOCS-384: Fix warnings when building documentation (385)
* DOCS-336: Add documentation about shared object storage (391)

Contributors
------------
YarShev
Retribution98
arunjose696

This release contains an important bugfix for MPI backend.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-https://github.com/modin-project/unidist/issues/329: Allow remaining communication requests to complete (https://github.com/modin-project/unidist/pull/330)

Contributors
------------
YarShev

This release makes MPI the default backend, includes new features and
important documentation updates for it, as well as contains a bunch of bugfixes.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-https://github.com/modin-project/unidist/issues/275: Pass python flags into workers (https://github.com/modin-project/unidist/pull/276)
* FIX-https://github.com/modin-project/unidist/issues/281: Do not shutdown MPI backend multiple times (https://github.com/modin-project/unidist/pull/282)
* FIX-https://github.com/modin-project/unidist/issues/283: Exit monitor and worker processes in SPMD mode (https://github.com/modin-project/unidist/pull/284)
* FIX-https://github.com/modin-project/unidist/issues/294: Fix MPI backend launch for Open MPI in a cluster (https://github.com/modin-project/unidist/pull/295)
* FIX-https://github.com/modin-project/unidist/issues/242: Fix behavior of unidist.wait for MPI backend (https://github.com/modin-project/unidist/pull/291)
* FIX-000: Add licence header to request_store.py (https://github.com/modin-project/unidist/pull/307)
* FIX-https://github.com/modin-project/unidist/issues/296: Make workers exit the loop correctly (https://github.com/modin-project/unidist/pull/312)
* FIX-https://github.com/modin-project/unidist/issues/318: Do not initialize MPI in serialization.py (https://github.com/modin-project/unidist/pull/319)
* FIX-https://github.com/modin-project/unidist/issues/320: Improve CI (https://github.com/modin-project/unidist/pull/321)
* FIX-https://github.com/modin-project/unidist/issues/322: Disable actor tests for Ray (https://github.com/modin-project/unidist/pull/323)
* FIX-https://github.com/modin-project/unidist/issues/324: Pin pydantic<2 to fix CI (https://github.com/modin-project/unidist/pull/325)
* FIX-https://github.com/modin-project/unidist/issues/326: Set I_MPI_SPAWN for Intel MPI when using dynamic spawn (https://github.com/modin-project/unidist/pull/327)
* Performance enhancements
* PERF-https://github.com/modin-project/unidist/issues/287: Optimize python list appends (https://github.com/modin-project/unidist/pull/290)
* New Features
* FEAT-https://github.com/modin-project/unidist/issues/288: Make MPI as default backend (https://github.com/modin-project/unidist/pull/293)
* FEAT-https://github.com/modin-project/unidist/issues/305: Introduce MpiBackoff configuration value (https://github.com/modin-project/unidist/pull/306)
* FEAT-https://github.com/modin-project/unidist/issues/316: Introduce MpiLog configuration value (https://github.com/modin-project/unidist/pull/317)
* Refactor Codebase
* REFACTOR-000: Refactor request store to make it clearer (https://github.com/modin-project/unidist/pull/311)
* Documentation improvements
* DOCS-000: Clarify the phrases regarding pyseq backend (https://github.com/modin-project/unidist/pull/272)
* DOCS-https://github.com/modin-project/unidist/issues/277: Add notes regarding installation of MPI implementations (https://github.com/modin-project/unidist/pull/278)
* DOCS-https://github.com/modin-project/unidist/issues/279: Advise original MPI binaries rather than conda-forge packages (https://github.com/modin-project/unidist/pull/280)
* DOCS-https://github.com/modin-project/unidist/issues/299: Extend troubleshooting page as to MPI errors (https://github.com/modin-project/unidist/pull/300)
* DOCS-https://github.com/modin-project/unidist/issues/301: Add Optimization Notes section to the docs (https://github.com/modin-project/unidist/pull/302)
* DOCS-https://github.com/modin-project/unidist/issues/303: Fix a typo in unidist_on_mpi.rst (https://github.com/modin-project/unidist/pull/304)
* DOCS-https://github.com/modin-project/unidist/issues/314: Add troubleshooting block for 'OpenSSL version mismatch' error (https://github.com/modin-project/unidist/pull/315)

Contributors
------------
YarShev
Retribution98
arunjose696
luweizheng

This release contains a few performance enhancements for MPI backend,
the renaming of backends "multiprocessing" to "pymp" and "python" to "pyseq",
includes a couple of important updates for documentation and multiple bugfixes.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-170: Fix serialization/deserialization for Ray actors (171)
* FIX-172: Fix serialization/deserialization for Dask actors (173)
* FIX-174: Fix serialization/deserialization for MPI actors (177)
* FIX-179: Handle None return values correctly on MPI backend (180)
* FIX-175; FIX-178: Fix nested remote calls of remote tasks and actor methods (176)
* FIX-187: Make Data_ID unique for different workers on MPI backend (188)
* FIX-183: Fix serialization of dict_keys for MPI backend (184)
* FIX-189: Fix Actor scheduling (190)
* FIX-191: Fix pendinq response after blocking GET request (192)
* FIX-193: Fix actor async execution (194)
* FIX-197: Initialize a backend when doing unidist.put (198)
* FIX-201: Fix warnings to be aligned with latest flake8 (202)
* FIX-211: Fix dask[complete] syntax in conda environment file (212)
* FIX-213: Fix NamedTuple serialization (215)
* FIX-209: Fix set up of conda in CI to make it work properly (210)
* FIX-216: Fix development environment for ray==2.1.0 (217)
* FIX-218: Install Ray from PyPI to make CI work (220)
* FIX-224: Fix recursion in processing remote task arguments and serialization of dictionary that has a tuple key (225)
* FIX-228: Add missing configs to workers initialization (229)
* FIX-233: Fix passing MpiHosts config to workers initialization (234)
* FIX-232: Fix sending out-of-band data over 1 GB (231)
* FIX-235: Adding handler only if logger does not have any handlers (236)
* FIX-246: Slight changes to file formatting to match new version of black (247)
* FIX-253: Append worker rank to logger name (254)
* FIX-256: Fix sending a cleanup command from a worker to itself (257)
* FIX-263: Update the year in the licence header (264)
* FIX-259: Fix cleanup data in the workers (260)
* FIX-269: Update min supported Ray version (270)
* Performance enhancements
* PERF-226: Do not use pickle5 for small object serialization (227)
* PERF-243: Using non-blocking send (244)
* PERF-195: Fix serialization for numpy objects (200)
* PERF-195: Fix serialization for pandas and numpy objects (196)
* New Features
* FEAT-185: Add communication logging (186)
* FEAT-204: Allow PERF prefix in the commit messages (205)
* FIX-248: Rename backends "multiprocessing" to "pymp" and "python" to "pyseq" (265)
* Refactor Codebase
* REFACTOR-237: Replace len_buffers with buffer_count (238)
* Documentation improvements
* DOCS-207: Add a doc section regarding installation of Intel MPI (208)
* DOCS-266: Fix documentation build (267)
* DOCS-221: Add mentioning about Modin and unidist integration in to docs (222)
* Update testing suite
* TEST-181: Added async actor tests (182)
* TEST-89: Set up testing for unidist on push (206)
* FIX-250: Require format for PR descriptions instead of commit descriptions (251)
* FIX-250: Replace lint-commit to check-pr-title in ci.yml (252)

Contributors
------------
YarShev
Retribution98
arunjose696