Unidist

This release makes MPI the default backend, includes new features and
important documentation updates for it, as well as contains a bunch of bugfixes.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-https://github.com/modin-project/unidist/issues/275: Pass python flags into workers (https://github.com/modin-project/unidist/pull/276)
* FIX-https://github.com/modin-project/unidist/issues/281: Do not shutdown MPI backend multiple times (https://github.com/modin-project/unidist/pull/282)
* FIX-https://github.com/modin-project/unidist/issues/283: Exit monitor and worker processes in SPMD mode (https://github.com/modin-project/unidist/pull/284)
* FIX-https://github.com/modin-project/unidist/issues/294: Fix MPI backend launch for Open MPI in a cluster (https://github.com/modin-project/unidist/pull/295)
* FIX-https://github.com/modin-project/unidist/issues/242: Fix behavior of unidist.wait for MPI backend (https://github.com/modin-project/unidist/pull/291)
* FIX-000: Add licence header to request_store.py (https://github.com/modin-project/unidist/pull/307)
* FIX-https://github.com/modin-project/unidist/issues/296: Make workers exit the loop correctly (https://github.com/modin-project/unidist/pull/312)
* FIX-https://github.com/modin-project/unidist/issues/318: Do not initialize MPI in serialization.py (https://github.com/modin-project/unidist/pull/319)
* FIX-https://github.com/modin-project/unidist/issues/320: Improve CI (https://github.com/modin-project/unidist/pull/321)
* FIX-https://github.com/modin-project/unidist/issues/322: Disable actor tests for Ray (https://github.com/modin-project/unidist/pull/323)
* FIX-https://github.com/modin-project/unidist/issues/324: Pin pydantic<2 to fix CI (https://github.com/modin-project/unidist/pull/325)
* FIX-https://github.com/modin-project/unidist/issues/326: Set I_MPI_SPAWN for Intel MPI when using dynamic spawn (https://github.com/modin-project/unidist/pull/327)
* Performance enhancements
* PERF-https://github.com/modin-project/unidist/issues/287: Optimize python list appends (https://github.com/modin-project/unidist/pull/290)
* New Features
* FEAT-https://github.com/modin-project/unidist/issues/288: Make MPI as default backend (https://github.com/modin-project/unidist/pull/293)
* FEAT-https://github.com/modin-project/unidist/issues/305: Introduce MpiBackoff configuration value (https://github.com/modin-project/unidist/pull/306)
* FEAT-https://github.com/modin-project/unidist/issues/316: Introduce MpiLog configuration value (https://github.com/modin-project/unidist/pull/317)
* Refactor Codebase
* REFACTOR-000: Refactor request store to make it clearer (https://github.com/modin-project/unidist/pull/311)
* Documentation improvements
* DOCS-000: Clarify the phrases regarding pyseq backend (https://github.com/modin-project/unidist/pull/272)
* DOCS-https://github.com/modin-project/unidist/issues/277: Add notes regarding installation of MPI implementations (https://github.com/modin-project/unidist/pull/278)
* DOCS-https://github.com/modin-project/unidist/issues/279: Advise original MPI binaries rather than conda-forge packages (https://github.com/modin-project/unidist/pull/280)
* DOCS-https://github.com/modin-project/unidist/issues/299: Extend troubleshooting page as to MPI errors (https://github.com/modin-project/unidist/pull/300)
* DOCS-https://github.com/modin-project/unidist/issues/301: Add Optimization Notes section to the docs (https://github.com/modin-project/unidist/pull/302)
* DOCS-https://github.com/modin-project/unidist/issues/303: Fix a typo in unidist_on_mpi.rst (https://github.com/modin-project/unidist/pull/304)
* DOCS-https://github.com/modin-project/unidist/issues/314: Add troubleshooting block for 'OpenSSL version mismatch' error (https://github.com/modin-project/unidist/pull/315)

Contributors
------------
YarShev
Retribution98
arunjose696
luweizheng

This release contains a few performance enhancements for MPI backend,
the renaming of backends "multiprocessing" to "pymp" and "python" to "pyseq",
includes a couple of important updates for documentation and multiple bugfixes.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-170: Fix serialization/deserialization for Ray actors (171)
* FIX-172: Fix serialization/deserialization for Dask actors (173)
* FIX-174: Fix serialization/deserialization for MPI actors (177)
* FIX-179: Handle None return values correctly on MPI backend (180)
* FIX-175; FIX-178: Fix nested remote calls of remote tasks and actor methods (176)
* FIX-187: Make Data_ID unique for different workers on MPI backend (188)
* FIX-183: Fix serialization of dict_keys for MPI backend (184)
* FIX-189: Fix Actor scheduling (190)
* FIX-191: Fix pendinq response after blocking GET request (192)
* FIX-193: Fix actor async execution (194)
* FIX-197: Initialize a backend when doing unidist.put (198)
* FIX-201: Fix warnings to be aligned with latest flake8 (202)
* FIX-211: Fix dask[complete] syntax in conda environment file (212)
* FIX-213: Fix NamedTuple serialization (215)
* FIX-209: Fix set up of conda in CI to make it work properly (210)
* FIX-216: Fix development environment for ray==2.1.0 (217)
* FIX-218: Install Ray from PyPI to make CI work (220)
* FIX-224: Fix recursion in processing remote task arguments and serialization of dictionary that has a tuple key (225)
* FIX-228: Add missing configs to workers initialization (229)
* FIX-233: Fix passing MpiHosts config to workers initialization (234)
* FIX-232: Fix sending out-of-band data over 1 GB (231)
* FIX-235: Adding handler only if logger does not have any handlers (236)
* FIX-246: Slight changes to file formatting to match new version of black (247)
* FIX-253: Append worker rank to logger name (254)
* FIX-256: Fix sending a cleanup command from a worker to itself (257)
* FIX-263: Update the year in the licence header (264)
* FIX-259: Fix cleanup data in the workers (260)
* FIX-269: Update min supported Ray version (270)
* Performance enhancements
* PERF-226: Do not use pickle5 for small object serialization (227)
* PERF-243: Using non-blocking send (244)
* PERF-195: Fix serialization for numpy objects (200)
* PERF-195: Fix serialization for pandas and numpy objects (196)
* New Features
* FEAT-185: Add communication logging (186)
* FEAT-204: Allow PERF prefix in the commit messages (205)
* FIX-248: Rename backends "multiprocessing" to "pymp" and "python" to "pyseq" (265)
* Refactor Codebase
* REFACTOR-237: Replace len_buffers with buffer_count (238)
* Documentation improvements
* DOCS-207: Add a doc section regarding installation of Intel MPI (208)
* DOCS-266: Fix documentation build (267)
* DOCS-221: Add mentioning about Modin and unidist integration in to docs (222)
* Update testing suite
* TEST-181: Added async actor tests (182)
* TEST-89: Set up testing for unidist on push (206)
* FIX-250: Require format for PR descriptions instead of commit descriptions (251)
* FIX-250: Replace lint-commit to check-pr-title in ci.yml (252)

Contributors
------------
YarShev
Retribution98
arunjose696

This release contains a few important bugfixes for MPI backend.

Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-201: Fix warnings to be aligned with latest flake8 (202)
* FIX-195: Fix serialization for numpy objects (200)
* FIX-211: Fix dask[complete] syntax in conda environment file (212)
* FIX-213: Fix NamedTuple serialization (215)
* FIX-224: Fix recursion in processing remote task arguments and serialization of dictionary that has a tuple key (225)
* FIX-228: Add missing configs to workers initialization (229)

Contributors
------------
YarShev
Retribution98

This release contains multiple important bugfixes for MPI backend,
which makes it be more functional and stable.

Key Features and Updates
------------------------

* Stability and Bugfixes
* FIX-170: Fix serialization/deserialization for Ray actors (171)
* FIX-172: Fix serialization/deserialization for Dask actors (173)
* FIX-174: Fix serialization/deserialization for MPI actors (177)
* FIX-179: Handle None return values correctly on MPI backend (180)
* FIX-175: FIX-178: Fix nested remote calls of remote tasks and actor methods (176)
* FIX-187: Make Data_ID unique for different workers on MPI backend (188)
* FIX-183: Fix serialization of dict_keys for MPI backend (184)
* FEAT-185: Add communication logging (186)
* FIX-189: Fix Actor scheduling (190)
* FIX-191: Fix pendinq response after blocking GET request (192)
* FIX-195: Fix serialization for pandas and numpy objects (196)
* FIX-193: Fix actor async execution (194)
* FIX-197: Initialize a backend when doing unidist.put (198)
* Update testing suite
* TEST-181: Added async actor tests (182)

Contributors
------------
YarShev
Retribution98

This release adds support for the dynamic spawning of MPI worker
processes, contains significant refactoring of MPI backend, cut off
unidist CLI, includes important updates for documentation and a bunch of
bugfixes.

Key Features and Updates
------------------------

* Stability and Bugfixes
* FIX-https://github.com/modin-project/unidist/issues/130: Make unidist be pure Python (https://github.com/modin-project/unidist/pull/131)
* FIX-https://github.com/modin-project/unidist/issues/133: Add support of running mpi backend via CLI on Windows (https://github.com/modin-project/unidist/pull/134)
* FIX-https://github.com/modin-project/unidist/issues/69: Pin min msgpack version (https://github.com/modin-project/unidist/pull/135)
* FIX-https://github.com/modin-project/unidist/issues/124: Set max_retries=0 for Ray backend (https://github.com/modin-project/unidist/pull/136)
* FIX-https://github.com/modin-project/unidist/issues/142: Fix flake8 CI fail (https://github.com/modin-project/unidist/pull/143)
* FIX-https://github.com/modin-project/unidist/issues/145: Synchronize Ray with latest changes (https://github.com/modin-project/unidist/pull/146)
* FIX-https://github.com/modin-project/unidist/issues/137: Set threads_per_worker=1 for local Dask cluster (https://github.com/modin-project/unidist/pull/151)
* FIX-https://github.com/modin-project/unidist/issues/152: Clean up MPI backend (https://github.com/modin-project/unidist/pull/153)
* FIX-https://github.com/modin-project/unidist/issues/161: Propagate Backend.put('mpi') to MPI workers (https://github.com/modin-project/unidist/pull/162)
* FIX-https://github.com/modin-project/unidist/issues/163: Fix unwrapping object refs and data IDs (https://github.com/modin-project/unidist/pull/164)
* FIX-https://github.com/modin-project/unidist/issues/167: Set lowest bound for Ray versions (https://github.com/modin-project/unidist/pull/168)
* New Features
* FEAT-https://github.com/modin-project/unidist/issues/3: Add cluster_resources in API (https://github.com/modin-project/unidist/pull/35)
* FEAT-https://github.com/modin-project/unidist/issues/139; REFACTOR-https://github.com/modin-project/unidist/issues/100: Implement the spawning of MPI processes dynamically (https://github.com/modin-project/unidist/pull/144)
* FEAT-https://github.com/modin-project/unidist/issues/159: Add support for Async Actors on MPI backend (https://github.com/modin-project/unidist/pull/160)
* FEAT-https://github.com/modin-project/unidist/issues/165: Add is_initialized feature to unidist API (https://github.com/modin-project/unidist/pull/166)
* Refactor Codebase
* REFACTOR-https://github.com/modin-project/unidist/issues/149: Cut off unidist CLI (https://github.com/modin-project/unidist/pull/150)
* Documentation improvements
* DOCS-https://github.com/modin-project/unidist/issues/128: Update installation section in docs (https://github.com/modin-project/unidist/pull/132)
* DOCS-https://github.com/modin-project/unidist/issues/85: Add docs on running unidist with different backends (https://github.com/modin-project/unidist/pull/140)
* DOCS-https://github.com/modin-project/unidist/issues/147: Update docs to be in line with the code base (https://github.com/modin-project/unidist/pull/148)
* DOCS-https://github.com/modin-project/unidist/issues/154: Make documentation fail on warning (https://github.com/modin-project/unidist/pull/155)
* DOCS-https://github.com/modin-project/unidist/issues/157: Set fail_on_warning: false (https://github.com/modin-project/unidist/pull/158)

Contributors
------------
YarShev, prutskov, no-ponomarev

This is the first release of undist, which includes support for different execution backends such as Ray, MPI, Dask, Python Multiprocessing, as well as a sequential Python backend for debugging. This release contains significant changes and upgrades to codebase, unidist's documentation, and some bugfixes.

Key Features and Updates
------------------------
* Initial changes
* initial commit [e9711d488f575b3ce09d9ef449f5b5b431b8bd6f](https://github.com/modin-project/unidist/commit/e9711d488f575b3ce09d9ef449f5b5b431b8bd6f)
* Stability and bugfixes
* FIX-13: Add busy wait mpi recv to remove unidist.get contention problem
* TEST-20: Add linters CI (19)
* FEAT-4: Add configs for unidist behavior tuning (21)
* FEAT-38: Add configs for Dask cluster setup (39)
* FIX-45: Add What do these changes do section in PRs description (46)
* FIX-29: Throw a user-friendly error message in case pickle5 is not installed (31)
* FIX-25: Throw a user-friendly error message in case mpi4py is not installed (32)
* FEAT-47: Add BackendName namespace class to use instead of strings (48)
* FIX-49: Add check on entry in Parameter.choices (50)
* FIX-51: Use lower-case for str TypeDescriptor (52)
* FEAT-2: Add CLI interface for unidist (14)
* FIX-58: Add exit status when run from CLI (59)
* FEAT-62: Add ValueSource enum to check source of value in `Parameter` (63)
* FIX-60: Align default value of REDIS_PASSWORD with Ray (61)
* TEST-24: Add tests run to CI (56)
* FIX-76: Rename UNIDIST_OBJECT_STORE_MEMORY to (77)
* FIX-71: Fix for detecting compatible types for out-of-band serialization (74)
* FEAT-96: Add python-versioneer to get exact unidist version (105)
* FIX-109: Fix test_get_ip for Ray backend (110)
* FIX-104: Add testing py3.9 in CI (112)
* FIX-118: Choose dependencies for pip install unidist[all] based on platform (119)
* FIX-79: Update copyright 2021 to 2021-2022 (80)
* Performance enhancements
* FIX-36: Use data serialization caches to improve performance for MPI backend (53)
* Refactor Codebase
* FIX-16: Apply black to all project (17)
* FIX-97: Remove readme file for MPI backend (98)
* Dependencies
* FIX-22: Add dependencies for development and packaging (23)
* DOCS-7: Add deps for building docs (28)
* FIX-33: Add black and flake8 to env files (34)
* FIX-40: Get rid of pandas dependency for Ray backend (42)
* FIX-43: Add mpi4py to pip distribution dependencies (44)
* FIX-41: Remove Pandas package reference (57)
* FIX-67: Replace mpi4py to mpi4py-mpich in pip dependencies (68)
* FIX-99: Upgrade min python version (103)
* FIX-121: Downgrade min supported python version to 3.7.1 (123)
* Documentation improvements
* DOCS-5: Add a contributing page for unidist (82)
* DOCS-73: Add documentation page on CLI usage (83)
* DOCS-37: Add rst documentation for unidist.config (78)
* DOCS-6: Update and extend readme.md (87)
* FIX-91: Add readthedocs config file (92)
* DOCS-81: Update High-Level Execution View section (90)
* FIX-94: Change docs copyright statement to Modin Authors (95)
* DOCS-88: Update main/getting_started pages of rst docs (93)
* FEAT-54 Add unidist main logo image (108)
* FIX-111: Fix build warning for communication module (113)
* FIX-114: Change readthedocs icon format (115)
* FIX-116: Add PyPI badge on README.md (117)
* FIX-120: Change unidist logo path to absolute (122)

Contributors
------------
YarShev, prutskov, no-ponomarev