Proxystore

Highlights

This release addresses some minor bugs and improves the documentation with update logos, colors, descriptions, and API formatting.

Upgrade Steps
bash
pip install --upgrade proxystore or proxystore[all]


What's Changed

Enhancements
* Enhance compatibility of the `StoreExecutor` with non-`concurrent.futures.Executor` types by gpauloski in https://github.com/proxystore/proxystore/pull/612

Bug Fixes
* Exclude state by default in `LocalConnector` config by gpauloski in https://github.com/proxystore/proxystore/pull/602
* Improve thread-safety in the `Store` by gpauloski in https://github.com/proxystore/proxystore/pull/609
* Fix deadlock in `StoreExecutor` test with MacOS runners by gpauloski in https://github.com/proxystore/proxystore/pull/614
* Improve which relay registration errors are raised versus retried by gpauloski in https://github.com/proxystore/proxystore/pull/621

Internal Changes
* Return `self` in `start()`/`stop()` of `Timer` by gpauloski in https://github.com/proxystore/proxystore/pull/611

Documentation
* Add TaPS to featured papers by gpauloski in https://github.com/proxystore/proxystore/pull/606
* Fix docstring link for websockets>=13.0 by gpauloski in https://github.com/proxystore/proxystore/pull/615
* Improve API documentation templates by gpauloski in https://github.com/proxystore/proxystore/pull/616
* Update logo, icon, and color palette in docs by gpauloski in https://github.com/proxystore/proxystore/pull/617
* Improve ProxyStore descriptions in docs, project metadata, and README by gpauloski in https://github.com/proxystore/proxystore/pull/623
* Update README intro and improve docs build process by gpauloski in https://github.com/proxystore/proxystore/pull/624

Package Changes
* Update documentation dependency versions by gpauloski in https://github.com/proxystore/proxystore/pull/610

Development Workflows
* Upgrade all MacOS runners to 14 (ARM) by gpauloski in https://github.com/proxystore/proxystore/pull/613

**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.7.0...v0.7.1

<!-- Release notes generated using configuration in .github/release.yml at v0.7.0 -->

Highlights

This releases introduces a variety of quality of life improvements for ProxyStore.

* The type and hash of proxied objects are cached in the proxy by default to avoid incidental proxy resolution for common object operations (e.g., `isinstance()` or insert into hash-map type). See the `populate_target` flag of the [`Store`](https://docs.proxystore.dev/0.7.0/api/store/base/#proxystore.store.base.Store.proxy) to learn more.
* Added the `StoreExecutor`, which wraps `concurrent.futures.Executor` instances with automatic input/output proxying.
* Improved serialization performance for common scientific datatypes (dataframes and numpy arrays).
* Improved the mypy plugin.
* Added the [Proxy Patterns](https://arxiv.org/abs/2407.01764) paper to the publications.

Upgrade Steps

bash
pip install --upgrade proxystore or proxystore[all]


What's Changed

Breaking Changes
* Migrate from `kafka-python` to `confluent_kafka` by gpauloski in https://github.com/proxystore/proxystore/pull/574
* Populate target objects by default in `Store` by gpauloski in https://github.com/proxystore/proxystore/pull/581

Enhancements
* Support `Union` types in the mypy plugin and create the `ProxyOr[T]` type alias by gpauloski in https://github.com/proxystore/proxystore/pull/575
* Add the `StoreExecutor` that wraps `concurrent.futures.Executor` instances with automatic input/output proxying by gpauloski in https://github.com/proxystore/proxystore/pull/578
* Improve mypy plugin error messages by gpauloski in https://github.com/proxystore/proxystore/pull/588
* Make `StoreConfig` a Pydantic `BaseModel` by gpauloski in https://github.com/proxystore/proxystore/pull/589
* Improve serialization performance with buffers and custom type support by KlaudiuszRydzy in https://github.com/proxystore/proxystore/pull/595

Bug Fixes
* Fix mypy plugin crashes by defaulting types to `Any` when internal assertions fail by gpauloski in https://github.com/proxystore/proxystore/pull/567
* Fix error messages printing "type" rather than the actual object's type name by gpauloski in https://github.com/proxystore/proxystore/pull/568
* Restore defaults in ref proxy trampoline by gpauloski in https://github.com/proxystore/proxystore/pull/580

Internal Changes
* Fix and/or remove `DeprecationWarning` and `ExperimentalWarning` by gpauloski in https://github.com/proxystore/proxystore/pull/576
* Refactor serialization logic by KlaudiuszRydzy in https://github.com/proxystore/proxystore/pull/594

Documentation
* Fix typo in FAQ by gpauloski in https://github.com/proxystore/proxystore/pull/571
* Add FAQ on `isinstance()` behavior with generic alias types by gpauloski in https://github.com/proxystore/proxystore/pull/585
* Clarify `populate_target=True` behavior with custom (de)serializers in `Store.proxy()` by gpauloski in https://github.com/proxystore/proxystore/pull/586
* Add Proxy Patterns preprint to publications by gpauloski in https://github.com/proxystore/proxystore/pull/592

Package Changes
* Remove `CITATION.cff` by gpauloski in https://github.com/proxystore/proxystore/pull/566
* Bump to ProxyStore v0.7.0 by gpauloski in https://github.com/proxystore/proxystore/pull/597

New Contributors
* KlaudiuszRydzy made their first contribution in https://github.com/proxystore/proxystore/pull/594

**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.5...v0.7.0

<!-- Release notes generated using configuration in .github/release.yml at v0.6.5 -->

Highlights

This primary goal of this release is to improve some of the subtle challenges of working with `Proxy` objects. Specifically, that (1) serialization can fail on `Proxy` class attribute access, (2) serializers/compute frameworks will accidentally resolve a `Proxy` instance when doing simple operations like checking types or hashing, and (3) mypy cannot infer the types of most operations on a `Proxy[T]`.

* ProxyStore now uses a new slots-based `Proxy` implementation which will improve serialization compatibility.
* `Store.proxy(..., populate_target=True)` will now cache some attributes of the target object so that calling `hash()` or `isinstance()` on a proxy does not require the proxy to be resolved.
* An FAQ guide has been created to discuss these common pitfalls and their remedies.
* The `proxystore.mypy_plugin`, when enabled, allows mypy to infer attribute types on `Proxy[T]`.
* The `Store` now has a `register` flag for configuring a `Store` instances to always register themselves.
* Added the `StaticLifetime`

Upgrade Steps

bash
pip install --upgrade proxystore or proxystore[all]


What's Changed

Breaking Changes
* Improve flexibility of `Lifetime` protocol and add `StaticLifetime` implementation by gpauloski in https://github.com/proxystore/proxystore/pull/542

Enhancements
* Introduce new slots-based `Proxy` implementation by gpauloski in https://github.com/proxystore/proxystore/pull/539
* Add the `proxystore.mypy` mypy plugin for resolving `Proxy[T]` types by gpauloski in https://github.com/proxystore/proxystore/pull/545
* Add the `register` flag to auto-register a `Store` after initialization by gpauloski in https://github.com/proxystore/proxystore/pull/550
* Support caching target attributes in `Proxy` to avoid resolving when using `hash()` or `isinstance()` by gpauloski in https://github.com/proxystore/proxystore/pull/551
* Generate `Store.__repr__()` from `Store.config()` by gpauloski in https://github.com/proxystore/proxystore/pull/554

Bug Fixes
* Fix incorrect ns to ms conversion in `Metrics` by gpauloski in https://github.com/proxystore/proxystore/pull/538

Documentation
* Add Harish et al featured paper by gpauloski in https://github.com/proxystore/proxystore/pull/537
* Add "latest" DOI badge by gpauloski in https://github.com/proxystore/proxystore/pull/540
* Add an FAQ page and refresh documentation by gpauloski in https://github.com/proxystore/proxystore/pull/546
* Add a Dask Distributed guide by gpauloski in https://github.com/proxystore/proxystore/pull/553

Package Changes
* Remove citation metadata to preserve Zenodo defaults by gpauloski in https://github.com/proxystore/proxystore/pull/541

**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.4...v0.6.5

<!-- Release notes generated using configuration in .github/release.yml at v0.6.4 -->

Highlights

* Automatic proxy memory management via the "Ownership" model and "Lifetime" managers. Learn more in the new [Object Lifetimes Guide](https://docs.proxystore.dev/0.6.4/guides/object-lifetimes/).
* Added the `populate_target` flag to `Store.proxy()` for "pre-resolving" proxies.
* Renamed `proxystore.store.future.ProxyFuture` to `proxystore.store.future.Future`.
* Added support for Pydantic V1.

Upgrade Steps
bash
pip install --upgrade proxystore or proxystore[all]


What's Changed

Breaking Changes
* Rename `ProxyFuture` to `Future` by gpauloski in https://github.com/proxystore/proxystore/pull/487

Enhancements
* Add `done()` method to `Future` by gpauloski in https://github.com/proxystore/proxystore/pull/488
* Add the `iter_with_metadata()` and `iter_objects_with_metadata()` methods to `StreamConsumer` by gpauloski in https://github.com/proxystore/proxystore/pull/489
* Implement proxy ownership and borrowing references, rules, and interfaces by gpauloski in https://github.com/proxystore/proxystore/pull/493
* Initial proxy ownership model integration for the `Store` and task execution by gpauloski in https://github.com/proxystore/proxystore/pull/494
* Support exponential backoff factor in `PollingStoreFactory` by gpauloski in https://github.com/proxystore/proxystore/pull/498
* Add `populate_target` flag to `Store.proxy()` to set created proxies as pre-resolved by gpauloski in https://github.com/proxystore/proxystore/pull/500
* Document and add utilities for taking ownership of returned proxies by gpauloski in https://github.com/proxystore/proxystore/pull/502
* Expose `publisher` and `subscriber` attributes in `StreamConsumer` and `StreamProducer` respectively by gpauloski in https://github.com/proxystore/proxystore/pull/504
* Leave stores open by default in stream interfaces by gpauloski in https://github.com/proxystore/proxystore/pull/505
* Add queue-based Redis shim for streaming by gpauloski in https://github.com/proxystore/proxystore/pull/511
* Add `Lifetime` support for managing shared objects by gpauloski in https://github.com/proxystore/proxystore/pull/513
* Improve `unregister_store` and `Store.future()` logging by gpauloski in https://github.com/proxystore/proxystore/pull/524
* Mark `Connector.close()` as idempotent by gpauloski in https://github.com/proxystore/proxystore/pull/526

Bug Fixes
* Invalidate cache entry when using `Store._set()` by gpauloski in https://github.com/proxystore/proxystore/pull/492
* Fix `Store.put_batch()` called with empty list in `StreamProducer.flush_topic()` by gpauloski in https://github.com/proxystore/proxystore/pull/503
* Copy target by default in `borrow()`, `mut_borrow()`, and `into_owned()` by gpauloski in https://github.com/proxystore/proxystore/pull/506
* Fix `submit()` resolving `RefProxy` and `RefMutProxy` arguments by gpauloski in https://github.com/proxystore/proxystore/pull/507
* Fix `submit()` resolving `Proxy` arguments (re: 507) by gpauloski in https://github.com/proxystore/proxystore/pull/508
* Handle and pass on SIGINT propagated by Uvicorn after cleanup by gpauloski in https://github.com/proxystore/proxystore/pull/514
* Prevent evicting the target object when invalidated `OwnedProxy` instances are garbage collected by gpauloski in https://github.com/proxystore/proxystore/pull/525
* Log warning when an `EndpointConnector` finds an unstarted endpoint by gpauloski in https://github.com/proxystore/proxystore/pull/529
* Improve serialization error handling when resolving a proxy by gpauloski in https://github.com/proxystore/proxystore/pull/531

Internal Changes
* Fix `LeaseLifetime` failing on MacOS runners by gpauloski in https://github.com/proxystore/proxystore/pull/523

Documentation
* Fix typo in Object Lifetimes guide by gpauloski in https://github.com/proxystore/proxystore/pull/528

Package Changes
* Update copyright year by gpauloski in https://github.com/proxystore/proxystore/pull/496
* Add backwards compatibility for Pydantic V1 by gpauloski in https://github.com/proxystore/proxystore/pull/501

Development Workflows
* Move ruff linter settings to lint section by gpauloski in https://github.com/proxystore/proxystore/pull/490
* Use Apple Silicon GitHub Actions Runners by gpauloski in https://github.com/proxystore/proxystore/pull/491
* Bump softprops/action-gh-release from 1 to 2 by dependabot in https://github.com/proxystore/proxystore/pull/509


**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.3...v0.6.4

<!-- Release notes generated using configuration in .github/release.yml at v0.6.3 -->

Highlights

This patch introduces the beta ProxyStore streaming interface. Check out the [guide](https://docs.proxystore.dev/main/guides/streaming/) for more info!

Upgrade Steps


pip install --upgrade proxystore or proxystore[all]


What's Changed

Enhancements
* Switch from dataclasses to Pydantic BaseModels for configuration classes by gpauloski in https://github.com/proxystore/proxystore/pull/446
* Add prototype streaming interface by gpauloski in https://github.com/proxystore/proxystore/pull/447
* Add the `RedisPublisher` and `RedisSubscriber` for use with ProxyStore streams by gpauloski in https://github.com/proxystore/proxystore/pull/451
* Add the `KafkaPublisher` and `KafkaSubscriber` for use with ProxyStore streams by gpauloski in https://github.com/proxystore/proxystore/pull/452
* Add the `ZeroMQPublisher` and `ZeroMQSubscriber` for use with ProxyStore streams by gpauloski in https://github.com/proxystore/proxystore/pull/453
* Make `StreamProducer`/`StreamConsumer` not generic by gpauloski in https://github.com/proxystore/proxystore/pull/466
* Consolidate streaming-related modules under `proxystore.stream` by gpauloski in https://github.com/proxystore/proxystore/pull/468
* Simplify `Producer`/`Subscriber` interfaces and adjust to match other libraries by gpauloski in https://github.com/proxystore/proxystore/pull/470
* Make `StreamProducer`/`StreamConsumer` generic type containers by gpauloski in https://github.com/proxystore/proxystore/pull/471
* Add end of stream events between the `StreamProducer` and `StreamConsumer` by gpauloski in https://github.com/proxystore/proxystore/pull/472
* `StreamProducer` controls which `Store` instances are used for each stream topic by gpauloski in https://github.com/proxystore/proxystore/pull/473
* Support aggregation, batching, and filtering in stream interfaces by gpauloski in https://github.com/proxystore/proxystore/pull/475
* Add the `next_object()` and `iter_object()` methods to `StreamConsumer` by gpauloski in https://github.com/proxystore/proxystore/pull/476

Internal Changes
* Remove redundant unit test docstring requirements by gpauloski in https://github.com/proxystore/proxystore/pull/469

Documentation
* Add new protein generation paper by Dharuman et al. by gpauloski in https://github.com/proxystore/proxystore/pull/438
* Update and pin `mkdocstrings` + related packages by gpauloski in https://github.com/proxystore/proxystore/pull/454
* Add a ProxyStore Streaming guide by gpauloski in https://github.com/proxystore/proxystore/pull/457
* Refresh README with better overview, links, and badges by gpauloski in https://github.com/proxystore/proxystore/pull/459
* Alias docs for latest release as stable and make the default by gpauloski in https://github.com/proxystore/proxystore/pull/460
* Use `latest` as canonical docs version and symlink it to latest release by gpauloski in https://github.com/proxystore/proxystore/pull/474
* Add Harb et al. featured publication by gpauloski in https://github.com/proxystore/proxystore/pull/478

Development Workflows
* Bump actions/cache from 3 to 4 by dependabot in https://github.com/proxystore/proxystore/pull/456
* Enable and fix ruff `C90`, `SIM`, and `YTT` lints by gpauloski in https://github.com/proxystore/proxystore/pull/458


**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.2...v0.6.3

<!-- Release notes generated using configuration in .github/release.yml at v0.6.2 -->

Highlights

This is a small patch release that improves compatibility with the [ProxyStore Extensions package](https://github.com/proxystore/extensions).

* Features from `proxystore-ex` are re-exported in `proxystore.ex`.
* Documentation has been improved to reflect changes in `proxystore-ex`.

In addition, Python 3.12 is now fully supported and tested against!

Upgrade Steps


$ pip install --upgrade proxystore[all]


What's Changed

Enhancements
* Modules in `proxystore_ex` are re-exported under `proxystore.ex` by gpauloski in https://github.com/proxystore/proxystore/pull/428

Internal Changes
* Upgrade to `pytest_asyncio>=0.23.2` by gpauloski in https://github.com/proxystore/proxystore/pull/430
* Test against Python 3.12 by gpauloski in https://github.com/proxystore/proxystore/pull/431
* Improve logging verification in NAT type tests by gpauloski in https://github.com/proxystore/proxystore/pull/434

Documentation
* List ProxyStore presentations on publications page of docs by gpauloski in https://github.com/proxystore/proxystore/pull/423
* Add `proxystore[extensions]` installation option to install the extensions package by gpauloski in https://github.com/proxystore/proxystore/pull/427
* Add link in table of contents to guides in the extensions package by gpauloski in https://github.com/proxystore/proxystore/pull/435

Development Workflows
* Switch auto-formatting from `black` to `ruff format` by gpauloski in https://github.com/proxystore/proxystore/pull/429
* Bump actions/setup-python from 4 to 5 by dependabot in https://github.com/proxystore/proxystore/pull/433
* Fix wrong Python version requested in MacOS Python 3.9 tests by gpauloski in https://github.com/proxystore/proxystore/pull/426


**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.1...v0.6.2