Proxystore

<!-- Release notes generated using configuration in .github/release.yml at v0.6.5 -->

Highlights

This primary goal of this release is to improve some of the subtle challenges of working with `Proxy` objects. Specifically, that (1) serialization can fail on `Proxy` class attribute access, (2) serializers/compute frameworks will accidentally resolve a `Proxy` instance when doing simple operations like checking types or hashing, and (3) mypy cannot infer the types of most operations on a `Proxy[T]`.

* ProxyStore now uses a new slots-based `Proxy` implementation which will improve serialization compatibility.
* `Store.proxy(..., populate_target=True)` will now cache some attributes of the target object so that calling `hash()` or `isinstance()` on a proxy does not require the proxy to be resolved.
* An FAQ guide has been created to discuss these common pitfalls and their remedies.
* The `proxystore.mypy_plugin`, when enabled, allows mypy to infer attribute types on `Proxy[T]`.
* The `Store` now has a `register` flag for configuring a `Store` instances to always register themselves.
* Added the `StaticLifetime`

Upgrade Steps

bash
pip install --upgrade proxystore or proxystore[all]


What's Changed

Breaking Changes
* Improve flexibility of `Lifetime` protocol and add `StaticLifetime` implementation by gpauloski in https://github.com/proxystore/proxystore/pull/542

Enhancements
* Introduce new slots-based `Proxy` implementation by gpauloski in https://github.com/proxystore/proxystore/pull/539
* Add the `proxystore.mypy` mypy plugin for resolving `Proxy[T]` types by gpauloski in https://github.com/proxystore/proxystore/pull/545
* Add the `register` flag to auto-register a `Store` after initialization by gpauloski in https://github.com/proxystore/proxystore/pull/550
* Support caching target attributes in `Proxy` to avoid resolving when using `hash()` or `isinstance()` by gpauloski in https://github.com/proxystore/proxystore/pull/551
* Generate `Store.__repr__()` from `Store.config()` by gpauloski in https://github.com/proxystore/proxystore/pull/554

Bug Fixes
* Fix incorrect ns to ms conversion in `Metrics` by gpauloski in https://github.com/proxystore/proxystore/pull/538

Documentation
* Add Harish et al featured paper by gpauloski in https://github.com/proxystore/proxystore/pull/537
* Add "latest" DOI badge by gpauloski in https://github.com/proxystore/proxystore/pull/540
* Add an FAQ page and refresh documentation by gpauloski in https://github.com/proxystore/proxystore/pull/546
* Add a Dask Distributed guide by gpauloski in https://github.com/proxystore/proxystore/pull/553

Package Changes
* Remove citation metadata to preserve Zenodo defaults by gpauloski in https://github.com/proxystore/proxystore/pull/541

**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.4...v0.6.5

<!-- Release notes generated using configuration in .github/release.yml at v0.6.4 -->

Highlights

* Automatic proxy memory management via the "Ownership" model and "Lifetime" managers. Learn more in the new [Object Lifetimes Guide](https://docs.proxystore.dev/0.6.4/guides/object-lifetimes/).
* Added the `populate_target` flag to `Store.proxy()` for "pre-resolving" proxies.
* Renamed `proxystore.store.future.ProxyFuture` to `proxystore.store.future.Future`.
* Added support for Pydantic V1.

Upgrade Steps
bash
pip install --upgrade proxystore or proxystore[all]


What's Changed

Breaking Changes
* Rename `ProxyFuture` to `Future` by gpauloski in https://github.com/proxystore/proxystore/pull/487

Enhancements
* Add `done()` method to `Future` by gpauloski in https://github.com/proxystore/proxystore/pull/488
* Add the `iter_with_metadata()` and `iter_objects_with_metadata()` methods to `StreamConsumer` by gpauloski in https://github.com/proxystore/proxystore/pull/489
* Implement proxy ownership and borrowing references, rules, and interfaces by gpauloski in https://github.com/proxystore/proxystore/pull/493
* Initial proxy ownership model integration for the `Store` and task execution by gpauloski in https://github.com/proxystore/proxystore/pull/494
* Support exponential backoff factor in `PollingStoreFactory` by gpauloski in https://github.com/proxystore/proxystore/pull/498
* Add `populate_target` flag to `Store.proxy()` to set created proxies as pre-resolved by gpauloski in https://github.com/proxystore/proxystore/pull/500
* Document and add utilities for taking ownership of returned proxies by gpauloski in https://github.com/proxystore/proxystore/pull/502
* Expose `publisher` and `subscriber` attributes in `StreamConsumer` and `StreamProducer` respectively by gpauloski in https://github.com/proxystore/proxystore/pull/504
* Leave stores open by default in stream interfaces by gpauloski in https://github.com/proxystore/proxystore/pull/505
* Add queue-based Redis shim for streaming by gpauloski in https://github.com/proxystore/proxystore/pull/511
* Add `Lifetime` support for managing shared objects by gpauloski in https://github.com/proxystore/proxystore/pull/513
* Improve `unregister_store` and `Store.future()` logging by gpauloski in https://github.com/proxystore/proxystore/pull/524
* Mark `Connector.close()` as idempotent by gpauloski in https://github.com/proxystore/proxystore/pull/526

Bug Fixes
* Invalidate cache entry when using `Store._set()` by gpauloski in https://github.com/proxystore/proxystore/pull/492
* Fix `Store.put_batch()` called with empty list in `StreamProducer.flush_topic()` by gpauloski in https://github.com/proxystore/proxystore/pull/503
* Copy target by default in `borrow()`, `mut_borrow()`, and `into_owned()` by gpauloski in https://github.com/proxystore/proxystore/pull/506
* Fix `submit()` resolving `RefProxy` and `RefMutProxy` arguments by gpauloski in https://github.com/proxystore/proxystore/pull/507
* Fix `submit()` resolving `Proxy` arguments (re: 507) by gpauloski in https://github.com/proxystore/proxystore/pull/508
* Handle and pass on SIGINT propagated by Uvicorn after cleanup by gpauloski in https://github.com/proxystore/proxystore/pull/514
* Prevent evicting the target object when invalidated `OwnedProxy` instances are garbage collected by gpauloski in https://github.com/proxystore/proxystore/pull/525
* Log warning when an `EndpointConnector` finds an unstarted endpoint by gpauloski in https://github.com/proxystore/proxystore/pull/529
* Improve serialization error handling when resolving a proxy by gpauloski in https://github.com/proxystore/proxystore/pull/531

Internal Changes
* Fix `LeaseLifetime` failing on MacOS runners by gpauloski in https://github.com/proxystore/proxystore/pull/523

Documentation
* Fix typo in Object Lifetimes guide by gpauloski in https://github.com/proxystore/proxystore/pull/528

Package Changes
* Update copyright year by gpauloski in https://github.com/proxystore/proxystore/pull/496
* Add backwards compatibility for Pydantic V1 by gpauloski in https://github.com/proxystore/proxystore/pull/501

Development Workflows
* Move ruff linter settings to lint section by gpauloski in https://github.com/proxystore/proxystore/pull/490
* Use Apple Silicon GitHub Actions Runners by gpauloski in https://github.com/proxystore/proxystore/pull/491
* Bump softprops/action-gh-release from 1 to 2 by dependabot in https://github.com/proxystore/proxystore/pull/509


**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.3...v0.6.4

<!-- Release notes generated using configuration in .github/release.yml at v0.6.3 -->

Highlights

This patch introduces the beta ProxyStore streaming interface. Check out the [guide](https://docs.proxystore.dev/main/guides/streaming/) for more info!

Upgrade Steps


pip install --upgrade proxystore or proxystore[all]


What's Changed

Enhancements
* Switch from dataclasses to Pydantic BaseModels for configuration classes by gpauloski in https://github.com/proxystore/proxystore/pull/446
* Add prototype streaming interface by gpauloski in https://github.com/proxystore/proxystore/pull/447
* Add the `RedisPublisher` and `RedisSubscriber` for use with ProxyStore streams by gpauloski in https://github.com/proxystore/proxystore/pull/451
* Add the `KafkaPublisher` and `KafkaSubscriber` for use with ProxyStore streams by gpauloski in https://github.com/proxystore/proxystore/pull/452
* Add the `ZeroMQPublisher` and `ZeroMQSubscriber` for use with ProxyStore streams by gpauloski in https://github.com/proxystore/proxystore/pull/453
* Make `StreamProducer`/`StreamConsumer` not generic by gpauloski in https://github.com/proxystore/proxystore/pull/466
* Consolidate streaming-related modules under `proxystore.stream` by gpauloski in https://github.com/proxystore/proxystore/pull/468
* Simplify `Producer`/`Subscriber` interfaces and adjust to match other libraries by gpauloski in https://github.com/proxystore/proxystore/pull/470
* Make `StreamProducer`/`StreamConsumer` generic type containers by gpauloski in https://github.com/proxystore/proxystore/pull/471
* Add end of stream events between the `StreamProducer` and `StreamConsumer` by gpauloski in https://github.com/proxystore/proxystore/pull/472
* `StreamProducer` controls which `Store` instances are used for each stream topic by gpauloski in https://github.com/proxystore/proxystore/pull/473
* Support aggregation, batching, and filtering in stream interfaces by gpauloski in https://github.com/proxystore/proxystore/pull/475
* Add the `next_object()` and `iter_object()` methods to `StreamConsumer` by gpauloski in https://github.com/proxystore/proxystore/pull/476

Internal Changes
* Remove redundant unit test docstring requirements by gpauloski in https://github.com/proxystore/proxystore/pull/469

Documentation
* Add new protein generation paper by Dharuman et al. by gpauloski in https://github.com/proxystore/proxystore/pull/438
* Update and pin `mkdocstrings` + related packages by gpauloski in https://github.com/proxystore/proxystore/pull/454
* Add a ProxyStore Streaming guide by gpauloski in https://github.com/proxystore/proxystore/pull/457
* Refresh README with better overview, links, and badges by gpauloski in https://github.com/proxystore/proxystore/pull/459
* Alias docs for latest release as stable and make the default by gpauloski in https://github.com/proxystore/proxystore/pull/460
* Use `latest` as canonical docs version and symlink it to latest release by gpauloski in https://github.com/proxystore/proxystore/pull/474
* Add Harb et al. featured publication by gpauloski in https://github.com/proxystore/proxystore/pull/478

Development Workflows
* Bump actions/cache from 3 to 4 by dependabot in https://github.com/proxystore/proxystore/pull/456
* Enable and fix ruff `C90`, `SIM`, and `YTT` lints by gpauloski in https://github.com/proxystore/proxystore/pull/458


**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.2...v0.6.3

<!-- Release notes generated using configuration in .github/release.yml at v0.6.2 -->

Highlights

This is a small patch release that improves compatibility with the [ProxyStore Extensions package](https://github.com/proxystore/extensions).

* Features from `proxystore-ex` are re-exported in `proxystore.ex`.
* Documentation has been improved to reflect changes in `proxystore-ex`.

In addition, Python 3.12 is now fully supported and tested against!

Upgrade Steps


$ pip install --upgrade proxystore[all]


What's Changed

Enhancements
* Modules in `proxystore_ex` are re-exported under `proxystore.ex` by gpauloski in https://github.com/proxystore/proxystore/pull/428

Internal Changes
* Upgrade to `pytest_asyncio>=0.23.2` by gpauloski in https://github.com/proxystore/proxystore/pull/430
* Test against Python 3.12 by gpauloski in https://github.com/proxystore/proxystore/pull/431
* Improve logging verification in NAT type tests by gpauloski in https://github.com/proxystore/proxystore/pull/434

Documentation
* List ProxyStore presentations on publications page of docs by gpauloski in https://github.com/proxystore/proxystore/pull/423
* Add `proxystore[extensions]` installation option to install the extensions package by gpauloski in https://github.com/proxystore/proxystore/pull/427
* Add link in table of contents to guides in the extensions package by gpauloski in https://github.com/proxystore/proxystore/pull/435

Development Workflows
* Switch auto-formatting from `black` to `ruff format` by gpauloski in https://github.com/proxystore/proxystore/pull/429
* Bump actions/setup-python from 4 to 5 by dependabot in https://github.com/proxystore/proxystore/pull/433
* Fix wrong Python version requested in MacOS Python 3.9 tests by gpauloski in https://github.com/proxystore/proxystore/pull/426


**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.1...v0.6.2

<!-- Release notes generated using configuration in .github/release.yml at v0.6.1 -->

This release adds some quality-of-life improvements and the new `ProxyFuture` interface.

Highlights
* Added the `ProxyFuture` and `Store.future()` interfaces. Learn how they work in the [Proxy Futures Guide](https://docs.proxystore.dev/main/guides/proxy-futures/).
* ProxyStore Endpoints support Globus Client Identities.
* Added tools for checking if ProxyStore Endpoint peering will work on your network.
* Better exception handling and error propogation when using ProxyStore endpoints.

Read the full changelog before for details.

Upgrade Steps


$ pip install --upgrade proxystore[all]


If you previously imported the `Connector` protocol, the import has changed:
python
from proxystore.connectors.connector import Connector old
from proxystore.connectors.procotols import Connector new


What's Changed

Breaking Changes
* Extend some `Connector`s to implement the `DeferrableConnector` protocol by gpauloski in https://github.com/proxystore/proxystore/pull/396
* Note this is only a breaking change because the `proxystore.connectors.connector` was renamed to `proxystore.connectors.protocols`.

Enhancements
* Add optional `max_message_size` check to `RelayServer` by gpauloski in https://github.com/proxystore/proxystore/pull/403
* Support Globus Client Identities for endpoint authentication by gpauloski in https://github.com/proxystore/proxystore/pull/407
* Add the `ProxyFuture` interface and `Store.future()` method by gpauloski in https://github.com/proxystore/proxystore/pull/414
* Improve propagation of errors in endpoint route handlers and the `EndpointConnector` by gpauloski in https://github.com/proxystore/proxystore/pull/416
* Add NAT type checks and tools for endpoint peering by gpauloski in https://github.com/proxystore/proxystore/pull/418

Bug Fixes
* Log unhandled endpoint serving exceptions by gpauloski in https://github.com/proxystore/proxystore/pull/401

Internal Changes
* Split `proxystore.store.base` into multiple modules by gpauloski in https://github.com/proxystore/proxystore/pull/395
* Catch `AttributeError` in `PeerManager` test during cancelled asyncio task clean up by gpauloski in https://github.com/proxystore/proxystore/pull/406
* Skip pytest-asyncio v0.22.0 by gpauloski in https://github.com/proxystore/proxystore/pull/409
* Remove unused parameter `metrics` to `StoreFactory` by gpauloski in https://github.com/proxystore/proxystore/pull/415

Documentation
* Fix bad relative link in endpoints guide by gpauloski in https://github.com/proxystore/proxystore/pull/399
* Remove references to legacy docs by gpauloski in https://github.com/proxystore/proxystore/pull/400
* Update preferred citations to SC23 proceedings by gpauloski in https://github.com/proxystore/proxystore/pull/408
* Add ProxyImports featured publication by gpauloski in https://github.com/proxystore/proxystore/pull/412

Development Workflows
* Test against all Python versions on MacOS by gpauloski in https://github.com/proxystore/proxystore/pull/387


**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.6.0...v0.6.1

<!-- Release notes generated using configuration in .github/release.yml at v0.6.0 -->

This release introduces major changes to the serving of endpoints and relay servers and removes some features marked as deprecated in v0.5.0. Please read the "Upgrade Steps" section to see if you will be affected by any of the breaking changes.

Highlights

* Endpoints now use Globus Auth and a default relay server for peering.
* Endpoint configuration files are now TOML and include more options.
* The `proxystore-globus-auth` CLI now includes `login` and `logout` subcommands.
* Moved `proxystore.connectors.dim` module to the [`proxystore-ex`](https://github.com/proxystore/extensions) package.
* Removed deprecated `XStore` compatibility wrappers.
* Removed Python 3.7 support.

Upgrade Steps

bash
$ pip install --upgrade proxystore[all]


Users of the following features will need to make additional changes:
* `GlobusConnector`
* ProxyStore Endpoints
* ProxyStore Relay Servers

Globus Auth Tokens for the `GlobusConnector`

The way Globus Auth tokens are stored has changed. To upgrade, remove the old tokens and reauthenticate. We suggest including the UUIDs of any Globus Collections you plan to use.
bash
$ rm ~/.local/share/proxystore/globus-tokens.json
$ proxystore-globus-auth login -c <COLLECTION_UUID> -c <COLLECTION_UUID>

You may also wish to revoke old tokens granted to the "ProxyStore Application." Visit [https://app.globus.org/settings/consents](https://app.globus.org/settings/consents) and select "Manage Your Consents" to see all grants and delete them.

ProxyStore Endpoints Configuration Files

Configuration files have changed, so existing endpoints will fail to start. You can perform a clean removal by removing the old endpoint directory.
bash
$ rm -r ~/.local/share/proxystore/<ENDPOINT_NAME>
$ proxystore-endpoint configure <ENDPOINT_NAME>

Alternatively, you can replace the old `config.json` files with a new `config.toml` file in your endpoint directory. A sample configuration file can be found in the [Endpoints Guide](https://docs.proxystore.dev/main/guides/endpoints/#endpoint-cli).

ProxyStore Relay Server Updates

The ProxyStore Relay Server implementation has changed, so v0.5 endpoints and relay servers are not compatible with v0.6. A [Relay Serving](https://docs.proxystore.dev/main/guides/relay-serving/) guide is now available to learn how to serve v0.6 relays.

What's Changed

Breaking Changes
* Drop Python 3.7 support by gpauloski in https://github.com/proxystore/proxystore/pull/337
* Remove deprecated `XStore` wrappers by gpauloski in https://github.com/proxystore/proxystore/pull/338
* Enable dependency injection model for relay client with `PeerManager` and `PeerConnection` by gpauloski in https://github.com/proxystore/proxystore/pull/363
* Enable dependency injection model for `PeerManager` and `Endpoint` by gpauloski in https://github.com/proxystore/proxystore/pull/365
* Refactor Globus authentication tools by gpauloski in https://github.com/proxystore/proxystore/pull/367
* Refactor `proxystore.p2p` and `proxystore.utils` by gpauloski in https://github.com/proxystore/proxystore/pull/369
* Merge basic and Globus relay server implementations by gpauloski in https://github.com/proxystore/proxystore/pull/372
* Use TOML for endpoint and relay configuration files by gpauloski in https://github.com/proxystore/proxystore/pull/374
* Migrate DIM connectors to extensions package by gpauloski in https://github.com/proxystore/proxystore/pull/375

Enhancements
* Improve error logging in background tasks by gpauloski in https://github.com/proxystore/proxystore/pull/342
* Improve relay server logging by gpauloski in https://github.com/proxystore/proxystore/pull/345
* Improve auto-reconnection in relay server client by gpauloski in https://github.com/proxystore/proxystore/pull/346
* Create prototype Globus Auth relay server by gpauloski in https://github.com/proxystore/proxystore/pull/370
* Make endpoints support Globus Auth relays (62) by gpauloski in https://github.com/proxystore/proxystore/pull/373
* Set default relay address and update endpoints guide by gpauloski in https://github.com/proxystore/proxystore/pull/379
* Improve logging when tokens are missing on endpoint start by gpauloski in https://github.com/proxystore/proxystore/pull/380

Bug Fixes
* Correct unit conversion from ns by WardLT in https://github.com/proxystore/proxystore/pull/358

Internal Changes
* Move object size check from `Endpoint` to `Storage.set()` by gpauloski in https://github.com/proxystore/proxystore/pull/376
* Fix flakey unit tests by gpauloski in https://github.com/proxystore/proxystore/pull/378

Documentation
* Move DOI badge to citation sections of docs and README by gpauloski in https://github.com/proxystore/proxystore/pull/348

Package Changes
* Create CITATION.cff by gpauloski in https://github.com/proxystore/proxystore/pull/349
* Create CODE_OF_CONDUCT.md by gpauloski in https://github.com/proxystore/proxystore/pull/350
* Create CONTRIBUTING.md and SECURITY.md by gpauloski in https://github.com/proxystore/proxystore/pull/351

Development Workflows
* Bump actions/checkout from 3 to 4 by dependabot in https://github.com/proxystore/proxystore/pull/355

**Full Changelog**: https://github.com/proxystore/proxystore/compare/v0.5.2...v0.6.0