Released on April 12, 2023
Description
This release of SmartSim had a focus on polishing and extending exiting
features already provided by SmartSim. Most notably, this release
provides support to allow users to colocate their models with an
orchestrator using Unix domain sockets and support for launching models
as batch jobs.
Additionally, SmartSim has updated its tool chains to provide a better
user experience. Notably, SmarSim can now be used with Python 3.10,
Redis 7.0.5, and RedisAI 1.2.7. Furthermore, SmartSim now utilizes
SmartRedis\'s aggregation lists to streamline the use and extension of
ML data loaders, making working with popular machine learning frameworks
in SmartSim a breeze.
A full list of changes and detailed notes can be found below:
- Add support for colocating an orchestrator over UDS
- Add support for Python 3.10, deprecate support for Python 3.7 and
RedisAI 1.2.3
- Drop support for Ray
- Update ML data loaders to make use of SmartRedis\'s aggregation
lists
- Allow for models to be launched independently as batch jobs
- Update to current version of Redis to 7.0.5
- Add support for RedisAI 1.2.7, pyTorch 1.11.0, Tensorflow 2.8.0,
ONNXRuntime 1.11.1
- Fix bug in colocated database entrypoint when loading PyTorch models
- Fix test suite behavior with environment variables
Detailed Notes
- Running some tests could result in some SmartSim-specific
environment variables to be set. Such environment variables are now
reset after each test execution. Also, a warning for environment
variable usage in Slurm was added, to make the user aware in case an
environment variable will not be assigned the desired value with
[\--export]{.title-ref}.
([SmartSim-PR270](https://github.com/CrayLabs/SmartSim/pull/270))
- The PyTorch and TensorFlow data loaders were update to make use of
aggregation lists. This breaks their API, but makes them easier to
use.
([SmartSim-PR264](https://github.com/CrayLabs/SmartSim/pull/264))
- The support for Ray was dropped, as its most recent versions caused
problems when deployed through SmartSim. We plan to release a
separate add-on library to accomplish the same results. If you are
interested in getting the Ray launch functionality back in your
workflow, please get in touch with us!
([SmartSim-PR263](https://github.com/CrayLabs/SmartSim/pull/263))
- Update from Redis version 6.0.8 to 7.0.5.
([SmartSim-PR258](https://github.com/CrayLabs/SmartSim/pull/258))
- Adds support for Python 3.10 without the ONNX machine learning
backend. Deprecates support for Python 3.7 as it will stop receiving
security updates. Deprecates support for RedisAI 1.2.3. Update the
build process to be able to correctly fetch supported dependencies.
If a user attempts to build an unsupported dependency, an error
message is shown highlighting the discrepancy.
([SmartSim-PR256](https://github.com/CrayLabs/SmartSim/pull/256))
- Models were given a [batch_settings]{.title-ref} attribute. When
launching a model through [Experiment.start]{.title-ref} the
[Experiment]{.title-ref} will first check for a non-nullish value at
that attribute. If the check is satisfied, the
[Experiment]{.title-ref} will attempt to wrap the underlying run
command in a batch job using the object referenced at
[Model.batch_settings]{.title-ref} as the batch settings for the
job. If the check is not satisfied, the [Model]{.title-ref} is
launched in the traditional manner as a job step.
([SmartSim-PR245](https://github.com/CrayLabs/SmartSim/pull/245))
- Fix bug in colocated database entrypoint stemming from uninitialized
variables. This bug affects PyTorch models being loaded into the
database.
([SmartSim-PR237](https://github.com/CrayLabs/SmartSim/pull/237))
- The release of RedisAI 1.2.7 allows us to update support for recent
versions of PyTorch, Tensorflow, and ONNX
([SmartSim-PR234](https://github.com/CrayLabs/SmartSim/pull/234))
- Make installation of correct Torch backend more reliable according
to instruction from PyTorch
- In addition to TCP, add UDS support for colocating an orchestrator
with models. Methods [Model.colocate_db_tcp]{.title-ref} and
[Model.colocate_db_uds]{.title-ref} were added to expose this
functionality. The [Model.colocate_db]{.title-ref} method remains
and uses TCP for backward compatibility
([SmartSim-PR246](https://github.com/CrayLabs/SmartSim/pull/246))