Smartredis

Released on 14 May 2024

Description

- Improve client error logging
- Fix pylint regression error
- Fix build wheel error
- Fix header styling issue
- Correct changelog indention
- Automate the creation of release notes
- Auto-post release PR to develop from master
- Upgrade ubuntu to 22.04 and gcc to 11
- Drop Python 3.8 support
- Fix C++ cosmetic defects leading to compiler warnings
- Re-enable SR_PEDANTIC for the Makefile targets
- Enforce changelog updates
- Removed unused TensorBase constructor parameter
- Remove unused parameter in internal redis cluster method
- Enforce matching TensorType for DataSet::unpack_tensor()
- Update CI for Intel suite
- Add socket time out environment variable
- Fix inconsistency in C-API ConfigOptions is_configured() parameters
- Bump redis dep to 7.2.4
- Fix widths field for list-table in install documentation
- Remove a vestigial requirements.txt file

Detailed Notes

- Ensure errors raised from client include details
([PR485](https://github.com/CrayLabs/SmartRedis/pull/485))
- Pin pylint to fix regression error
([PR492](https://github.com/CrayLabs/SmartRedis/pull/492))
- Add cstdint import to fix ubuntu with gcc wheel build
([PR491](https://github.com/CrayLabs/SmartRedis/pull/491))
- Incorrect lineup of the changelog page index. This fixes the header
sizes to avoid this issue.
([PR489](https://github.com/CrayLabs/SmartRedis/pull/489))
- After converting from rst to md, readthedocs began throwing indention
errors in old release info. This fixes the styling.
([PR488](https://github.com/CrayLabs/SmartRedis/pull/488))
- Add a configuration file to the root of .github/ to configure
the generated release notes.
([PR487](https://github.com/CrayLabs/SmartRedis/pull/487))
- Add to github release workflow to auto generate a pull request from master
into develop for release.
([PR486](https://github.com/CrayLabs/SmartRedis/pull/486))
- After dropping support for Python 3.8, ubuntu and gcc need to be
upgraded. ([PR484](https://github.com/CrayLabs/SmartRedis/pull/484))
- Python 3.8 is reaching its end-of-life in October, 2024, so it will
no longer continue to be supported.
([PR482](https://github.com/CrayLabs/SmartRedis/pull/482))
- Fixes some mainly cosmetic defects in the C++ client that were
leading to warnings when pedantic compiler flags were enabled
([PR476](https://github.com/CrayLabs/SmartRedis/pull/476))
- Re-enable SR_PEDANTIC for the [test-lib]{.title-ref} and
[test-lib-with-fortran]{.title-ref} Makefile targets
([PR476](https://github.com/CrayLabs/SmartRedis/pull/476))
- Add Github Actions workflow that checks if changelog is edited on
pull requests into develop.
([PR480](https://github.com/CrayLabs/SmartRedis/pull/480))
- The TensorBase constructor SRMemoryLayout parameter was removed
because it was not used. It is not needed as a member variable
because all Tensor\<T\> objects store internal representations in
contiguous memory.
([PR479](https://github.com/CrayLabs/SmartRedis/pull/479))
- Client::unpack_tensor() enforces that the user-provided TensorType
matches the known tensor type. Now DataSet::unpack_tensor() enforces
the same condition.
([PR478](https://github.com/CrayLabs/SmartRedis/pull/478))
- Removes an unused parameter in the
RedisCluster::\_get_model_script_db() method.
([PR477](https://github.com/CrayLabs/SmartRedis/pull/477))
- Version numbers changed for the Intel Compiler chain that lead to
the C and C++ compilers not being available. Now, the entirety of
the Base and HPC kits are installed to ensure consistent versions.
([PR475](https://github.com/CrayLabs/SmartRedis/pull/475))
- Add the socket timeout parameter as a user-configurable option via
environment variables.
([PR474](https://github.com/CrayLabs/SmartRedis/pull/474))
- Fix an inconsistency in the C-API ConfigOptions is_configured()
parameter names.
([PR471](https://github.com/CrayLabs/SmartRedis/pull/471))
- Fix an issue where incorrect compiler flags are defined and result
in build failures due to the redis_fstat macro.
([PR470](https://github.com/CrayLabs/SmartRedis/pull/470))
- Fix wrong widths value which was preventing table from displaying.
([PR468](https://github.com/CrayLabs/SmartRedis/pull/468))
- The requirements.txt file is unused, therefore removing.
([PR462](https://github.com/CrayLabs/SmartRedis/pull/462))

Released on February 16, 2024

Description

- Fixed bug which was sending tensors to the database twice (Python
Client)

Detailed Notes

- A previous bug fix for the Python client which addressed a problem
when sending numpy views inadvertently kept the original put_tensor
call in place. This essentially doubles the cost of the operation.
([PR464](https://github.com/CrayLabs/SmartRedis/pull/464))

Released on February 15, 2024

Description

- Fix bug when sending an array view
- Add concurrency groups for Github Action testing
- Update license to include 2024
- Increase build space for Github Actions
- Update README python versions
- Expose Typehints
- Update supported python versions \[Add 3.11, remove 3.7\]
- Tweak the build system to enable building SmartRedis with Nvidia\'s
NVHPC toolchain
- Improvements/upgrades to the container used for Github actions
- Code updates to avoid compiler warnings
- Added developer documentation on how to run a single test case and
eliminated duplicative environment variables
- Resolve a linting issue with pybind-to-python error propagation
- Use mutable fields to enable Dataset get methods that store memory
to be marked const

Detailed Notes

- Detect whether the tensor the user is sending is a view and if so,
make an explicit copy.
([PR453](https://github.com/CrayLabs/SmartRedis/pull/453))
- Add support to concurrency groups in the [run_tests]{.title-ref}
workflow. ([PR456](https://github.com/CrayLabs/SmartRedis/pull/456))
- Update license to include 2024.
([PR454](https://github.com/CrayLabs/SmartRedis/pull/454))
- Add new Github Action that removes unneeded packages and resizes the
root disk space.
([PR455](https://github.com/CrayLabs/SmartRedis/pull/455))
- Update developer documentation to reflect newly supported versions
of Python ([PR450](https://github.com/CrayLabs/SmartRedis/pull/450))
([PR452](https://github.com/CrayLabs/SmartRedis/pull/452))
- Add and ship [py.typed]{.title-ref} marker to expose inline type
hints ([PR451](https://github.com/CrayLabs/SmartRedis/pull/451))
- Deprecate support for Python 3.7 by removing from the allowed Python
versions ([PR450](https://github.com/CrayLabs/SmartRedis/pull/450))
- Update Python package dependencies to add support for Python 3.11
([PR450](https://github.com/CrayLabs/SmartRedis/pull/450))
- Change the order of arguments in our MakeFile to ensure that all
dependencies are compiled with GCC
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- Add new user-configurable parameters DEP_CC, DEP_CXX to control
which compiler is used to build dependencies
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- Ameliorate some compiler warnings related that were flagged in GCC
12 (unreachable code blocks, signed/unsigned mismatches)
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- CI/CD: Bump the container version used in Github Actions Ubuntu
22.04 to be able to start testing GCC 12
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- CI/CD: Bump the versions of GCC used in testing to the currently
maintained versions
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- CI/CD: Add NVHPC to the testing matrix
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- CI/CD: Test the shared/static compilations and examples with all
compilers ([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- CI/CD: Compile Redis and RedisAI and use those versions in testing
instead of extracting from a container
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- CI/CD: Bump the version of Redis used in testing to 7.0.5, the same
version as we use with SmartSim
([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- CI/CD: Pin the Torch version to 1.11.0, the same as supported in
SmartSim ([PR448](https://github.com/CrayLabs/SmartRedis/pull/448))
- Added developer documentation on how to run a single test case with
the new test/build system and eliminated use of
SMARTREDIS_TEST_DEVICE and SMARTREDIS_TEST_CLUSTER environment
variables ([PR445](https://github.com/CrayLabs/SmartRedis/pull/445))
- Resolve a linting issue with pybind-to-python error propagation by
changing import format and narrowing the lookup of pybind error
names to the error module
([PR444](https://github.com/CrayLabs/SmartRedis/pull/444))
- Use mutable fields to enable Dataset get methods that store memory
to be marked const
([PR443](https://github.com/CrayLabs/SmartRedis/pull/443))

Released on December 18, 2023

Description

- Unpin the Intel Fortran compiler in CI/CD
- Added a missing space in an error message
- Improved consistency of namespace declarations for C++ pybind
interface
- Improved const correctness of C++ Client
- Improved const correctness of C++ Dataset
- Updated documentation
- Added test cases for all Client construction parameter combinations
- Centralized dependency tracking to setup.cfg
- Improved robustness of Python client construction
- Updated Client and Dataset documentation
- Expanded list of allowed characters in the SSDB address
- Added coverage to SmartRedis Python API functions
- Improved responsiveness of library when attempting connection to
missing backend database
- Moved testing of examples to on-commit testing in CI/CD pipeline
- Added name retrieval function to the DataSet object
- Updated RedisAI version used in post-commit check-in testing in
Github pipeline
- Allow strings in Python interface for Client.run_script,
Client.run_script_multiGPU
- Improved support for model execution batching
- Added support for model chunking
- Updated the third-party RedisAI component
- Updated the third-party lcov component
- Add link to contributing guidelines
- Added link to contributing guidelines
- Added support for multiple backend databases via a new Client
constructor that accepts a ConfigOptions object

Detailed Notes

- Unpin the Intel Fortran compiler in CI/CD. This requires running the
compiler setup script twice, once for Fortran and once for other
languages, since they\'re on different releases
([PR436](https://github.com/CrayLabs/SmartRedis/pull/436))
- Added a missing space in an error message
([PR435](https://github.com/CrayLabs/SmartRedis/pull/435))
- Made the declaration of the py namespace in py\*.h consistently
outside the SmartRedis namespace declaration
([PR434](https://github.com/CrayLabs/SmartRedis/pull/434))
- Fields in several C++ API methods are now properly marked as const
([PR430](https://github.com/CrayLabs/SmartRedis/pull/430))
- The Dataset add_tensor method is now const correct, as are all
internal the methods it calls
([PR427](https://github.com/CrayLabs/SmartRedis/pull/427))
- Some broken links in the documentation were fixed, and the
instructions to run the tests were updated
([PR423](https://github.com/CrayLabs/SmartRedis/pull/423))
- Added test cases for all Client construction parameter combinations
([PR422](https://github.com/CrayLabs/SmartRedis/pull/422))
- Merged dependency lists from requirements.txt and
requirements-dev.txt into setup.cfg to have only one set of
dependencies going forward
([PR420](https://github.com/CrayLabs/SmartRedis/pull/420))
- Improved robustness of Python client construction by adding
detection of invalid kwargs
([PR419](https://github.com/CrayLabs/SmartRedis/pull/419)),
([PR421](https://github.com/CrayLabs/SmartRedis/pull/421))
- Updated the Client and Dataset API documentation to clarify which
interacts with the backend db
([PR416](https://github.com/CrayLabs/SmartRedis/pull/416))
- The SSDB address can now include \'-\' and \'\_\' as special
characters in the name. This gives users more options for naming the
UDS socket file
([PR415](https://github.com/CrayLabs/SmartRedis/pull/415))
- Added tests to increase Python code coverage
- Employed a Redis++ ConnectionsObject in the connection process to
establish a TCP timeout of 100ms during connection attempts
([PR413](https://github.com/CrayLabs/SmartRedis/pull/413))
- Moved testing of examples to on-commit testing in CI/CD pipeline
([PR412](https://github.com/CrayLabs/SmartRedis/pull/412))
- Added a function to the DataSet class and added a test
- Updated RedisAI version used in post-commit check-in testing in
Github pipeline to a version that supports fetch of model chunking
size ([PR408](https://github.com/CrayLabs/SmartRedis/pull/408))
- Allow users to pass single keys for the inputs and outputs
parameters as a string for Python run_script and run_script_multigpu
- Exposed access to the Redis.AI MINBATCHTIMEOUT parameter, which
limits the delay in model execution when trying to accumulate
multiple executions in a batch
([PR406](https://github.com/CrayLabs/SmartRedis/pull/406))
- Models will now be automatically chunked when sent to/received from
the backed database. This allows use of models greater than 511MB in
size. ([PR404](https://github.com/CrayLabs/SmartRedis/pull/404))
- Updated from RedisAI v1.2.3 (test target)/v1.2.4 and v1.2.5 (CI/CD
pipeline) to v1.2.7
([PR402](https://github.com/CrayLabs/SmartRedis/pull/402))
- Updated lcov from version 1.15 to 2.0
([PR396](https://github.com/CrayLabs/SmartRedis/pull/396))
- Create CONTRIBUTIONS.md file that points to the contribution
guideline for both SmartSim and SmartRedis
([PR395](https://github.com/CrayLabs/SmartRedis/pull/395))
- Migrated to ConfigOptions-based Client construction, adding multiple
database support
([PR353](https://github.com/CrayLabs/SmartRedis/pull/353))

Released on September 13, 2023

Description

- Reduced number of suppressed lint errors
- Expanded documentation of aggregation lists
- Updated third-party software dependencies to current versions
- Updated post-merge tests in CI/CD to work with new test system
- Enabled static builds of SmartRedis
- Improve robustness of test runs
- Fixed installation link
- Updated supported languages documentation
- Removed obsolete files
- Added pylint to CI/CD pipeline and mitigate existing errors
- Improved clustered redis initialization

Detailed Notes

- Refactor factory for ConfigOptions to avoid using protected member
outside an instance
([PR393](https://github.com/CrayLabs/SmartRedis/pull/393))
- Added a new advanced topics documentation page with a section on
aggregation lists
([PR390](https://github.com/CrayLabs/SmartRedis/pull/390))
- Updated pybind (2.10.3 =\> 2.11.1), hiredis (1.1.0 =\> 1.2.0), and
redis++ (1.3.5 =\> 1.3.10) dependencies to current versions
([PR389](https://github.com/CrayLabs/SmartRedis/pull/389))
- Post-merge tests in CI/CD have been updated to interface cleanly
with the new test system that was deployed in the previous release
([PR388](https://github.com/CrayLabs/SmartRedis/pull/388))
- Static builds of SmartRedis can now work with Linux platforms.
Fortran is tested with GNU, PGI, Intel compilers
([PR386](https://github.com/CrayLabs/SmartRedis/pull/386))
- Preserve the shell output of test runs while making sure that server
shutdown happens unconditionally
([PR381](https://github.com/CrayLabs/SmartRedis/pull/381))
- Fix incorrect link to installation documentation
([PR380](https://github.com/CrayLabs/SmartRedis/pull/380))
- Update language support matrix in documentation to reflect updates
from the last release
([PR379](https://github.com/CrayLabs/SmartRedis/pull/379))
- Fix typo causing startup failure in utility script for unit tests
([PR378](https://github.com/CrayLabs/SmartRedis/pull/378))
- Update pylint configuration and version, mitigate most errors,
execute in CI/CD pipeline
([PR371](https://github.com/CrayLabs/SmartRedis/pull/371),
[PR382](https://github.com/CrayLabs/SmartRedis/pull/382))
- Deleted obsolete build and testing files that are no longer needed
with the new build and test system
([PR366](https://github.com/CrayLabs/SmartRedis/pull/366))
- Reuse existing redis connection when mapping the Redis cluster
([PR364](https://github.com/CrayLabs/SmartRedis/pull/364))

Released on July 5, 2023

Description

This release revamps the build and test systems for SmartRedis as well
as improving compatibility with different Fortran compilers and laying
the groundwork for future support for interacting with multiple
concurrent backend databases:

- Documentation improvements
- Improved compatibility of type hints with third-party software
- Added type hints to the Python interface layer
- Add support for Python 3.10
- Updated setup.py to work with the new build system
- Remove unneeded method from Python SRObject class
- Fixed a memory leak in the C layer
- Revamp SmartRedis test system
- Remove debug output in pybind layer
- Update Hiredis version to 1.1.0
- Enable parallel build for the SmartRedis examples
- Experimental support for Nvidia toolchain
- Major revamp of build and test systems for SmartRedis
- Refactor Fortran methods to return default logical kind
- Update CI/CD tests to use a modern version of MacOS
- Fix the spelling of the Dataset destructor\'s C interface (now
DeallocateDataSet)
- Update Redis++ version to 1.3.8
- Refactor third-party software dependency installation
- Add pip-install target to Makefile to automate this process going
forward (note: this was later removed)
- Added infrastructure for multiDB support

Detailed Notes

- Assorted updates and clarifications to the documentation
([PR367](https://github.com/CrayLabs/SmartRedis/pull/367))
- Turn [ParamSpec]{.title-ref} usage into forward references to not
require [typing-extensions]{.title-ref} at runtime
([PR365](https://github.com/CrayLabs/SmartRedis/pull/365))
- Added type hints to the Python interface layer
([PR361](https://github.com/CrayLabs/SmartRedis/pull/361))
- List Python 3.10 support and loosen PyTorch requirement to allow for
versions support Python 3.10
([PR360](https://github.com/CrayLabs/SmartRedis/pull/360))
- Streamlined setup.py to simplify Python install (PR359)
- Remove from_pybind() from Python SRObject class as it\'s not needed
and didn\'t work properly anyway
([PR358](https://github.com/CrayLabs/SmartRedis/pull/358))
- Fixed memory leaked from the C layer when calling
get_string_option()
([PR357](https://github.com/CrayLabs/SmartRedis/pull/357))
- Major revamp to simplify use of SmartRedis test system, automating
most test processes
([PR356](https://github.com/CrayLabs/SmartRedis/pull/356))
- Remove debug output in pybind layer associated with put_dataset
([PR352](https://github.com/CrayLabs/SmartRedis/pull/352))
- Updated to the latest version of Hiredis (1.1.0)
([PR351](https://github.com/CrayLabs/SmartRedis/pull/351))
- Enable parallel build for the SmartRedis examples by moving utility
Fortran code into a small static library
([PR349](https://github.com/CrayLabs/SmartRedis/pull/349))
- For the NVidia toolchain only: Replaces the assumed rank feature of
F2018 used in the Fortran client with assumed shape arrays, making
it possible to compile SmartRedis with the Nvidia toolchain.
([PR346](https://github.com/CrayLabs/SmartRedis/pull/346))
- Rework the build and test system to improve maintainability of the
library. There have been several significant changes, including that
Python and Fortran clients are no longer built by defaults and that
there are Make variables that customize the build process. Please
review the build documentation and `make help` to see all that has
changed. ([PR341](https://github.com/CrayLabs/SmartRedis/pull/341))
- Many Fortran routines were returning logical kind = c_bool which
turns out not to be the same default kind of most Fortran compilers.
These have now been refactored so that users need not import
[iso_c\_binding]{.title-ref} in their own applications
([PR340](https://github.com/CrayLabs/SmartRedis/pull/340))
- Update MacOS version in CI/CD tests from 10.15 to 12.0
([PR339](https://github.com/CrayLabs/SmartRedis/pull/339))
- Correct the spelling of the C DataSet destruction interface from
DeallocateeDataSet to DeallocateDataSet
([PR338](https://github.com/CrayLabs/SmartRedis/pull/338))
- Updated the version of Redis++ to v1.3.8 to pull in a change that
ensures the redis++.pc file properly points to the generated
libraries ([PR334](https://github.com/CrayLabs/SmartRedis/pull/334))
- Third-party software dependency installation is now handled in the
Makefile instead of separate scripts
- New pip-install target in Makefile will be a dependency of the lib
target going forward so that users don\'t have to manually pip
install SmartRedis in the future
([PR330](https://github.com/CrayLabs/SmartRedis/pull/330))
- Added ConfigOptions class and API, which will form the backbone of
multiDB support
([PR303](https://github.com/CrayLabs/SmartRedis/pull/303))