Pydvl

Fixed

- `FutureWarning` for `ParallelConfig` constantly raised without actually
instantiating the object
[PR 562](https://github.com/aai-institute/pyDVL/pull/562)

Added

- New method `MSR Banzhaf` with accompanying notebook, and new stopping
criterion `RankCorrelation`
[PR 520](https://github.com/aai-institute/pyDVL/pull/520)
- New method: `NystroemSketchInfluence`
[PR 504](https://github.com/aai-institute/pyDVL/pull/504)
- New preconditioned block variant of conjugate gradient
[PR 507](https://github.com/aai-institute/pyDVL/pull/507)
- Improvements to documentation: fixes, links, text, example gallery, LFS and
more [PR 532](https://github.com/aai-institute/pyDVL/pull/532),
[PR 543](https://github.com/aai-institute/pyDVL/pull/543)
- Glossary of data valuation and influence terms in the documentation
[PR 537](https://github.com/aai-institute/pyDVL/pull/537
- Documentation about writing notes for new features, changes or deprecations
[PR 557](https://github.com/aai-institute/pyDVL/pull/557)

Fixed

- Bug in `LissaInfluence`, when not using CPU device
[PR 495](https://github.com/aai-institute/pyDVL/pull/495)
- Memory issue with `CgInfluence` and `ArnoldiInfluence`
[PR 498](https://github.com/aai-institute/pyDVL/pull/498)
- Raising specific error message with install instruction, when trying to load
`pydvl.utils.cache.memcached` without `pymemcache` installed.
If `pymemcache` is available, all symbols from `pydvl.utils.cache.memcached`
are available through `pydvl.utils.cache`
[PR 509](https://github.com/aai-institute/pyDVL/pull/509)

Changed

- Add property `model_dtype` to instances of type `TorchInfluenceFunctionModel`
- Bump versions of CI actions to avoid warnings
[PR 502](https://github.com/aai-institute/pyDVL/pull/502)
- Add Python Version 3.11 to supported versions
[PR 510](https://github.com/aai-institute/pyDVL/pull/510)
- Documentation improvements and cleanup
[PR 521](https://github.com/aai-institute/pyDVL/pull/521),
[PR 522](https://github.com/aai-institute/pyDVL/pull/522)
- Simplified parallel backend configuration
[PR 549](https://github.com/mkdocstrings/mkdocstrings/issues/615)

Added

- Implement new method: `EkfacInfluence`
[PR 451](https://github.com/aai-institute/pyDVL/issues/451)
- New notebook to showcase ekfac for LLMs
[PR 483](https://github.com/aai-institute/pyDVL/pull/483)
- Implemented exact games in Castro et al. 2009 and 2017
[PR 341](https://github.com/appliedAI-Initiative/pyDVL/pull/341)

Fixed

- Bug in using `DaskInfluenceCalcualator` with `TorchnumpyConverter`
for single dimensional arrays
[PR 485](https://github.com/aai-institute/pyDVL/pull/485)
- Fix implementations of `to` methods of `TorchInfluenceFunctionModel`
implementations [PR 487](https://github.com/aai-institute/pyDVL/pull/487)
- Fixed bug with checking for converged values in semivalues
[PR 341](https://github.com/appliedAI-Initiative/pyDVL/pull/341)

Changed

- Add applications of data valuation section, display examples more prominently,
make all sections visible in table of contents, use mkdocs material cards
in the home page [PR 492](https://github.com/aai-institute/pyDVL/pull/492)

Added

- New cache backends: InMemoryCacheBackend and DiskCacheBackend
[PR 458](https://github.com/aai-institute/pyDVL/pull/458)
- New influence function interface `InfluenceFunctionModel`
- Data parallel computation with `DaskInfluenceCalculator`
[PR 26](https://github.com/aai-institute/pyDVL/issues/26)
- Sequential batch-wise computation and write to disk with
`SequentialInfluenceCalculator`
[PR 377](https://github.com/aai-institute/pyDVL/issues/377)
- Adapt notebooks to new influence abstractions
[PR 430](https://github.com/aai-institute/pyDVL/issues/430)

Changed

- Refactor and simplify caching implementation
[PR 458](https://github.com/aai-institute/pyDVL/pull/458)
- Simplify display of computation progress
[PR 466](https://github.com/aai-institute/pyDVL/pull/466)
- Improve readme and explain better the examples
[PR 465](https://github.com/aai-institute/pyDVL/pull/465)
- Simplify and improve tests, add CodeCov code coverage
[PR 429](https://github.com/aai-institute/pyDVL/pull/429)
- **Breaking Changes**
- Removed `compute_influences` and all related code.
Replaced by new `InfluenceFunctionModel` interface. Removed modules:
- influence.general
- influence.inversion
- influence.twice_differentiable
- influence.torch.torch_differentiable

Fixed
- Import bug in README [PR 457](https://github.com/aai-institute/pyDVL/issues/457)

Added

- New method: Class-wise Shapley values
[PR 338](https://github.com/aai-institute/pyDVL/pull/338)
- New method: Data-OOB by BastienZim
[PR 426](https://github.com/aai-institute/pyDVL/pull/426),
[PR $431](https://github.com/aai-institute/pyDVL/pull/431)
- Added `AntitheticPermutationSampler`
[PR 439](https://github.com/aai-institute/pyDVL/pull/439)
- Faster semi-value computation with per-index check of stopping criteria (optional)
[PR 437](https://github.com/aai-institute/pyDVL/pull/437)

Fixed

- Fix initialization of `data_names` in `ValuationResult.zeros()`
[PR 443](https://github.com/aai-institute/pyDVL/pull/443)

Changed

- No longer using docker within tests to start a memcached server
[PR 444](https://github.com/aai-institute/pyDVL/pull/444)
- Using pytest-xdist for faster local tests
[PR 440](https://github.com/aai-institute/pyDVL/pull/440)
- Improvements and fixes to notebooks
[PR 436](https://github.com/aai-institute/pyDVL/pull/436)
- Refactoring of parallel module. Old imports will stop working in v0.9.0
[PR 421](https://github.com/aai-institute/pyDVL/pull/421)

This is our first β release! We have worked hard to deliver improvements across
the board, with a focus on documentation and usability. We have also reworked
the internals of the `influence` module, improved parallelism and handling of
randomness.

Added

- Implemented solving the Hessian equation via spectral low-rank approximation
[PR 365](https://github.com/aai-institute/pyDVL/pull/365)
- Enabled parallel computation for Leave-One-Out values
[PR 406](https://github.com/aai-institute/pyDVL/pull/406)
- Added more abbreviations to documentation
[PR 415](https://github.com/aai-institute/pyDVL/pull/415)
- Added seed to functions from `pydvl.utils.numeric`, `pydvl.value.shapley` and
`pydvl.value.semivalues`. Introduced new type `Seed` and conversion function
`ensure_seed_sequence`.
[PR 396](https://github.com/aai-institute/pyDVL/pull/396)
- Added `batch_size` parameter to `compute_banzhaf_semivalues`,
`compute_beta_shapley_semivalues`, `compute_shapley_semivalues` and
`compute_generic_semivalues`.
[PR 428](https://github.com/aai-institute/pyDVL/pull/428)
- Added classwise Shapley as proposed by (Schoch et al. 2021)
[https://arxiv.org/abs/2211.06800]
[PR 338](https://github.com/aai-institute/pyDVL/pull/338)

Changed

- Replaced sphinx with mkdocs for documentation. Major overhaul of documentation
[PR 352](https://github.com/aai-institute/pyDVL/pull/352)
- Made ray an optional dependency, relying on joblib as default parallel backend
[PR 408](https://github.com/aai-institute/pyDVL/pull/408)
- Decoupled `ray.init` from `ParallelConfig`
[PR 373](https://github.com/aai-institute/pyDVL/pull/383)
- **Breaking Changes**
- Signature change: return information about Hessian inversion from
`compute_influence_factors`
[PR 375](https://github.com/aai-institute/pyDVL/pull/376)
- Major changes to IF interface and functionality. Foundation for a framework
abstraction for IF computation.
[PR 278](https://github.com/aai-institute/pyDVL/pull/278)
[PR 394](https://github.com/aai-institute/pyDVL/pull/394)
- Renamed `semivalues` to `compute_generic_semivalues`
[PR 413](https://github.com/aai-institute/pyDVL/pull/413)
- New `joblib` backend as default instead of ray. Simplify MapReduceJob.
[PR 355](https://github.com/aai-institute/pyDVL/pull/355)
- Bump torch dependency for influence package to 2.0
[PR 365](https://github.com/aai-institute/pyDVL/pull/365)

Fixed

- Fixes to parallel computation of generic semi-values: properly handle all
samplers and stopping criteria, irrespective of parallel backend.
[PR 372](https://github.com/aai-institute/pyDVL/pull/372)
- Optimises memory usage in IF calculation
[PR 375](https://github.com/aai-institute/pyDVL/pull/376)
- Fix adding valuation results with overlapping indices and different lengths
[PR 370](https://github.com/aai-institute/pyDVL/pull/370)
- Fixed bugs in conjugate gradient and `linear_solve`
[PR 358](https://github.com/aai-institute/pyDVL/pull/358)
- Fix installation of dev requirements for Python3.10
[PR 382](https://github.com/aai-institute/pyDVL/pull/382)
- Improvements to IF documentation
[PR 371](https://github.com/aai-institute/pyDVL/pull/371)