Tensorflow-data-validation

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* TFDV is now tested against macOS 12.5 (Monterey).

Known Issues

* N/A

Breaking Changes

* Depends on `tensorflow>=2.11,<3`
* Depends on `tfx-bsl>=1.12.0,<1.13.0`.
* Depends on `tensorflow-metadata>=1.12.0,<1.13.0`.

Deprecations

* N/A

Major Features and Improvements

* This is the last version that supports TensorFlow 1.15.x. TF 1.15.x support
will be removed in the next version. Please check the
[TF2 migration guide](https://www.tensorflow.org/guide/migrate) to migrate
to TF2.

* Add a `custom_validate_statistics` function to the validation API, and
support passing custom validations to `validate_statistics`. Note that
custom validation is not supported on Windows.

Bug Fixes and Other Changes

* Fix bug in implementation of `semantic_domain_stats_sample_rate`.

* Add beam metrics on string length

* Determine whether to calculate string statistics based on the
`is_categorical` field in the schema string domain.

* Histograms counts should now be more accurate for distributions with few
distinct values, or frequent individual values.

* Nested list length histogram counts are no longer based on the number of
values one up in the nested list hierarchy.

* Support using jensen-shannon divergence to detect drift and skew for string
and categorical features.

* `get_drift_skew_dataframe` now includes a `threshold` column.

* Adds support for NormalizedAbsoluteDifference comparator.

* Depends on `tensorflow>=1.15.5,<2` or `tensorflow>=2.10,<3`
* Depends on `joblib>=1.2.0`.

Known Issues

* N/A

Breaking Changes

* Histogram semantics are slightly changed, so that buckets include their
upper bound instead of their lower bound. STANDARD histograms will no longer
generate buckets that contain infinite and finite endpoints together.
* Introduces StatsOptions.use_sketch_based_topk_uniques replacing
experimental_use_sketch_based_topk_uniques. The latter option can still be
written, but not read.

Deprecations

* N/A

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Skew pipeline supports counting pairs of feature values in base/test.
* Depends on `apache-beam[gcp]>=2.40,<3`.
* Depends on `pyarrow>=6,<7`.
* Depends on `tfx-bsl>=1.10.1,<1.11.0`.
* Depends on `tensorflow-metadata>=1.10.0,<1.11.0`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `tensorflow>=1.15.5,<2` or `tensorflow>=2.9,<3`
* Depends on `tfx-bsl>=1.9.0,<1.10.0`.
* Depends on `tensorflow-metadata>=1.9.0,<1.10.0`.

Known Issues

* N/A

Breaking Changes

* Some fields in feature skew results proto changed names to be more generic.
* Removes the unused skew_config.proto

Deprecations

* N/A

Major Features and Improvements

* From this version we will be releasing python 3.9 wheels.

Bug Fixes and Other Changes

* Adds `get_statistics_html` to the public API.
* Fixes several incorrect type annotations.
* Schema inference handles derived features.
* `StatsOptions.to_json` now raises an error if it encounters unsupported
options.
* Depends on `apache-beam[gcp]>=2.38,<3`.
* Depends on `tensorflow-metadata>=1.8.0,<1.9.0`.
* Depends on `tfx-bsl>=1.8.0,<1.9.0`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* Adds the `DetectFeatureSkew` PTransform to the public API, which can be used
to detect feature skew between training and serving examples.
* Uses sketch-based top-k/uniques in TFDV inmemory mode.

Bug Fixes and Other Changes

* Fixes a bug in load_statistics that would cause failure when reading binary
protos.
* Depends on `pyfarmhash>=0.2,<0.4`.
* Depends on
`tensorflow>=1.15.5,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3`.
* Depends on `tensorflow-metadata>=1.7.0,<1.8.0`.
* Depends on `tfx-bsl>=1.7.0,<1.8.0`.
* Depends on `apache-beam[gcp]>=2.36,<3`.
* Updated the documentation for CombinerStatsGenerator to clarify that the
first accumulator passed to merge_accumulators may be modified.
* Added compression type detection when reading csv header.
* Detection of invalid utf8 strings now works regardless of relative frequency.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* N/A