Tensorflow-data-validation

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `apache-beam[gcp]>=2.25,!=2.26.*,<2.29`.

Known Issues

* N/A

Breaking changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* Added support for per-feature example weights which allows associating each
column its specific weight column. See the `per_feature_weight_override`
parameter in `StatsOptions.__init__`.

Bug Fixes and Other Changes

* Newly added LifecycleStage.DISABLED is now exempt from validation (similar
to LifecycleStage.DEPRECATED, etc).
* Fixed a bug where TFDV blindly trusts the claim type in the provided schema.
TFDV now computes the stats according to the actual type of the data, and
only when the actual type matches the claim in the schema will it compute
type-specific stats (e.g. categorical ints).
* Added an option to control whether to add default stats generators when
`tfdv.GenerateStatistics()`.
* Started using a new quantiles computation routine that does not depend on
TF. This could potentially increase the performance of TFDV under certain
workloads.
* Extending schema_util to support sematic domains.
* Moving natural_language_stats_generator to
natural_language_domain_inferring_stats_generator and creating a new
natural_language_stats_generator based on the fields of
natural_language_domain.
* Providing vocab_utils to assist in opening / loading vocabulary files.
* A SchemaDiff will be reported upon J-S skew/drift.
* Fixed a bug in FLOAT_TYPE_SMALL_FLOAT anomaly message.
* Depends on `apache-beam[gcp]>=2.25,!=2.26.*,<3`.
* Depends on `tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.4.*,<3`.
* Depends on `tensorflow-metadata>=0.26,<0.27`.
* Depends on `tensorflow-transform>=0.26,<0.27`.
* Depends on `tfx-bsl>=0.26,<0.27`.

Known Issues

* N/A

Breaking changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* Add support for detecting drift and distribution skew in numeric features.
* `tfdv.validate_statistics` now also reports the raw measurements of
distribution skew/drift (if any is done), regardless whether skew/drift is
detected. The report is in the `drift_skew_info` of the `Anomalies` proto
(return value of `validate_statistics`).
* From this release TFDV will also be hosting nightly packages on
https://pypi-nightly.tensorflow.org. To install the nightly package use the
following command:


pip install --extra-index-url https://pypi-nightly.tensorflow.org/simple tensorflow-data-validation


Note: These nightly packages are unstable and breakages are likely to
happen. The fix could often take a week or more depending on the complexity
involved for the wheels to be available on the PyPI cloud service. You can
always use the stable version of TFDV available on PyPI by running the
command `pip install tensorflow-data-validation` .

Bug Fixes and Other Changes

* Added `tfdv.load_stats_binary` to load stats what were written using
`tfdv.WriteStatisticsToText` (now `tfdv.WriteStatisticsToBinaryFile`).
* Anomalies previously (un)classified as UKNOWN_TYPE now trigger more specific
anomaly types: DOMAIN_INVALID_FOR_TYPE, UNEXPECTED_DATA_TYPE,
FEATURE_MISSING_NAME, FEATURE_MISSING_TYPE, INVALID_SCHEMA_SPECIFICATION
* Fixed a bug that `import tensorflow_data_validation` would fail if IPython
is not installed. IPython is an optional dependency of TFDV.
* Depends on `apache-beam[gcp]>=2.25,<3`.
* Depends on `tensorflow-metadata>=0.25,<0.26`.
* Depends on `tensorflow-transform>=0.25,<0.26`.
* Depends on `tfx-bsl>=0.25,<0.26`.
* Depends on `scikit-learn>=1.0,<2` (mutual-information installation).

Known Issues

* N/A

Breaking Changes

* `tfdv.WriteStatisticsToText` is renamed as
`tfdv.WriteStatisticsToBinaryFile`. The former is still available but will
be removed in a future release.

Deprecations

* N/A

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `apache-beam[gcp]>=2.24,<3`.
* Depends on `tensorflow-transform>=0.24.1,<0.25`.
* Depends on `tfx-bsl>=0.24.1,<0.25`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* You can now build the TFDV wheel with `python setup.py bdist_wheel`. Note:
* If you want to build a manylinux2010 wheel you'll still need
to use Docker.
* Bazel is still required.
* You can now build manylinux2010 TFDV wheel for Python 3.8.

Bug Fixes and Other Changes

* Support allowlist and denylist features in `tfdv.visualize_statistics`
method.
* Depends on `absl-py>=0.9,<0.11`.
* Depends on `pandas>=1.0,<2`.
* Depends on `protobuf>=3.9.2,<4`.
* Depends on `tensorflow-metadata>=0.24,<0.25`.
* Depends on `tensorflow-transform>=0.24,<0.25`.
* Depends on `tfx-bsl>=0.24,<0.25`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* Deprecated Py3.5 support.
* Deprecated `sample_count` option in `tfdv.StatsOptions`. Use `sample_rate`
option instead.

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `apache-beam[gcp]>=2.24,<3`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* Deprecating python 3.5 support.