Tensorflow-datasets

Added

- New Beam writer `NoShuffleBeamWriter` that doesn't shuffle, which speeds up
dataset generation significantly, but does not have deterministic order
guarantees. Can be enabled with the flag `--nondeterministic_order`.
- CroissantBuilder now supports Croissant files that define splits; and new
feature types: feature dictionaries and multidimensional arrays.
- New datasets.

Changed

Deprecated

Removed

Fixed

- Various small bug fixes.
- Various performance improvements.

Security

Added

- New datasets.

Changed

- `CroissantBuilder`'s API to generate TFDS datasets from Croissant files.

Deprecated

Removed

Fixed

- Versions for existing datasets.

Security

Added

- Full support for Python 3.12.

Added

- Support to download and prepare datasets using the
[Parquet](https://parquet.apache.org) data format.
python
builder = tfds.builder('fashion_mnist', file_format='parquet')
builder.download_and_prepare()
ds = builder.as_dataset(split='train')
print(next(iter(ds)))


- [`tfds.data_source`](https://www.tensorflow.org/datasets/api_docs/python/tfds/data_source)
is pickable, thus working smoothly with
[PyGrain](https://github.com/google/grain). Learn more by following the
[tutorial](https://www.tensorflow.org/datasets/data_source).

- TFDS plays nicely with
[Croissant](https://mlcommons.org/working-groups/croissant). Learn more by
following the
[recipe](https://colab.research.google.com/github/mlcommons/croissant/blob/main/python/mlcroissant/recipes/tfds_croissant_builder.ipynb).

Changed

Deprecated

Removed

Fixed

Security

Added

- A new [CroissantBuilder](https://www.tensorflow.org/datasets/format_specific_dataset_builders#croissantbuilder)
which initializes a DatasetBuilder based on a [Croissant](https://github.com/mlcommons/croissant)
metadata file.
- New conversion options between different bounding boxes formats.
- Better support for `HuggingfaceDatasetBuilder`.
- A [script](https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/scripts/convert_format.py)
to convert a dataset from one format to another.

Changed

Deprecated

- Python 3.9 support. TFDS now uses Python 3.10

Removed

Fixed

Security

Added

- [Segment Anything](https://ai.facebook.com/datasets/segment-anything-downloads)
(SA-1B) dataset.

Changed

- Hugging Face datasets accept `None` values for any features. TFDS has no
`tfds.features.Optional`, so `None` values are converted to default values.
Those default values used to be `0` and `0.0` for int and float. Now, it's
`-inf` as defined by NumPy (e.g., `np.iinfo(np.int32).min` or
`np.finfo(np.float32).min`). This avoids ambiguous values when `0` and `0.0`
exist in the values of the dataset. The roadmap is to implement
`tfds.features.Optional`.

Deprecated

- Python 3.8 support. As per
[NEP 29](https://numpy.org/neps/nep-0029-deprecation_policy.html), TFDS now
uses Python>=3.9.

Removed

Fixed

Security