Tensorflow-federated

Major Features and Improvements

* `tff.analytics`
* Added new `tff.analytics.IbltFactory` aggregation factory.
* Added new IBTL tensor encoder/decoder libraries and uses them in
`tff.analytics.heavy_hitters.iblt.build_iblt_computation`.
* `tff.aggregator`
* Added `as_weighted_aggregator` to the `tff.aggregator.Factory` API.
* `tff.learning`
* Improved compilation and execution performance of
`tff.learning.metrics.secure_sum_then_finalize` by grouping tensors by
DType.
* Added `set_model_weights` method and default implementation to
`tff.learning.templates.LearningProcess`.
* Added a new `reset_metrics` attribute to `tff.learning.Model`.
* Added `schedule_learning_rate` to `tff.learning.optimizers`.
* Added new `tff.learning.ddp_secure_aggregator` for Distributed
Differential Privacy.
* `tff.simulation`
* Added an option to distort train images in the CIFAR-100 baseline task.
* Changed the default sequence length for the Shakespeare baseline task to
a more reasonable value.
* Core
* Switched runtime to create new RemoteExecutors with different
cardinalities, rather than resetting the cardinality in the remote
service.

Breaking Changes

* Removed support for Python 3.7 and 3.8, TFF supports 3.9 and later.
* Removed deprecated attributes `report_local_outputs` and
`federated_output_computation` from `tff.learning.Model`
* Removed the `ingest` method from `tff.Context`

Bug Fixes

* Multiple typos in tests, code comments, and pydoc.

Known Bugs

* Sequences (datasets) of SparseTensors don't work on the C++ runtime.
* Computations when `CLIENTS` cardinality is zero doesn't work on the Python
runtime.
* Assigning variables to a Keras model after construction inside a `model_fn`
results in a non-deterministic graph.

Major Features and Improvements

* Added `tff.program` API; this API is still in active development but can be
used to compose shared and platform specific: program logic, components, and
privacy concepts to create federated programs.
* Added support for Python 3.9.
* Added CelebA and iNaturalist datasets to `tff.simulation.datasets`.
* Added `tff.analytics` API for federated analytics, including private heavy
hitters algorithms.
* Added `tff.learning.algorithms` API, including TFF implementations of
FedProx, FedAvg with learning rate scheduling, federated k-Means, and
MimeLite.
* Added `tff.learning.metrics` API to support easy configuration of
cross-client metrics aggregation via the new `metrics_aggregator` argument.
* Added `metrics_aggregator` argument to
`tff.learning.build_federated_averaging_process` and
`tff.learning.build_federated_evaluation`.
* Added `report_local_unfinalized_metrics` and `metric_finalizers` methods to
`tff.learning.Model` and deprecated `report_local_outputs` and
`federated_output_computation`.
* Added `tff.learning.optimizers` API for building purely functional
optimizers and implementations of SGD, Adagrad, Rmsprop, Adam, Yogi,
* Added `tff.learning.reconstruction` API for building partially local
federated learning algorithms, including Federated Reconstruction.
* Added `tff.learning.templates` API to support building learning algorithms
in a modular fashion.
* Added `tff.simulation.baselines` API to support evaluating learning
algorithms on a suite of representative tasks.
* Added `tff.aggregators.DifferentiallyPrivateFactory.tree_aggregation` to
support the [DP-FTRL algorithm](https://arxiv.org/abs/2103.00039).
* Added `tff.aggregators.SecureModularSumFactory`
* Added `tff.aggregators.DiscreteFourierTransformFactory` and
`tff.aggregators.HadamardTransformFactory` to support rotation-based
aggregators.
* Added `tff.aggregators.concat_factory` for aggregating structures as a
single tensor.
* Added `tff.backends.native.create_mergeable_comp_execution_context`,
`tff.backends.native.set_mergeable_comp_execution_context`; these can be
used with a distributed runtime to scale to tens of thousands of clients.
* Improved performance of many `tff.simulation.datasets.ClientData`
subclasses.
* Added `tff.simulation.datasets.ClientData.serializable_dataset_fn`
attribute, enabling dataset creation within TF/TFF computations.
* Added `debug_measurements` option to aggregators in `tff.learning`.
* Added support for unambiguous zero-client aggregations.
* Added support for Python dataclasses as function parameters and return
values for TFF computations.
* Added automatic insertion of `tff.federated_zip` to invocation of
user-defined TFF federated computations.
* Added utilities to `tff.simulation.datasets` for saving federated datasets
to a SQL database compatible with `tff.simulation.datasets.SqlClientData`.
* Added `tff.learning.models.FunctionalModel` and
`tff.learning.models.functional_model_from_keras`.
* Increased max flow of tensors. Tensors now flow here, there, and everywhere.
* Updated the Python dependencies:
* Updated `absl-py` to version `1.0.0`.
* Updated `attrs` to version `21.2.0`.
* Added `farmhashpy` version `0.4.0`.
* Updated `jax` to version `0.2.27`.
* Updated `jaxlib` to version `0.1.76`.
* Updated `numpy` to version `1.21.4`.
* Removed `retrying`.
* Updated `tensorflow-model-optimization` to version `0.7.1`.
* Updated `tensorflow-model-optimization` to version `0.7.3`.
* Updated `tensorflow` to version `2.8.0`.
* Added support for building many dependencies including `tensorflow` using
Bazel.
* Updated the Bazel dependencies:
* Updated `rules_python` to version `0.5.0`.
* Updated `com_google_protobuf` to version `v3.18.0-rc1`.
* Added `absl_py` version `1.0.0`.
* Added `com_google_googletest` version `release-1.11.0`.
* Added `io_bazel_rules_go` version `v0.29.0`.
* Added `bazel_skylib` version `1.0.3`.
* Added `pybind11_abseil`.
* Added `pybind11_bazel`.
* Added `pybind11_protobuf`.
* Added `com_google_absl` version `20211102.0`.
* Added `tensorflow_org` version `v2.8.0`.

Breaking Changes

* Removed support for building source on MacOS.
* Removed support for Python 3.6.
* Removed symbol `tff.framework.type_contains`, use `tff.types.contains`
instead.
* Removed many symbols from `tff.simulation`, these can be found in
`tff.program` instead.
* Removed support for converting non-OrderedDict mapping types to
`tff.Value`s.
* Removed `tff.simulation.datasets.ClientData.from_clients_and_fn` in favor of
`tff.simulation.datasets.ClientData.from_clients_and_tf_fn`.
* Restricted `tff.simulation.datasets.ClientData.preprocess` to only support
TF-serializable functions.
* Removed `tff.backends.reference`, and the reference context it contained.
* Removed `tff.learning.build_federated_sgd_process` in favor of
`tff.learning.algorithms.build_fed_sgd`.
* Removed `tff.simulation.run_simulation` in favor of
`tff.simulation.run_training_process`.
* Removed `tff.learning.framework.EnhancedModel`.
* Removed `tff.learning.framework.build_stateless_mean`.

Bug Fixes

* Fixed broken links in documentation.
* Fixed many pytype errors.
* Fixed some inconsistencies in Bazel visibility.
* Fixed bug where `tff.simulation.datasets.gldv2.load_data()` would result in
an error.

Major Features and Improvements

* Introduced new intrinsics: `federated_select` and `federated_secure_select`.
* New `tff.structure_from_tensor_type_tree` to help manipulate structures of
`tff.TensorType` into structures of values.
* Many new `tff.aggregators` factory implementations.
* Introduced `tf.data` concept for data URIs.
* New `tff.type` package with utilities for working with `tff.Type` values.
* Initial experimental support for `tff.jax_computation`.
* Extend `tff.tf_computation` support to `SpareTensor` and `RaggedTensor`.

Breaking Changes

* Update gRPC dependency to 1.34.
* Moved `ClientData` interface and implementations to
`tff.simulation.datasets`.
* Renamed `tff.utils.update_state` to `tff.structure.update_struct`.
* Removed the `tff.utils` namespace, all symbols have migrated, many to
`tff.aggregators`.
* Moved infinite EMNIST dataset to federated research repository.
* Removes `rpc_mode` argument to remote executors, along with streaming mode.
* Removes deprecated `tff.federated_apply`.
* Removes `tff.federated_reduce`, all usages can use
`tff.federated_aggregate`.
* Removes `HDF5ClientData` and `h5py` pip dependency.
* Removes `setattr` functionality on `tff.ValueImpl`.

Bug Fixes

* Improved `tf.GraphDef` comparisons.
* Force close generators used for sending functions to computation wrappers,
avoiding race conditions in Colab.
* Fix tracing libraries asyncio usage to be Python3.9 compatible.
* Fix issue with destruction of type intern pool destructing and `abc`.
* Fix type interning for tensors with unknown dimensions.
* Fix `ClientData.create_dataset_from_all_clients` consuming unreasonable
amounts of memory/compute time.

Major Features and Improvements

* Extended the `tff.simulation` package to add many new tools for running
simulations (checkpoints and metrics managers, client sampling functions).
* Extended the `tff.aggregators` package with a number of new aggregation
factories.
* Added the `tff.structure` API to expose the `Struct` class and related
functions.
* Added the `tff.profiler` API to expose useful profiling related functions.
* Added the `tff.backends.test` package to expose backends that focused on
testing specifically a way to test a computation with a
`federated_secure_sum` intrinsic.
* Added the `tff.experimental` package to expose less stable API.

Breaking Changes

* Replaced the `tff.aggregators.AggregationProcessFactory` abstract base class
with the `tff.aggregators.UnweightedAggregationFactory` and the
`tff.aggregators.WeightedAggregationFactory` classes.
* Replaced the `tff.aggregators.ZeroingFactory` class with a
`tff.aggregators.zeroing_factory` function with the same input arguments.
* Replaced the `tff.aggregators.ClippingFactory` class with a
`tff.aggregators.clipping_factory` function with the same input arguments.
* Updated `tensorflow` package dependency to `2.4.0`.
* Updated `absl-py` package dependency to `0.10`.
* Updated `grpcio` package dependency to `1.32.0`.
* Added a `jaxlib` package dependency at `0.1.55`.
* Updated `numpy` package dependency to `1.19.2`.
* Updated `tensorflow-addons` package dependency to `0.12.0`.
* Updated `tensorflow-model-optimization` package dependency to `0.5.0`.

Bug Fixes

* Fixed issue with the `sequence_reduce` intrinsic handling federated types.

Major Features and Improvements

* New `tff.aggregators` package with interfaces for stateful aggregation
compositions.
* New Google Landmark Dataset `tff.simulations.dataset.gldv2`
* New convenience APIs `tff.type_clients` and `tff.type_at_server`
* Invert control of computation tracing methods to produce clearer Python
stack traces on error.
* Move executor creation to a factory pattern in executor service, allowing
distributed runtimes to be agnostic to number of clients.
* Significant improvements of type serialization/deserialization
* New `tff.simulations.compose_dataset_computation_with_iterative_process` API
to move execution of client dataset construction to executor stack leaves.
* Extend parameterization of `tff.learning.build_federated_averaging_process`
with `use_experimental_simulation_loop` argument to better utilize multi-GPU
setups.

Breaking Changes

* Removed `tff.utils.StatefulFn`, replaced by `tff.templates.MeasuredProcess`.
* Removed `tff.learning.assign_weights_to_keras_model`
* Stop removing `OptimizeDataset` ops from `tff.tf_computation`s.
* The `research/` directory has been moved to
http://github.com/google-research/federated.
* Updates to `input_spec` argument for `tff.learning.from_keras_model`.
* Updated TensorFlow dependency to `2.3.0`.
* Updated TensorFlow Model Optimization dependency to `0.4.0`.

Bug Fixes

* Fixed streaming mode hang in remote executor.
* Wrap `collections.namedtuple._asdict` calls in `collections.OrderedDict` to
support Python 3.8.
* Correctly serialize/deserialize `tff.TensorType` with unknown shapes.
* Cleanup TF lookup HashTable resources in TFF execution.
* Fix bug in Shakespeare dataset where OOV and last vocab character were the
same.
* Fix TFF ingestion of Keras models with shared embeddings.
* Closed hole in compilation to CanonicalForm.

Known Bugs

* "Federated Learning for Image Classification" tutorial fails to load
`projector` plugin for tensorboard.
(https://github.com/tensorflow/federated/issues/914)
* Certain Keras models with activity regularization fail in execution with
unliftable error (https://github.com/tensorflow/federated/issues/913).

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

amitport, ronaldseoh

Bug Fixes

* Fixed issue preventing Python `list`s from being `all_equal` values.