Release 0.40.0

Major Features and Improvements

• Skip model updates that are non-finite in
  tff.learning.templates.build_apply_optimizer_finalizer.

Breaking Changes

• Removed deprecated APIs in tff.learning.framework
• Update the Python package scripts to use Python 3.10 by default.
• Remove module wildcard imports from init.py files in TFF.
• Update the Python package scripts to use Python 3.10 by default.

Bug Fixes

• Remove functools.wraps within tff.tf_computation.
• Fix typo in iNaturalist dataset docstring.

Release 0.39.0

Major Features and Improvements

• Added tff.learning.models.FunctionModel support to all methods in
  tff.learning.algorithms.
• Added support for tf.data.DataSpec to tff.types.infer_unplaced_type.
• Use a tensorflow::ThreadPool for parallelism inside the C++
  TensorFlowExecutor.
• Introduced a new tff.experimental_tf_fn_computation tracing decorator that
  uses FunctionDef instead of GraphDef tracing, providing tf.function
  auto-control-dependencies.
• Renamed number_of_clients to num_clients in the federated program API.
• Replaced the following API with composers API in tff.learning.templates.
  • tff.learning.framework.build_model_delta_optimizer_process
  • tff.learning.framework.ClientDeltaFn

Bug Fixes

• Fixed a bug in the “Client-efficient large-model federated learning”
  tutorial to use the correct dense shape.

Release 0.38.0

Major Features and Improvements

• Added tff.learning.models.FunctionalModel support to
  tff.learning.algorithms.build_mime_lite.
• Updated tensorflow-privacy to version 0.8.6.
• Added an abstract interface describing an asynchronous context
• Removed references to tff.framework.Context.
• Added tff.simulation.datasets.gldv2.get_synthetic.
• Added prefetching data source in tff.program.PrefetchingDataSource.

Breaking Changes

• Deleted deprecated
  tff.learning.framework.build_encoded_broadcast_process_from_model.
• Deprecated tff.learning.ModelWeights and alias
  tff.learning.framework.ModelWeights, has now moved to
  tff.learning.models.ModelWeights. Code should be updated before the next
  release.

Bug Fixes

• Fixed a bug with variable creation order of metrics in
  tff.learning.models.functional_model_from_keras.
• Improved tff.tf_computation tracing to also trace functools.partial
  objects.

Known Bugs

• Colab compatibility: TFF requires Python 3.9 while Colab runtime uses Python
  3.7.

Release 0.37.0

Major Features and Improvements

• Added support for Python 3.10.
• Improved support for numpy values in the tff.program API.
• Increased dataset serialization size limit to 100MB.
• Added a new method tff.learning.ModelWeights.convert_variables_to_arrays.
• Added new metrics aggregation factories under tff.learning.metrics.
• Parallelized aggregation in tff.framework.ComposingExecutorFactory.

Breaking Changes

• Updated to use jax and jaxlib version 0.3.14.
• Renamed tff.program.CoroValueReference to
  tff.program.AwaitableValueReference to reflect the relaxed contract.

Bug Fixes

• Improved documentation for tff.simulation.build_uniform_sampling_fn,
  tff.learning.robust_aggregator,
  tff.aggregators.PrivateQuantileEstimationProcess.
• Fixed documentation bug for tutorial “High-performance Simulation with
  Kubernetes”.
• Fixed bug where momentum hyperparameters were added to SGDM optimizer when
  momentum was set to 0.
• Removed assertion that preprocessed datasets in a
  tff.simulation.baselines.BaselineTask have the same element structure.
• Fixed a memory leak when moving numpy arrays across the Python and C++
  boundary in the C++ executor.
• Fixed bug in the federated program API when using multiple release managers
  to release the same value.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:
Madhava Jay, nbishdev@

Release 0.36.0

Major Features and Improvements

• Added support for tff.learning.models.FunctionalModel to
  tff.learning.algorithms.build_fed_sgd and
  tff.learning.algorithms.build_fed_prox.
• Increased the gRPC message limit from 1 GB to 2 GB.
• Added hyperparameter getters/setters to various components in tff.learning.

Breaking Changes

• Updated tensorflow to version 2.10.

Bug Fixes

• Improved documentation for
  tff.analytics.heavy_hitters.iblt.build_iblt_computation().
• Fixed incorrect docstring of tff.federated_select.
• Fixed typo in federated program example.

Release 0.35.0

Major Features and Improvements

• Added get/set_hparams methods to tff.learning.templates.ClientWorkProcess.
• Added tff.learning.algorithms.build_mime_lite_with_optimizer_schedule.
• Updated tensorflow-privacy to version 0.8.5.
• Added tff.learning.entropy_compression_aggregator.
• Added tff.aggregators.EliasGammaEncodedSumFactory.
• Added tff.program.ClientIdDataSource and
  tff.program.ClientIdDataSourceIterator, for working with a data source of
  ids of federated clients.

Breaking Changes

• Removed prototype IREE backend.
• Added new dependency on TensorFlow Compression.

Bug Fixes

• Fixed implementation of the loading_remote_data tutorial.
• Corrected the docstring of
  tff.simulation.datasets.stackoverflow.get_synthetic.

Known Bugs

• TFF's Python 3.9 typing conflicts with Colab's Python 3.7 runtime.

Release 0.34.0

Major Features and Improvements

• Updated to use Bazel version 5.3.0.
• Updated the conventions used to specify the version of a Python dependency,
  see https://github.com/tensorflow/federated/blob/main/requirements.txt for
  more information.
• Updated the setup.py to explicitly fail to pip install in Python 3.10.
  This has always been failing at runtime, but now explicitly fails to install
  using pip.
• Refreshed loading_remote_data notebook content and added content for
  FederatedDataSource.
• Added a TFF type_signature attribute to objects of type MapReduceForm.
• Added a series of slides to the GitHub repo (so not part of the PIP package) which detail a technical
  deep dive into TFF.

Breaking Changes

• Bumped tf-privacy version to 0.8.4.
• Bumped tf-model-optimization version to 0.8.3.
• Removed initialize from MapReduceForm.
• SequenceType now automatically casts any StructWithPythonType that
  contains a list to a tuple for tf.data compatibility.
• Unified the model_fn and model parameters of
  tff.learning.algorithms.build_weighted_fed_avg.
• MapReduceForm now takes a type_signature argument in its constructor,
  and no longer takes an initialize argument.
• MapReduceForm no longer contains an initialize attribute.

Bug Fixes

• Relaxed overly strict type equivalence check to assignability in TFF-TF code
  generation.

Release 0.33.0

Major Features and Improvements

• Extend tff.analytics.heavy_hitters.iblt with create_chunker API for
  encoding non-Unicode strings.
• Extend tff.aggregators.DifferentiallyPrivateFactory.tree_aggregation with
  an optional record_aggregation_factory argument.

Breaking Changes

• Replaced ModularClippingSumFactory with SecureModularSumFactory in
  tff.analytics.build_hierarchical_histogram_process.

Known Bugs

• TFF's python 3.9 typing conflicts with Colab's Python 3.7 support.

Release 0.32.0

Major Features and Improvements

• Add a MimeLite implementation that allows from optimizer learning rate
  scheduling in
  tff.learning.algorithms.build_mime_lite_with_optimizer_schedule.

Breaking Changes

• None

Bug Fixes

• None

Known Bugs

• TFF's python 3.9 typing conflicts with Colab's Python 3.7 support.

Release 0.31.0

Major Features and Improvements

• Added ReleaseManagers to make authoring program logic more convenient.
• Updated TFFs attrs dependency to version 21.4.0.
• Update TFFs tensorflow-privacy dependency to version 0.8.1.

Breaking Changes

• Changed tff.learning.BatchOutput from an attrs class to a namedtuple.
• Removed unused tff.learning.framework.parameter_count_from_model API.

Release 0.30.0

Major Features and Improvements

• Add tests for namedtuples in the tff.program package.
• Add num_subrounds parameter to the mergeable context, allowing callers to
  optionally sequentialize subrounds.
• Add metrics support to tff.learning.models.FunctionalModel, including
  updates to the helper function create_functional_metric_fns and the
  downstream caller tff.learning.algorithms.build_weighted_fed_avg.

Bug Fixes

• Fix typo in the types constructed for testing the tff.program package.
• Fix some program example typos.
• Fix tests that don't seem to be running under the CI.
• Fix naming bug for Python mergeable execution.
• Ensure exceptions raised from remote executor stub implement gRPC error
  interface.
• Update tff.structure.Struct integration with JAX pytrees to not flatten
  the entire structure.
• Use Python 3.7 compatible type annotations until Colab updates to Python
  3.9.

Release 0.29.0

Major Features and Improvements

• Update the MemoryReleaseManager to save type_signature when releasing
  values.
• Add a type_signature parameter to the ReleaseManager.release method.
• Unify retryability logic between TFF-C++ and TFF-Python.
• Update the TFF contributions and collaboration links to point to the Discord
  server.

Breaking Changes

• Move Python executor stacks file to python_executor_stacks.py in
  executor_stacks directory.

Bug Fixes

• Ensure that dataset deserialization will return ragged and sparse tensors,
  as needed according to the TFF type of the dataset.
• Make metric_finalizers use metric constructors if available.

Release 0.28.0

Major Features and Improvements

• Updated tutorials to use tff.learning.algorithms API.
• Asynchronous TFF execution contexts no longer assume a single global
  cardinality; they concurrently invoke any computation for which concurrency
  is requested.

Breaking Changes

• Removed tff.learning.build_federated_averaging_process; users should
  migrate to tff.learning.algorithms.build_weighted_fed_avg.

Bug Fixes

• Clarified semantics for TFF-C++ multimachine Dispose, DisposeExecutor,
  and executor keying, to avoid raising exceptions and spamming logs in the
  course of normal operation.
• Fixed unsigned integer overflow for TFF-C++
  max_concurrent_computation_calls.
• Normalizes on call-dominant form before attempting to compile to
  MergeableCompForm, removing spurious failures on dependent-aggregation
  checking.

Known Bugs

• Serialization / deserialization of tf.data.Datasets yielding non-dense
  tensors for multimachine runtime may encounter issues:
  • tff.framework.deserialize_value may fail to deserialize
    tf.data.Datasets yielding RaggedTensors or SparseTensors.
  • tff.framework.serialize_value may fail to serialize tf.data.Datasets
    yielding SparseTensors.

Release 0.27.0

Major Features and Improvements

• New Colab notebook illustrating how to use DataBackend to load remote datasets.
• Added a CreateDataDescriptor helper function.
• Added a worker binary serving the TFF-C++ executor service.

Bug Fixes

• Fixed bug with intermediate aggregation and controller failures, causing hangs.

Release 0.26.0

Major Features and Improvements

• Updated TensorFlow to 2.9.1.
• Update pybind11 to 2.9.2.
• Re-enable cpp_fast_protos.
• Introduces container class to run coroutines in a dedicated thread, allowing
  TFF’s synchronous execution interfaces to be used in conjunction with other
  asyncio code.
• Use latest TFF version in Colab notebook links.
• Rename the helper functions that create test MeasuredProcesses.
• Add a compiler transform checking Tensorflow computations against list of
  allowed ops.
• Explicitly specify return types in the program package.
• Adds convenience function for setting a local async CPP execution context.
• Move jax components into a non-experimental namespace.

Breaking Changes

• Switch compilation flag _GLIBCXX_USE_CXX11_ABI to 1.

Release 0.25.0

Major Features and Improvements

• Adds error message logging to TFF C++ execution context.
• Adds test coverage for C++ runtime with aggregators.
• Redefines 'workers going down with fixed clients per round' test.
• Add complete examples of using DataBackend with TFF comps.
• Updated the MapReduceForm documentation to include the two additional secure
  sum intrinsics.
• tff.learning
  • Relax the type check on LearningProcess from strictly SequenceType to
    also allow structures of SequenceType.

Breaking Changes

• Remove usage of tff.test.TestCase, tff.test.main(), and delete
  test_case module.
• Update test utility docstrings to use consistent vocabulary.
• Update to TensorFlow 2.9.0
• Rename up compiler/test_utils to compiler/building_block_test_utils.
• Remove some unnecessary usage of pytype: skip-file.
• Specify the None return type of ReleaseManager.release.
• Remove usage of deprecated numpy types.
• Replace depreciated random_integers with randint.

Bug Fixes

• Fix numpy warning.

Release 0.24.0

Major Features and Improvements

• Added asyncio.run call to metrics manager release calls to ensure
  compatibility with https://github.com/tensorflow/federated/commit/a98b5ed6894c536549da06b4cc7ed116105dfe65.
• Added an example and documentation for the Federated Program API.
• Improved model_update_aggregator to support structures with mixed floating dtypes.
• Create a mirror of tff.simulation.compose_dataset_computation_with_iterative_process for tff.learning.templates.LearningProcess.
• Added logging of expected sequential computations to local TFF-C++ runtime.

Breaking Changes

• Moved asserts from tff.test.TestCase to tff.test.* as functions.
• Removed assert_type_assignable_from function.
• Moved assert_nested_struct_eq to the type_conversions_test module.
• Removed client_train_process and fedavg_ds_loop comparisons.

Bug Fixes

• Fixed comparisons to enums in the benchmarks package.
• Fixed async_utils.SharedAwaitable exception raiser.
• Fixed various lint errors.

Release 0.23.0

Major Features and Improvements

• Deprecated tff.learning.build_federated_averaging_process.
• Added an API to convert tf.keras.metrics.Metric to a set of pure
  tf.functions.

Breaking Changes

• Renamed ProgramStateManager.version to ProgramStateManager.get_versions.

Bug Fixes

• Fixed the "datasets/" path in the working with TFF's ClientData tutorial.

Release 0.22.0

Major Features and Improvements

• Updated .bazelversion to 5.1.1.
• Updated the tff.program API to use asyncio.
• Exposed new APIs in the tff.framework package:
  • tff.framework.CardinalitiesType.
  • tff.framework.PlacementLiteral.
  • tff.framework.merge_cardinalities.
• tff.analytics
  • Added new analytic_gauss_stddev API.

Breaking Changes

• Renamed ProgramStateManager.version to ProgramStateManager.get_versions.

Bug Fixes

• Fixed some Python lint errors related to linting Python 3.9.
• Cleaned up stale TODOs throughout the codebase.

Known Bugs

• Version 0.21.0 currently fails to import in
  colab if the version of Python is less
  than Python 3.9. Please use a runtime with a version of Python greater than
  Python 3.9 or use TFF version 0.20.0.

Release 0.21.0

Major Features and Improvements

• tff.analytics
  • Added new tff.analytics.IbltFactory aggregation factory.
  • Added new IBTL tensor encoder/decoder libraries and uses them in
    tff.analytics.heavy_hitters.iblt.build_iblt_computation.
• tff.aggregator
  • Added as_weighted_aggregator to the tff.aggregator.Factory API.
• tff.learning
  • Improved compilation and execution performance of
    tff.learning.metrics.secure_sum_then_finalize by grouping tensors by
    DType.
  • Added set_model_weights method and default implementation to
    tff.learning.templates.LearningProcess.
  • Added a new reset_metrics attribute to tff.learning.Model.
  • Added schedule_learning_rate to tff.learning.optimizers.
  • Added new tff.learning.ddp_secure_aggregator for Distributed
    Differential Privacy.
• tff.simulation
  • Added an option to distort train images in the CIFAR-100 baseline task.
  • Changed the default sequence length for the Shakespeare baseline task to
    a more reasonable value.
• Core
  • Switched runtime to create new RemoteExecutors with different
    cardinalities, rather than resetting the cardinality in the remote
    service.

Breaking Changes

• Removed support for Python 3.7 and 3.8, TFF supports 3.9 and later.
• Removed deprecated attributes report_local_outputs and
  federated_output_computation from tff.learning.Model
• Removed the ingest method from tff.Context

Bug Fixes

• Multiple typos in tests, code comments, and pydoc.

Known Bugs

• Sequences (datasets) of SparseTensors don't work on the C++ runtime.
• Computations when CLIENTS cardinality is zero doesn't work on the Python
  runtime.
• Assigning variables to a Keras model after construction inside a model_fn
  results in a non-deterministic graph.