Known Caveats

• GPflow requires TensorFlow >= 2.2.

Deprecations

• The gpflow.utilities.utilities submodule has been deprecated and will be removed in GPflow 2.3. User code should access functions directly through gpflow.utilities instead (#1650).

Major Features and Improvements

• Improves compatibility between monitoring API and Scipy optimizer (#1642).
• Adds _add_noise_cov method to GPR model class to make it more easily extensible (#1645).

Bug Fixes

• Fixes a bug in ModelToTensorBoard (#1619) when max_size=-1 (#1619)

• Fixes a dynamic shape issue in the quadrature code (#1626).

• Fixes #1651, a bug in fully_correlated_conditional_repeat (#1652).

• Fixes #1653, a bug in the "fallback" code path for multioutput Kuf (#1654).

• Fixes a bug in the un-whitened code path for the fully correlated conditional function (#1662).

• Fixes a bug in independent_interdomain_conditional (#1663).

• Fixes an issue with the gpflow.config API documentation (#1664).

• Test suite

  • Fixes the test suite for TensorFlow 2.4 / TFP 0.12 (#1625).
  • Fixes mypy call (#1637).
  • Fixes a bug in test_method_equivalence.py (#1649).

Thanks to our Contributors

This release contains contributions from:

johnamcleod, st--, vatsalaggarwal, sam-willis, vdutor

Improvements

• Replace len(inducing_variable) with inducing_variable.num inducing property (#1594).
  Adds support for inducing variables with dynamically changing shape (compatibility with tf.function).
  (Resolves #1578.)

• HeteroskedasticTFPConditional should construct tensors at class-construction, not at module-import time (#1598).

Improvements

• kullback_leiblers.gauss_kl: when running TensorFlow >= 2.2, we avoid a tf.tile() that was a workaround for broadcasting issues in TensorFlow <= 2.1; the workaround is now only executed when running in TensorFlow 2.1 (#1585).

Bugfixes

• model.predict_y/predict_log_density now explicitly raise NotImplementedError when passed full_(output_)cov=True (#1582). Actual implementation of the full_(output_)cov=True case remains an open issue (#1461).
• bugfix for conditionals.multioutput.util.independent_interdomain_conditional (#1583).

Docs and infrastructure

• CI/infrastructure updates (#1580, #1581, #1587, #1592).
• heteroskedastic notebook cleaned up, links added (#1579).
• intro_to_gpflow2 notebook updated to account for saved_model now working (#1586).

New features and improvements

• Likelihoods now use the new Quadrature class (#1566, #1571)
• Multi-latent likelihoods for heteroskedastic models (#1559)
• isort for consistent imports (#1572)

Bugfixes

• Fix-up to our workaround for https://github.com/tensorflow/probability/issues/547 for tensorflow-probability==0.11.1 (#1574)
• Add missing packaging dependency (#1568)

Deprecations

• You should no longer use likelihood.num_gauss_hermite_points. Interact with its quadrature attribute instead.

This patch release restores full compatibility with TensorFlow 2.3.

Known installation issue: if import gpflow gives you an ImportError, please install the "packaging" package manually (pip install packaging). Fixed by #1568, which will be included in the next patch release (GPflow 2.1.2).

Known bug: The deepcopy workaround for https://github.com/tensorflow/probability/issues/547 turned out not to be compatible with tensorflow-probability==0.11.1 which was released on 29 September 2020 on PyPI. We fixed this in #1574, which will be included in the following patch release (GPflow 2.1.2).

Improvements

• Enable compatibility with TensorFlow 2.3 and tensorflow_probability 0.11.0 (#1553)
• Recommend pip --use-feature=2020-resolver for installation
• Ensure all gpflow code files contain copyright & license note (#1480)

Bugfixes

• Fix SGPR upper bound for non zero mean functions (#1560)
• Fix broadcasting for constant and zero mean functions (#1550)

Release 2.1.0

Dependency constraints:
To work around incompatibilities in our TensorFlow dependencies, GPflow 2.1.0 has the following new version requirements (#1522, #1537, #1551): tensorflow<2.3, tensorflow_probability>=0.10.1,<0.11, cloudpickle==1.3.0.
We have restored compatibility with TensorFlow 2.3 and tensorflow_probability>=0.11 in the GPflow 2.1.1 patch release.

Key improvements

• gpflow.Parameter finally plays well with TensorFlow's saving; instead of pretending to be a tf.Variable, we're now building on top of tfp.util.TransformedVariable (#1518) - see backwards incompatibilities below
• Gauss-Hermite quadrature code was heavily refactored. The old function gpflow.quadrature.ndiagquad has been replaced by the gpflow.quadrature.NDiagGHQuadrature class - we strongly recommend upgrading (note that the dimensions of quadrature points have been moved from the end to the start for better broadcasting); ndiagquad will eventually be deprecated (#1505, #1542).

Minor changes

• Add base_conditional_with_lm function that gets passed cholesky(K) instead of computing it explicitly (#1528)
• Update to use stock rather than custom Docker image for CI (#1545)
• Check dependency versions are valid on CI (#1536)
• Docs: Readme updated to include new project that is using GPflow (#1530)

Bug Fixes

• Restore pytest-xdist compatibility for parallelizing local tests (#1541)
• Fix for quadrature failure mode when autograph was set to False (#1548)
• Fix formatting in docs (intro.md) and restore link removed by #1498 (#1520)
• Fix bug in varying_noise notebook (#1526)
• Fix: separate_independent_conditional now correctly handles q_sqrt=None. (#1533)

Backwards incompatibilities

• gpflow.Parameter is no longer pretending to be a tf.Variable, but instead subclasses tensorflow_probability's TransformedVariable. Several of the Variable-specific attributes/methods disappeared. Specifically, if you had previously called a Parameter's value() or read_value() methods, you should simply remove these.

Improvements

• gpflow.utilities.set_trainable() can now handle an Iterable of tf.Modules in one go (#1486)
• Clarification: the NaturalGradient optimizer does not handle the diagonal (q_diag=True) case; emphasised this in the documentation and added shape checks (#1489)
• Added type hints (#1487)
• Potentially backwards-incompatible: gpflow.optimizers.Scipy: the callable passed in as step_callback is now passed the arguments (step, variables, values) by position, no longer by name (#1454)
• Minor documentation improvements (#1488, #1492, #1498, #1503)

Bug fixes

• multi-output combination kernels now support the presliced argument (#1495)
• GPR can now be used with tf.Variables holding the data (#1490)
• fixed indentation error in Makefile (#1482)

Improvements

• Add type hints to gpflow.base module (https://github.com/GPflow/GPflow/pull/1427)
• Enable mypy checking on CI (https://github.com/GPflow/GPflow/pull/1471)

Bugfixes

• TensorFlow Probability 0.10 introduces a new _parameters attribute in tfp.bijectors.Bijector classes. That attribute contains a self-reference that broke module traversing. In this fix, module traverse ignores _parameters attribute.
• Handcrafted utilities.deepcopy was improved using memorization argument of python's copy.deepcopy method. The change prevents double copying of objects. (https://github.com/GPflow/GPflow/pull/1476)

This release contains an update to setup.py for TensorFlow 2.2, fixing PyPi issue (#1460).

• Refactor of Natural Gradient optimizer implementation to be more easily extendable and to more efficiently handle updating several variational distributions simultaneously (#1443)
• Change function signature of sample_mvn to take full_cov boolean instead of cov_structure string (#1448)
• Fixes bug in Changepoint kernel evaluation (#1446)
• Fixes #1439 (TensorFlow error when using larger datasets with tf.function compilation) in GPflow models that store data by converting it to tensors first, introduces data_input_to_tensor utility function (#1452, #1458)
• Workaround for bug in TensorFlow <= 2.2.0rc3 in the leading_transpose utility function (#1442)
• Improved representation of GPflow objects in IPython shell and Jupyter notebooks (#1453)
• Minor cleanups and added type hints (#1421, #1438, #1441, #1450)

Release notes:

• Improve structure of likelihoods subdirectory (#1416)
• Update README.md (#1401) and GPflow 2 upgrade guide (#1414)
• Improved handling of invalid values for constrained Parameters (#1408)
• Improvements on types/function annotations (#1406, #1420)
• Documentation improvements (metalearning with GPs: #1382, coregionalization notebook: #1402, MCMC notebook: #1410, intro to gpflow with tensorflow 2: #1413)
• Minor documentation fixes (#1429, #1430, #1433)
• Fix: move matplotlib import inside ImageToTensorBoard (#1399)
• Fix: tf.function compilation of ndiagquad (#1418)
• Fix: cache tensorboard file writers and re-use them (#1424)

GPflow 2.0 supports TF2.1 and does not support older versions TF1.x and TF2.0. GPflow stops accepting bug reports and PRs to the GPflow develop-1.0 branch. Please check the “GPflow 2 upgrade guide” and “Intro to GPflow with TensorFlow 2” notebooks in the documentation.

• Parameter behaves as a tf.Tensor
• Parameter has an option to set a prior on constrained or unconstrained space (#1177)
• Parameter uses bijectors from TensorFlow Probability as a transformation. A positive bijector is provided in gpflow.utilities that can be configured to exp or softplus and has a variable lower bound.
• Parameter doesn't have the setter for the trainable attribute. New utility function set_trainable added as a replacement.
• Add GPflow config module for global settings: default float and integer types, default jitter, default positive transformation, minimal positive value, default printing format. Default settings can be changed using environment variables. The gpflowrc file is not supported anymore (#1148, #1267, #1135)
• Add GPflow utility module. This module has many useful functions: custom operations like leading_transpose and broadcasting_elementwise, utility functions like customized deepcopy for tf.Module objects, freeze method that turns mutable models to immutable copies, to_default_float and many more (#1293)
• Likelihoods have a more tightly defined interface, increasing shape robustness and making it easier to incorporate non-scalar likelihoods (#1334)
• Add new Scipy optimizer that mimics standard scipy.minimize interface.
• Remove custom GPflow optimizers and HMC implementation. A user can exploit standard TensorFlow optimizers and TensorFlow Probability MCMC sampling methods accordingly (#1100)
• Add new monitor implementation (#1344)
• Refactor stationary kernels (#1088)
• Add Convolutional kernel (#996, #1034)
• Add generalized Periodic kernel (#1158)
• Add ChangePoint kernel (#1094)
• Conditional calculations only take the lower triangle parts of q_sqrt into inconsideration (#1223)
• Numerous improvements to notebooks and documentation:
  • Raw Jupyter notebook files replaced with jupytext python files (#1239)
  • Documentation generation in a separate repo (#1275)
• Improved Bayesian model classes hierarchy reflecting the difference between data handling of models (#1276)
• log_prior was renamed to log_prior_density (#1329)
• predict_density was renamed to predict_log_density
• full in the kernel.call was renamed to full_cov (#1319)
• num_latent was renamed to num_latent_gps (#1310)

Thanks a lot to everyone who made GPflow 2.0 happen:

Artem Artemev, @awav
S.T. John, @st--
Vincent Dutordoir, @vdutor
Mark van der Wilk, @markvdw
James Hensman, @jameshensman
Eric Hambro, @condnsdmatters
Joel Berkeley, @joelberkeley-pio
Alexis Boukouvalas, @alexisboukouvalas
Sergio Pascual Diaz, @spascual
Vincent Adam, @thevincentadam
Fergus Simpson, @frgsimpson
@ilia-kats
@marcoadurno
Joe Hall, @jahall
Tadej Krivec, @tadejkrivec (SGPR theory derivation fix)
Alexandra Hayes, @akhayes; James Leedham, (copyediting on notebooks)
Nicolas Durrande, @NicolasDurrande; Felix Leibfried, @fleibfried; Jordi Graumo, @jordigraumo (porting notebooks to gpflow 2.0)
Neil Ferguson, @pio-neil (GPflow 2 upgrade guide)

GPflow works with TensorFlow 2.0!

Features:

• No static graphs, pythonic way of writing the code
• Refactored GPflow configuration
• Updated model printing
• Fast multiple assign
• Convolutional kernel
• ChangePoint kernel

Features and improvements:

• Copyedits in documentation and notebooks
• Make likelihood quadrature tests comprehensive (#1081)

Bugfixes:

• Fix an issue with graph pollution in the monitor task ImageToTensorboardTask (#1041)

Features and improvements:

• Reusable ScipyOptimizer (#1011)
• Documentation copyedits (#1019, #1013-1017)
• Convolutional kernel (#996)
• Saver works with file-like objects as well as path names (#912)

Bugfixes:

• The misc.initialize_variables() does not keep adding ops to the graph (#1001)
• Variables restored in monitoring notebook (#1003)
• Support TensorFlow 1.14.0 (#997)
• Shape checks for multivariate_normal (#964)

Features, bugfixes and improvements:

• Minor performance improvement: pass cholesky to the gauss_kl (#933)
• Documentation revamp:
  • Organize documentation and notebooks (#924)
  • GPflow glossary document (#923)
  • Fix AutoBuild metaclass for method signatures (#907)
• CircleCI instead of TravisCI (#892)
• Broadcasting conditionals via tf.broadcast_to (#895)
• Support TensorFlow >= 1.12 (#877)

Main features and improvements

• Cleaning up stationary kernel implementations: now defined in terms of K_r or K_r2. (#827)
• Support broadcasting over arbitrarily many leading dimensions for kernels and conditional. (#829)
• Analytic expectation of the cross-covariance between different RBF kernels. (#754)
• New MixedKernelSeparateMof feature class for multi-output GPs. (#830)
• The sample_conditional returns mean and var as well as samples, and can generate more than one sample. (#836)
• Support monitoring with ScipyOptimizer. (#856)

Bug fixes

• Fix bug in ndiag_mc for multi-dimensional kwargs. (#813)
• Fix parameter.trainable to be a property. (#814)
• Remove references to six module. (#816)
• Fix tf.control_dependencies in likelihoods. (#821)
• Fix active_dims for slice type. (#840)

Main features and improvements

• Added SoftMax likelihood (#799)
• Added likelihoods where expectations are evaluated with Monte Carlo, MonteCarloLikelihood (#799)
• GPflow monitor refactoring, check monitor-tensorboard.ipynb for details (#792)
• Speedup testing on Travis using utility functions for configuration in notebooks (#789)
• Support Python 3.5.2 in typing checks (Ubuntu 16.04 default python3) (#787)
• Corrected scaling in Students-t likelihood variance (#777)
• Removed jitter before taking the cholesky of the covariance in NatGrad optimizer (#768)
• Added GPflow logger. Created option for setting logger level in gpflowrc (#764)
• Improved quadrature for likelihoods. Unified quadrature method introduced - ndiagquad (#736), (#747)
• Added support for multi-output GPs, check multioutput.ipynb for details (#724)
  • Multi-output features
  • Multi-output kernels
  • Multi-dispatch for conditional
  • Multi-dispatch for Kuu and Kuf
• Support Exponential distribution as prior (#717)
• Added notebook to demonstrate advanced usage of GPflow, such as combining GP with Neural Network (#712)
• Minibatch shape is None by default to allow dynamic change of data size (#704)
• Epsilon parameter of the Robustmax likelihood is trainable now (#635)
• GPflow model saver (#660)
  • Supports native GPflow models and provides an interface for defining custom savers for user's models
  • Saver stores GPflow structures and pythonic types as numpy structured arrays and serializes them using HDF5

Bug fixes

• Fixed bug at params_as_tensors_for (#751)
• Fixed GPflow SciPy optimizer to pass options to actual scipy optimizer correctly (#738)

Release 1.1.1

• pytest is a part of the dependency list

Release 1.1.0

Main features and improvements

• Actions framework - new approach for running optimization. It provides full control over optimization schedule.
• Natural gradient optimizer.
• New expectations framework.
• Analytic kernel expectations for the RBF, Linear and Sum kernels.
• Randomness in default GPflow naming.
• Pretty pandas printing for GPflow models and parameters.
• backward_tensor methods for transformations.
• gauss_kl works with K matrices of shape L x M x M.
• params_as_tensors_for can accept multiple parameterized objects.

Bug fixes

• Adam like optimizers work with TensorFlow 1.5 and higher.
• The free vector for the LowerTriangular transform wasn't being handled consistently. The LowerTriangular transform no longer expects 'free form’ variable to have a flat shape.
• LowerTriangular transformation squeezed all dimensions.
• Correct PCA implementation.
• Gradient of sqrt(RBF(Z, Z)) is no longer NaN.