Major Features and Improvements

• Add params as an optional third argument to the embedding_fn argument of
  nsl.estimator.add_graph_regularization. This is similar to the params
  argument of an Estimator's model_fn, which allows users to pass arbitrary
  states through. Adding this as an argument to embedding_fn will allow
  users to access that state in the implementation of embedding_fn.
• Both nsl.keras.AdversarialRegularization and
  nsl.keras.GraphRegularization now support the save method which will
  save the base model.
• nsl.keras.AdversarialRegularization now supports a tf.keras.Sequential
  base model with a tf.keras.layers.DenseFeatures layer.
• nsl.configs.AdvNeighborConfig has a new field random_init. If set to
  True, a random perturbation will be performed before FGSM/PGD steps.
• nsl.lib.gen_adv_neighbor now has a new parameter use_while_loop. If set
  to True, the PGD steps are done in a tf.while_loop which is potentially
  more memory efficient but has some restrictions.
• New library functions:
  • nsl.lib.random_in_norm_ball for generating random tensors in a norm
    ball.
  • nsl.lib.project_to_ball for projecting tensors onto a norm ball.

Bug Fixes and Other Changes

• Dropped Python 2 support (which was deprecated 2+ years ago).
• nsl.keras.AdversarialRegularization and nsl.lib.gen_adv_neighbor will
  not attempt to calculate gradients for tensors with a non-differentiable
  dtype. This doesn’t change the functionality, but only suppresses excess
  warnings.
• Both estimator/adversarial_regularization.py and
  estimator/graph_regularization.py explicitly import estimator from
  tensorflow as a separate import instead of accessing it via tf.estimator
  and depend on the tensorflow estimator target.
• The new top-level workshops directory contains presentation materials from
  tutorials we organized on NSL at KDD 2020, WSDM 2021, and WebConf 2021.
• The new usage.md page describes featured usage of NSL, external talks,
  blog posts, media coverage, and more.
• End-to-end examples under the examples directory:
  • New examples about graph neural network modules with graph-regularizer
    and graph convolution.
  • New README file providing an overview of the examples.
• New tutorial examples under the examples/notebooks directory:
  • Graph regularization for image classification using synthesized graphs
  • Adversarial Learning: Building Robust Image Classifiers
  • Saving and loading NSL models

Thanks to our Contributors

This release contains contributions from many people at Google Research and from
TF community members: @angela-wang1 , @dipanjanS, @joshchang1112, @SamuelMarks,
@sayakpaul, @wangbingnan136, @zoeyz101

Major Features and Improvements

None.

Bug Fixes and Other Changes

• Fixed the NSL graph builder to ignore lsh_rounds when lsh_splits < 1. By
  default, the prior version of the graph builder would repeat the work twice
  by default. In addition, the default value for lsh_rounds has been changed
  from 2 to 1.
• Updated the NSL IMDB tutorial to use the new LSH support when building the
  graph, thereby speeding up the graph building time by ~5x.

Thanks to our Contributors

This release contains contributions from many people at Google.

Major Features and Improvements

• Added locality-sensitive hashing (LSH) support to the graph builder tool.
  This allows the graph builder to scale up to larger input datasets. As part
  of this change, the new nsl.configs.GraphBuilderConfig class was
  introduced, as well as a new nsl.tools.build_graph_from_config function.
  The new parameters for controlling the LSH algorithm are named lsh_rounds
  and lsh_splits.

Bug Fixes and Other Changes

• Changed nsl.tools.add_edge to return a boolean result indicating if a new
  edge was added or not; previously, this function was not returning any
  value.
• Fixed a bug in nsl.tools.read_tsv_graph that was incrementing the reported
  edge count too often.
• Removed Python 2 unit tests.
• Fixed a bug in nsl.estimator.add_adversarial_regularization and
  nsl.estimator.add_graph_regularization so that the UPDATE_OPS can be
  triggered correctly.
• Updated graph-NSL tutorials not to parse neighbor features during
  evaluation.
• Added scaled graph and adversarial loss values as scalars to the summary in
  nsl.estimator.add_graph_regularization and
  nsl.estimator.add_adversarial_regularization respectively.
• Updated graph and adversarial regularization loss metrics in
  nsl.keras.GraphRegularization and nsl.keras.AdversarialRegularization
  respectively, to include scaled values for consistency with their respective
  loss term contributions.

Thanks to our Contributors

This release contains contributions from many people at Google.

Release 1.2.0

Major Features and Improvements

• Changed nsl.tools.build_graph(...) to be more efficient and use far less
  memory. In particular, the memory consumption is now proportional only to
  the size of the input, not the size of the input plus the size of the
  output. Since the size of the output can be quadratic in the size of the
  input, this can lead to large memory savings. nsl.tools.build_graph(...)
  now also produces a log message every 1M edges it writes to indicate
  progress.
• Introduces nsl.lib.strip_neighbor_features, a function to remove graph
  neighbor features from a feature dictionary.
• Restricts the expectation of graph neighbor features being present in the
  input to the training mode for both the Keras and Estimator graph
  regularization wrappers. So, during evaluation, prediction, etc, neighbor
  features need not be fed to the model anymore.
• Change the default value of keep_rank from False to True as well as
  flip its semantics in nsl.keras.layers.NeighborFeatures.call and
  nsl.utils.unpack_neighbor_features.
• Supports feature value constraints for adversarial neighbors. See
  clip_value_min and clip_value_max in nsl.configs.AdvNeighborConfig.
• Supports adversarial regularization with PGD in Keras and estimator models.
• Support for generating adversarial neighbors using Projected Gradient
  Descent (PGD) via the nsl.lib.adversarial_neighbor.gen_adv_neighbor API.

Bug Fixes and Other Changes

• Clarifies the meaning of the nsl.AdvNeighborConfig.feature_mask field.
• Updates notebooks to avoid invoking the nsl.tools.build_graph and
  nsl.tools.pack_nbrs utilities as binaries.
• Replace deprecated API in notebooks when testing for GPU availability.
• Fix typos in documentation and notebooks.
• Improvements to example trainers.
• Fixed the metric string to 'acc' to be compatible with both TF1.x and 2.x.
• Allow passing dictionaries to sequential base models in adversarial
  regularization.
• Supports input feature list in nsl.lib.gen_adv_neighbor.
• Supports input with a collection of tensors in
  nsl.lib.maximize_within_unit_norm.
• Adds an optional parameter base_with_labels_in_features to
  nsl.keras.AdversarialRegularization for passing label features to the base
  model.
• Fixes the tensor ordering issue in nsl.keras.AdversarialRegularization
  when used with a functional Keras base model.

Thanks to our Contributors

This release contains contributions from many people at Google as well as
@mzahran001.

Release 1.1.0

Major Features and Improvements

• Introduces nsl.tools.build_graph, a function for graph building.

• Introduces nsl.tools.pack_nbrs, a function to prepare input for
  graph-based NSL.

• Adds tf.estimator.Estimator support for NSL. In particular, this release
  introduces two new wrapper functions named
  nsl.estimator.add_graph_regularization and
  nsl.estimator.add_adversarial_regularization to wrap existing
  tf.estimator.Estimator-based models with NSL. These APIs are currently
  supported only for TF 1.x.

Bug Fixes and Other Changes

• Adds version information to the NSL package, which can be queried as
  nsl.__version__.

• Fixes loss computation with Loss objects in AdversarialRegularization.

• Adds a new parameter to nsl.keras.adversarial_loss which can be used to
  pass additional arguments to the model.

• Fixes typos in documentation and notebooks.

• Updates notebooks to use the release version of TF 2.0.

Thanks to our Contributors

This release contains contributions from many people at Google.

Release 1.0.1

Major Features and Improvements

• Adds make_graph_reg_config, a new API to help construct a nsl.configs.GraphRegConfig object

• Updates the package description on PyPI

Bug Fixes and Other Changes

• Fixes metric computation with Metric objects in AdversarialRegularization

• Fixes typos in documentation and notebooks

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

@joaogui1, @aspratyush.

Initial release of Neural Structured Learning.