🐛 Bug Fixes

What

• Fixup for fetch_data when a cachedir is provided but the parent directory has not yet been created.

Why

• Should make the downloading of assets (i.e. models/datasets) more robust to order and existing directory structure.

🚀 Features

This PR adds the SpotNet dataset to the list of new authenticated datasets, following the established pattern. It also adds a SpotNetExampleData class, which loads the example data for the Polaris example notebooks. This class deviates slightly from the existing pattern for loading data in order to accommodate for the different file types that need to be loaded. The SpotNet dataset has also been added to the datasets gallery in the documentation.

🐛 Bug Fixes

What

Closes #665. As noted there, the issue arises from pads of (0, 0) which subsequently lead to 0:0 slices in untiling, giving arrays with shape 0 for that dimension.

AFAICT, the only way to hit this corner case is when one of the dimensions is smaller than model_image_shape, but the other is exactly model_image_shape. If both dimensions are >= model_image_shape, then padding should be False if I'm following the logic correctly.

Why

Bugfix

🧰 Maintenance

This PR updates the three notebooks that are associated with the tracking paper to match our current scripts for training and running the application. I tested each notebook to verify that everything runs.

This PR updates the models used by the NuclearSegmentation and CellTracking applications to the latest versions associated with the upcoming paper.

What

Update codebase to reflect the adoption of ruff instead of pylint.

• Removes pylint configs & pylint ignore flags
• Updates the contributor guide

Why

General cleanup.

What

This PR applies pyupgrade (via ruff) to automatically modernize some coding patterns. The way this works: you tell pyupgrade what minimum version of Python you support (3.7 in our case), then it automatically applies linting patterns based on language features in the minimum supported version.

The changes here generally fall in the following categories:

• Stop inheriting from object
• Remove from __future__ imports
• Remove super() args in cases of single inheritance
• Automatically convert older-style string formatting to fstrings.

Why

The main improvement is removing cruft related to old Python versions - perhaps the most notable is the removal of the from __future__ imports and related dels in the __init__.py files. The automatic switch to f-strings is also (IMO) a nice improvement.

📚️ Documentation

This PR updates the three notebooks that are associated with the tracking paper to match our current scripts for training and running the application. I tested each notebook to verify that everything runs.

Main highlights:

• Add support for Python 3.10
• Add support for numpy version >= 1.24
• Pin scikit-image to <0.20 for compatibility

🐛 Bug Fixes

• The CellTracker had a bug where the crop mode was not being set during inference. This update to the tracking application sets the crop mode correctly.
• Reorganizes model metadata and parameters to set of global variables in each application so that they are easier to maintain and update.

Warning: This PR is dependent on a tracking release after merging https://github.com/vanvalenlab/deepcell-tracking/pull/108.

🧰 Maintenance

This should help catch incompatibilities between unreleased versions of libraries in the deepcell ecosystem.

What

• Code remains unaffected - this PR is just dedicated to bolstering testing infrastructure

Why

• The deepcell- libraries are interdependent: deepcell-tf depends on deepcell-tracking and deepcell-toolbox. If there is a change in one of these dependencies, there is no way to tell in the automated test running whether this will break something in deepcell-tf until the underlying libraries are released. Testing against the dev branches will catch potential issues sooner, at the expense of being noisier and reducing test specificity (failures can originate from either deepcell-tf or the dependencies). Overall however I think this should improve the ability to things consistent across libraries.

What

• Bump version numbers in final step before next patch release

What

• Update year in copyright notice. Accomplished with:

find . -type f -exec sed -i "s/2016-2022/2016-2023/g" {} \;

Why

• I plan to do a 0.12.5 release soon (primarily for the scikit-image pin) so I figured I'd get this in as well.

What

pydot is listed as a dependency, but is not actually used in the project, so should be safe to remove.

Why

Decreasing the dependency footprint is always beneficial. Doubly-so in the case of pydot, which has not been actively maintained in a while, see e.g. networkx/networkx#5723

What

The builtin github actions checkout, setup-python, and cache have all been updated to a later version of node. There are now deprecation warnings for the previous versions in the actions logs.

Why

General maintenance to keep the CI in good shape.

Note there may be other actions that need to be updated, but I'm starting with the main ones so I can see what remains in the logs after these updates.

What

• Fix for #655 .

Why

My vote is to pin scikit-image then do a patch release. For the next minor release the pin should be updated to >=0.19.

What

A followup to #648. With the deprecation of export_model_to_tflite, it is now the case that every function in the export_utils module has been deprecated. Therefore I propose to deprecate the entire module. We can do this using the module getattr to emit warnings if a user ever tries to access the two public names (i.e. export_model or export_utils).

In practice this means import patterns like:

>>> from deepcell.utils import export_model  # or export utils

will now raise a deprecation warning as well.

The module getattr was added in Python 3.7 - see PEP 562 for details.

Why

Further cleanup related to the export_model functions, all of which are deprecated in favor of using tf.keras.models.save_model directly.

What

Deprecate export_utils.export_model_to_tflite.

Why

Notify users who may still be using this function to switch to tf.keras.models.save_model. Closes gh-645.

What

Adopt ruff as a linter for the project. See also: vanvalenlab/deepcell-toolbox#137 and vanvalenlab/deepcell-tracking#113.

Why

Primarily to add automated linting for future code submissions, though this PR also contains a few minor fixups to address existing issues.

This one's a bit of a bear in terms of files touched and lines modified - I'm more than happy to split this up into smaller PRs to make review easier, just LMK!

What

Closes gh-643. If there's a reason not to use the nsc2 input, then alternatively we can delete that var.

Why

See gh-643 for context.

What

Add support for Python 3.10

Why

General software updates. See also vanvalenlab/deepcell-tracking#111 and vanvalenlab/deepcell-toolbox#128

What

Fixes the failures in deepcell-tf for numpy v1.24 by updating to use the

Why

Remove the upper bound on numpy.

Marking as draft for now, as this depends on vanvalenlab/deepcell-tracking#112 as well. The tracking tests in deepcell-tf will continue to fail until those changes make it into a release.

What

• Removed jupyter notebook call

What

The major change is removing the pins to sphinx, docutils, etc. AFAICT the motivating factors for the pins are no longer relevant - see e.g. #320 and #526.

Some other minor changes include:

• Minor configuration updates to get rid of warnings
• Updating intersphinx to point to the stable Python docs instead of 3.7
• Modifying the heading levels in one of the example notebooks to fix the toctree nav column

Why

Sphinx 2.3.1 is 2 major releases behind stable - being pinned this far back will make it difficult to reliably change/update the docs.

📚️ Documentation

What

Removing blurb from README about running with Python2/TensorFlow 1.

Why

The chances of this working out of a containerized environment is practically nil and certainly not worth the effort for users.

I'm also using this change as a test for the RTD docs preview feature in CI.

What

The major change is removing the pins to sphinx, docutils, etc. AFAICT the motivating factors for the pins are no longer relevant - see e.g. #320 and #526.

Some other minor changes include:

• Minor configuration updates to get rid of warnings
• Updating intersphinx to point to the stable Python docs instead of 3.7
• Modifying the heading levels in one of the example notebooks to fix the toctree nav column

Why

Sphinx 2.3.1 is 2 major releases behind stable - being pinned this far back will make it difficult to reliably change/update the docs.

🧰 Maintenance

Resolve failures in deepcell due to code that depends on numpy features that were removed in numpy v1.24.

Updates the nuclear segmentation model from model-registry # 34 and the tracking model from model-registry # 36

🧰 Maintenance

Bump version number for new release

Also includes a change from m2r to m2r2 for our documentation pipeline. m2r is no longer being maintained so it has been replaced with a fork with more active maintenance. https://github.com/CrossNox/m2r2

What

• Add support for EfficientNetV2 backbones to the get_backbone utility function

Why

• Exposes EfficientNetV2 backbones for use in deepcell model architectures

🐛 Bug Fixes

What

• Add matplotlib requirement to setup.py

Why

• Pip installations used the requirements listed in setup.py so currently matplotlib is not installed when pip installing deepcell

What

Updated the post-processing parameters for the Mesmer model. Also updates the notebook to describe how post-processing can be modified.

Why

The newly retrained model has different parameters that give the best results. In addition, I've gotten questions from a few different people about how to tweak the model output, having it in the notebook will make it easy for people to see the effects.

🚀 Features

What

• Added functionality to create TFRecords for tracking datasets

Why

• As the training datasets grow in size, they are no longer able to fit in memory (as is the case with image generators). Adding functionality for TFRecords will let us train on larger datasets as they are loaded dynamically from disk during training rather than into memory all at once.

🐛 Bug Fixes

What

• Fixed a bug that required the tracking model to have n_filters, encoder_dim, and embedding_dim be pinned to 64

Why

• Model optimization is going to require us to change these parameters to improve performance and reduce model size. This pull request makes that substantially easier by fixing this bug.

🧰 Maintenance

What

• Modify the Track class so that it allows hooks into get_image_features for crop_mode and norm

Why

• Creating the appearance image feature by cropping and resizing removes information about cell size that the model can use to make more accurate tracking predictions. A previous update to deepcell-tracking (https://github.com/vanvalenlab/deepcell-tracking/pull/98) introduced the crop_mode (either 'fixed' or 'resize') and norm flags to get_image_features. This pull request exposes these flags to the Track class.

What

• Bump deepcell-tracking to the new minor release
• Update imports to match the reorganization introduced in this release

What

Doc-string for format_output_mesmer is now correctly saying "ValueError: if model output list is not len(4)"

Why

Because format_output_mesmer code diverged from the doc in what is expected length of model output list

🚀 Features

What

Included functions to save datasets as tfrecords and load them into tf.data.Dataset objects

Why

As our training datasets grow, it is becoming difficult to load full datasets into memory. By introducing support for tfrecords, we can load portions of datasets from disk on the fly during training.

🧰 Maintenance

What

• Update models with versions that were trained on tensorflow 2.8 (https://github.com/vanvalenlab/model-registry/pull/17)

Why

• Models should use the same version of tensorflow for predictions as they were trained on

What

• Provide the option to select either BatchNormalization or LayerNormalization in GNNTrackingModel

Why

• This option makes it possible to train the model with a batch size of 1 when layer normalization is enabled.

The TF_VERSION build arg has to be updated manually

This PR updates tensorflow to 2.8 and drops support for python 3.6. The following changes were necessary to make this upgrade possible:

• Change imports from tensorflow.python.keras to tensorflow.keras which was a change introduced with tensorflow 2.6
• Remove convolutional recurrent layers and their functionality from featurenet and panopticnet. Key functions that were used in the convolutional recurrent layer are no longer available in keras.
• Change imports from tensorflow.keras to keras: keras_parameterized, conv_utils, test_utils
• Drop support for python 3.6

I retrained the nuclear model in the model-registry using this branch of deepcell and the performance was comparable.

What

• Provide the option to select either BatchNormalization or LayerNormalization in GNNTrackingModel

Why

• This option makes it possible to train the model with a batch size of 1 when layer normalization is enabled.

🧰 Maintenance

The TF_VERSION build arg has to be updated manually

🧰 Maintenance

This PR updates tensorflow to 2.8 and drops support for python 3.6. The following changes were necessary to make this upgrade possible:

• Change imports from tensorflow.python.keras to tensorflow.keras which was a change introduced with tensorflow 2.6
• Remove convolutional recurrent layers and their functionality from featurenet and panopticnet. Key functions that were used in the convolutional recurrent layer are no longer available in keras.
• Change imports from tensorflow.keras to keras: keras_parameterized, conv_utils, test_utils
• Drop support for python 3.6

I retrained the nuclear model in the model-registry using this branch of deepcell and the performance was comparable.

To dos before reviewing:

• Fix requirements that reference github branches
• Update setup.py with new requirements
• Remove git installation from Dockerfile

🧰 Maintenance

Update the mesmer model from model-registry#15. I'll leave this PR as a draft until @ngreenwald has a chance to test post-processing parameters and determine if any changes are needed.

What

• The nuclear tracking model has been updated in the model registry. This PR redirects the current pointers to grab this new version. The division parameter has also been updated to match the result of the latest parameter sweep.

Why

• The applications should match the latest verified version from the model registry.

📚️ Documentation

What

• Pins jinja2<3.1 in docs/requirements-docs.txt

Why

• The new jinja2 release deprecates functions that are required for sphinx.

All-

Someone in the OME community recently ran into some confusion with the licensing of deepcell-tf. The license badge lists "Apache 2.0" but the LICENSE file states "Modified Apache 2.0". This proposes use of to increase the clarity.

All the best,
~Josh Moore

--- LICENSE-2.0.txt     2022-04-01 14:41:12.806279265 +0100
+++ LICENSE     2022-04-01 14:41:45.921931426 +0100
@@ -1,7 +1,5 @@
-
-                                 Apache License
+                            Modified Apache License
                            Version 2.0, January 2004
-                        http://www.apache.org/licenses/

    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

@@ -65,18 +63,19 @@
       subsequently incorporated within the Work.

    2. Grant of Copyright License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      copyright license to reproduce, prepare Derivative Works of,
-      publicly display, publicly perform, sublicense, and distribute the
-      Work and such Derivative Works in Source or Object form.
+      this License, each Contributor hereby grants to You a non-commercial,
+      academic perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+      irrevocable copyright license to reproduce, prepare Derivative Works
+      of, publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form. For any other
+      use, including commercial use, please contact: [email protected].

    3. Grant of Patent License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      (except as stated in this section) patent license to make, have made,
-      use, offer to sell, sell, import, and otherwise transfer the Work,
-      where such license applies only to those patent claims licensable
+      this License, each Contributor hereby grants to You a non-commercial,
+      academic perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+      irrevocable (except as stated in this section) patent license to make,
+      have made, use, offer to sell, sell, import, and otherwise transfer the
+      Work, where such license applies only to those patent claims licensable
       by such Contributor that are necessarily infringed by their
       Contribution(s) alone or by combination of their Contribution(s)
       with the Work to which such Contribution(s) was submitted. If You
@@ -174,6 +173,10 @@
       incurred by, or claims asserted against, such Contributor by reason
       of your accepting any such warranty or additional liability.

+  10. Neither the name of Caltech nor the names of its contributors may be
+      used to endorse or promote products derived from this software without
+      specific prior written permission.
+
    END OF TERMS AND CONDITIONS

    APPENDIX: How to apply the Apache License to your work.

🐛 Bug Fixes

What

• Updated the tracking notebook so Track and concat_tracks were imported from deepcell.data.tracking

Why

• The location of these files changed in the latest version of deepcell

🧰 Maintenance

• Update copyright in deepcell/_version.py to 2022

What

• Exposes an option to use the graph attention layer GATConv from spektral

Why

• Allows us to test the impact of different graph layers on the tracking model

• Updates nuclear segmentation model with the version created by https://github.com/vanvalenlab/data-registry/pull/188
• Update cytoplasm segmentation model with the version created by https://github.com/vanvalenlab/data-registry/pull/164

The same models have been uploaded to the deepcell-models bucket in GCP for deployment through the kiosk.

What

• Add a new GitHub workflow to build a docker image and run the unit tests inside it.

Why

• This ensures that the docker images work as expected rather than relying solely on the tests passing in the test-runner environment.

What

• Update the ReadMe with the link for the paper

Why

• I'll merge this in tomorrow AM once the link goes live

What

• Revert TensorFlow version from 2.5.2 to 2.5.1. The patch is already able to be installed via setup.py and requirements.txt.

Why

• No 2.5.2 docker images are available, the patch can be installed via pip.
• Fixes #568

What

• Fix bug in #566 with missing change in requirements.txt

Why

• Finish update of base TensorFlow version.

What

• Update TF_VERSION used to 2.5.2

Why

• Fixes several vulnerabilities

🚀 Features

What

• Update the adjacency matrix data from to a sparse tensor to significantly reduce memory footprint.
• Move Track and concat_tracks functions from deepcell-tracking to deepcell.data.tracking. They are really just .trk preprocessing and are unnecessary outside of prepare_data.
• Update temporal_slice to not slice into padded frames.
• Bump deepcell-tracking to 0.5.0.

Why

• Continue to upgrade the tracking model to make it more usable.
• The temporal_slice fix should improve the precision metrics of the model by not training on padded data.

🐛 Bug Fixes

What

• Update numpy version constraints to match setup.py.

Why

• Fixes #559.

What

• Revert the change to prepare_dataset that allows it to process .trks files.

Why

• This change caused complications downstream (primarily in using different train and val files) and is not useful.

What

• Fix the filter_and_flatten function.
• Add the graph_layer argument to the model call.

Why

• This function was not filtering the padded data properly, fixing it resolves the training issues.
• Improve overall clarity of options for tracking models.

🧰 Maintenance

What

• Update the package version to 0.11.0.

Why

• Getting ready for the next release.

What

• Pin docutils to 0.16 to resolve readthedocs build failures
• Add a monthly downloads badge
• Clean up grammar and whitespace.

Why

• General improvement for the README

What

• Retrain tracking models using SparseTensors

Why

• Provide the latest and greatest models in the next release.

🐛 Bug Fixes

What

• Update numpy version constraints to match setup.py.

Why

• Fixes Python 3.9 compatibility issue (#559)

🚀 Features

What

• Update the adjacency matrix data from to a sparse tensor to significantly reduce memory footprint.
• Move Track and concat_tracks functions from deepcell-tracking to deepcell.data.tracking. They are really just .trk preprocessing and are unnecessary outside of prepare_data.
• Update temporal_slice to not slice into padded frames.
• Bump deepcell-tracking to 0.5.0.

Why

• Continue to upgrade the tracking model to make it more usable.
• The temporal_slice fix should improve the precision metrics of the model by not training on padded data.

🐛 Bug Fixes

What

• Update the tracking model files used by the CellTracker application.

Why

• The previous model was trained on data with errors. That data was identified and pruned, the resulting model has significantly better performance.

What

• Convert all optimizer lr arguments to learning_rate.

Why

• Fixes deprecation warning.

🧰 Maintenance

What

• Bump version to 0.10.1

Why

• Getting ready for patch release before moving to 0.11.0.

🚀 Features

What

• End-to-End updates for DeepCell's approach to the tracking problem in live-cell imaging. A dataset builder module was added to to more closely follow TensorFlow's preferred style (with tracking dataset object being the first example). The previous LSTM and Siamese Neural Network (SNN) architecture was moved from featurenet.py in the model_zoo to a new 'tracking.py,` where a graph-based architecture for classification was also installed. This new model required 2 new "merge" layers and an additional loss function compatible with masking out portions of the adjacency matrix. The Tracking Application has been updated to take advantage of this new model and approach.

Why

• These updates represent the natural evolution of the repo's approach to tracking. It addresses key needs to further enable adoption of TF2 and dramatically improves tracking speed.

What

Add new InferenceTimer callback to measure inference time for a pre-defined number of samples:

Average inference speed per sample for 100 total samples: 0.00204s ± 0.00158s.

Why

• Fixes #275

What

• Set preprocessing_fn and postprocessing_fn as arguments to init to allow them to be overridden when creating applications.

Why

• This allows the applications to be more flexible and prevents users from subclassing the Application when they use a different preprocessing function.

Enables overriding the default pad_mode to use "reflect" by default.

What

• Add pad_mode as a new argument to CytoplasmSegmentation.predict with default value "reflect".

Why

• Allows overriding the padding mode for tiling the input. Prevents edge effects for models not trained on zero-padded data.

What

Previously, the default values for the Mesmer post-processing kwargs were configured such that if the user passed any args to the app, all of the other defaults would be reset. This PR modifies the behavior so that only the kwargs specified by the user are changed, the other defaults remain unchanged

Why

Removes confusing behavior where users would think they're only overriding a single arg, when in fact they would be resetting all of the args back to the deep_watershed defaults, which are not the same as the Mesmer defaults

What

• Remove the Fully Connected layers from the tracking decoder
• Enable multiple graph convolutional layer types with new graph_layer argument, currently supports GCN and GCS.

Why

• Improve model performance and ease of use.

🐛 Bug Fixes

What

• Set include_top to True by default in __create_semantic_head
• Remove include_top from example notebook to avoid confusion

Why

• Fixes #515

What

• Add Semantic data generators to data_generators.__all__.

Why

• Allow sphinx to build the docs, fixes #513.

What

Modifies the applications to use internal batch prediction function rather than the default model.predict batching functionality. Closes #538

Why

The default model.predict batch function creates a tf.dataset object with all images, not just the specified batch size. This leads to memory issues on the GPU when batch processing tiles from large images.

What

• Use sed to replace tensorflow~= with tensorflow-cpu~=. Including the ~ prevents other deps that start with the word tensorflow from getting replaced.
• Bump version to 0.10.0-rc.2

Why

• Fixes #547

🧰 Maintenance

What

• Update TensorFlow to the latest 2.5.x release.

Why

• Fixes several CVEs.

What

• Add template file for release-drafter workflow.

Why

• Creates a template to build release drafts with release-drafter.
• The workflow will be added as a separate PR because it needs this template in the default branch.

What

• Add release-drafter workflow YAML file.

Why

• Automatically update release drafts on commits to master.

What

• Add pydot>=1.4.2,<2 to setup.py and requirements.txt
• Install graphviz in the Dockerfile

Why

• Enable use of tf.keras.utils.plot_model out of the box
• Fixes #504

What

• Update version of both Nuclear and Cytoplasm Segmentation models.

Why

• These new models have been trained using the data registry and are the latest models available.

What

• Bump deepcell-toolbox version to 0.10.0
• Update deepcell.metrics to remove old metrics imports
• Update Mesmer to use new combined deep_watershed

Why

• Update to latest version of the toolbox

What

• Include Python 3.9 in the GitHub Actions testing workflow.
• Update setup.py to include support for Python 3.9.

Why

• TensorFlow 2.5.0 supports Python3.9 and so should DeepCell.

What

• Cache the entire Python environment in the testing GitHub Action workflow.

Why

• Drastically speed up build times.

What

• Update version to 0.10.0

Why

• Get ready for the next release

What

• Update neighborhood encoder and inference model hashes for the newly trained model

Why

• New model reflects architecture changes and has been trained on diverse data.

0.8.8

Bugfixes

• Upgrade tensorflow to v2.3.4 to fix several CVEs.