serving

TensorFlow Serving using TensorFlow 1.9.0-rc2

TensorFlow Serving using TensorFlow 1.7.1.

Same serving sources as 1.7.0 release, but re-releasing binary packages to account new compiler flags introduced in 503cc0c (helps control memory blowup in ModelServer) and rebasing to TensorFlow 1.7.1 release.

TensorFlow Serving 1.6.1 using TensorFlow 1.6.0.

Same sources as 1.6.0 release, but re-releasing binary packages to account new compiler flags introduced in https://github.com/tensorflow/serving/commit/503cc0c2da0c75c5f8d94be394439776771f2d5f (helps control memory blowup in ModelServer).

TensorFlow Serving using TensorFlow 1.8.0.

This release now supports a RESTful API. You can now easily make classification, regression, and prediction requests to your TensorFlow models using JSON objects.

TensorFlow Serving using TensorFlow 1.7.0.

TensorFlow Serving using TensorFlow 1.6.0.

TensorFlow Serving using TensorFlow 1.5.0.

TensorFlow Serving using TensorFlow 1.4.0.

TensorFlow Serving using TensorFlow 1.3.0.

New Features:

• Prebuilt ModelServer binary installable with apt-get and a PIP package for the Python client API. Install instructions can be found here.
• Extended stable API: In addition to the original Predict function, we added Regress, Classify, and MultiInference. See the PredictionService for proto definitions.
• Batching scheduler moved into TensorFlow, and used in new Batch/Unbatch ops.
• Propagate ModelServer RPC deadline to Session::Run().

Updates:

• Deprecated SessionBundle format.
  • SessionBundle is deprecated. The ModelServer still works with it for now, but support is going away in the near future so it's no longer officially supported. Please move to use SavedModel instead.
• Moved documentation to tensorflow.org along with the rest of TensorFlow - https://www.tensorflow.org/serving.
• Various bug fixes and code cleanups.
• Improved documentation and updated all codelabs to work with SavedModel.

TensorFlow Serving running with TensorFlow 1.2.0.

TensorFlow Serving running with TensorFlow 1.0.0.

New Features:

• Model Server binary in tensorflow_serving/model_servers with a PredictionService API.
• Support SavedModel format and added ability to upconvert legacy SessionBundle exports. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md
  • NOTE: SessionBundle is now deprecated and we plan to end official support in the upcoming 1.0 release. Please move to use SavedModel.
• Multi-model batch scheduling: interleaveing batches for different models.
• Registry of servable types for Model Server to handle ones other than SessionBundle/SavedModel, including third-party ones not in TF-Serving codebase.
• Resource management: using model size estimates to avoid exceeding server memory capacity.
  Concurrent model loading & fast initial load.
• Request logging:
  • A protobuf based logging API.
  • Provides ability for users to log a configurable sample, or all, of the queries served.
• Support querying named signatures.

Other:

• Assembled core manager setup code into a ServerCore object.
• Various bug fixes.
• Documentation updates.

This release adds Docker support, and an end-to-end Inception tutorial. It also makes various minor fixes and clean-ups to code and documentation.

New features:

• Dockerfile and README for building a container with a TensorFlow Serving development environment.
• End-to-end example and tutorial for serving an InceptionV3 model in Kubernetes.
• Jenkins continuous integration.
• DynamicManager re-tries failed servable load attempts.
• Utility to wait until a manager has loaded certain servables.
• Utility for tracking the states of servables in a manager, by listening for servable state changes on an event bus.
• Modules for managing servable resources (but not yet integrated into DynamicManager).
• Add GetNamedSignature() to signatures, independent from any signature type.
• Misc. additions to util/.

Compatibility:

• Exporter supports both python 2.7 and python 3.4.
• Migrate to gRPC 0.13.
• Migrate to latest TensorFlow and TF-Models submodules.

Bug fixes:

• Allow importing graphs with no variable nodes.
• Misc small documentation bug-fixes.

Clean-ups:

• Depend on gRPC via a Bazel git repository, rather than via a git submodule.
• Eliminate the batching sub-namespace.
• Misc small code clean-ups.
• Misc minor documentation clean-ups.

TensorFlow Serving 0.4.0: initial public release.