Serve, optimize and scale PyTorch models in production
APACHE-2.0 License
Bot releases are visible (Hide)
Published by maaquib about 4 years ago
This is the release of TorchServe v0.2.0
serve/examples
.v0.1.1
NameError
in default image_classifier handler #489Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+, Windows subsystem for Linux (Windows Server 2019, WSLv1, Ubuntu 18.0.4)
Additionally, you can get started at https://pytorch.org/serve/ with installation instructions, tutorials and docs.
Lastly, if you have questions, please drop it into the PyTorch discussion forums using the ‘deployment’ tag or file an issue on GitHub with a way to reproduce.
Published by mycpuorg over 4 years ago
This is the release of TorchServe v0.1.1
v0.1.0
--model-store
should point to a user-relative directory. #248Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+
Additionally, you can get started at pytorch.org/serve with installation instructions, tutorials and docs.
Lastly, if you have questions, please drop it into the PyTorch discussion forums using the ‘deployment’ tag or file an issue on GitHub with a way to reproduce.
Published by mycpuorg over 4 years ago
This is the first release of TorchServe (Experimental), a new open-source model serving framework under the PyTorch project (RFC #27610).
Clean APIs - Support for an Inference API for predictions and a Management API for managing the model server.
Secure Deployment - Includes HTTPS support for secure deployment.
Robust model management capabilities - Allows full configuration of models, versions, and individual worker threads via command line interface, config file, or run-time API.
Model archival - Provides tooling to perform a ‘model archive’, a process of packaging a model, parameters, and supporting files into a single, persistent artifact. Using a simple command-line interface, you can package and export in a single ‘.mar’ file that contains everything you need for serving a PyTorch model. This `.mar’ file can be shared and reused. Learn more here.
Built-in model handlers - Support for model handlers covering the most common use-cases (image classification, object detection, text classification, image segmentation). TorchServe also supports custom handlers
Logging and Metrics - Support for robust logging and real-time metrics to monitor inference service and endpoints, performance, resource utilization, and errors. You can also generate custom logs and define custom metrics.
Model Management - Support for management of multiple models or multiple versions of the same model at the same time. You can use model versions to roll back to earlier versions or route traffic to different versions for A/B testing.
Prebuilt Images - Ready to go Dockerfiles and Docker images for deploying TorchServe on CPU and NVIDIA GPU based environments. The latest Dockerfiles and images can be found here.
- Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+