Lightweight tool to deploy PyTorch models to AWS Lambda
MIT License
torchlambda is a tool to deploy PyTorch models on Amazon's AWS Lambda using AWS SDK for C++ and custom C++ runtime.
Using statically compiled dependencies whole package is shrunk to only 30MB
.
Due to small size of compiled source code users can pass their models as AWS Lambda layers. Services like Amazon S3 are no longer necessary to load your model.
torchlambda has it's PyTorch & AWS dependencies always tested & up to date because of daily continuous deployment runs.
Docs | Deployment | Package | Python | PyTorch | Docker | CodeBeat | Images |
---|---|---|---|---|---|---|---|
torchlambda
?30MB
. Previous approach to PyTorch network deployment on AWS Lambda (fastai) uses outdated PyTorch (1.1.0
) as dependency layer and requires AWS S3 to host your model. Now you can only use AWS Lambda and host your model as layer and PyTorch master
and latest stable release are supported on a daily basis.torchlambda
has at most4
commands and deployment is done via YAML settings. No need to modify your PyTorch code.torchlambda
focusesaws-cli
). You can also use your favorite third party tools (say saws, Terraform with AWS and MLFlow, PyTorch-Lightning to train your model).torchlambda
uses Amazon Linux 2 Docker images under the hood & allows you to use lambci/docker-lambda to test your deployment on localhost
before pushing deployment to the cloud (see Test Lambda deployment locally tutorial).torchlambda build
--flags
(changing various properties of PyTorch and AWS dependencies themselves). You can also write your own C++ deployment code (generate template via torchlambda template
command).3000
LOC (most being convenience wrapper creating this tool)Benchmarks can be seen in BENCHMARKS.md
file and are comprised of around ~30000 test cases.
Results are divided based on settings used, model type, payload, AWS Lambda timing etc. Below is an example of how inference performance changes due to higher resolution images and type of encoding:
128x128 | 256x256 | 512x512 | 1024x1024 | |
---|---|---|---|---|
base64 | 120.622 | 165.184 | 311.129 | 995.249 |
byte | 133.315 | 203.628 | 498.391 | 1738.97 |
char | 128.331 | 209.306 | 517.482 | 1822.56 |
short | 135.859 | 207.389 | 497.818 | 1740.91 |
int | 133.42 | 216.163 | 519.502 | 1783.02 |
long | 126.979 | 228.497 | 516.98 | 1760.93 |
float | 135.825 | 223.045 | 515.245 | 1802.25 |
double | 137.281 | 209.267 | 536.959 | 1811.83 |
Clearly the bigger image, the more important it is to use base64
encoding. For all results and description click here.
If you find an issue or you think some functionality may be useful to you, please open new Issue or create Pull Request with your changes, thanks!