🤗 HuggingFace Inference Toolkit for Google Cloud Vertex AI (similar to SageMaker's Inference Toolkit, but for Vertex AI and unofficial)
APACHE-2.0 License
Bot releases are hidden (Show)
Published by alvarobartt 7 months ago
🎉 First release of vertex-ai-huggingface-inference-toolkit
!
[!WARNING]
This is still very at a very early stage and subject to major changes.
transformers.pipeline
google-cloud-aiplatform
SDK for custom modelstransformers
logging
messages for better monitoring and debugging via Google Cloud LoggingPublished by alvarobartt 8 months ago
🤞🏻
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b11...0.0.1.b12
Published by alvarobartt 8 months ago
This release adds more logging
messages, automatic generation for both instance_schema_uri
and prediction_schema_uri
only for HF_TASK='zero-shot-classification'
, as well as some other minor fixes.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b10...0.0.1.b11
Published by alvarobartt 8 months ago
🤞🏻
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b9...0.0.1.b10
Published by alvarobartt 8 months ago
This release adds more logging
messages.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b8...0.0.1.b9
Published by alvarobartt 8 months ago
This release adds the re.compile
check back when falling back from ValueError
on device_map
, and also adds some more logging
messages to better monitor the issue.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b7...0.0.1.b8
Published by alvarobartt 8 months ago
This release skips the re.compile
check on the device_map="auto"
issue, in order to better debug the Vertex AI Endpoint, in order to fallback to device=get_device()
by default.
This release is mainly for debugging purposes.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b6...0.0.1.b7
Published by alvarobartt 8 months ago
This release changes the default logging messages from DEBUG
to INFO
, and removes the escaped characters from re.compile
pattern.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b5...0.0.1.b6
Published by alvarobartt 8 months ago
This release fixes an issue related to some model architectures not supporting device_map="auto"
from 🤗accelerate
, so on, a fallback mechanism has been implemented in order to use the available device via a get_device
function if the device_map
is not supported, otherwise an exception is raised normally.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b4...0.0.1.b5
Published by alvarobartt 8 months ago
This release adds the transformers
extra installing both transformers
and accelerate
as otherwise the TransformersPredictor
won't work. Besides that the Dockerfile.cpu
and Dockerfile.gpu
files have been updated so that a specific version of vertex_ai_huggingface_inference_toolkit
is installed, as well as the transformers
extras (which are the default ones atm).
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b3...0.0.1.b4
Published by alvarobartt 8 months ago
This release fixes a bug within the tarfile
compression of the model.tar.gz
, as well as in upload_file_to_gcs
to return the URI to the directory in Google Cloud Storage instead of the URI to the file, as aiplatform.Model.upload
expects the directory not the file.
Besides that, the build_docker_image
has also been fixed since it was not using the platform=linux/amd64
as it was being provided as build-arg
instead of as a Docker argument.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b2...0.0.1.b3
Published by alvarobartt 8 months ago
This release fixes a bug within the previous release where shutil.rmtree
was being called with a file instead of a directory, so it has been replaced now with pathlib.Path.unlink
instead, also to keep everything within pathlib
without having to use shutil
at all.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b0...0.0.1.b1
Published by alvarobartt 8 months ago
This is an iteration on top of 0.0.1.b0
to fix some existing issues when trying to run the example within the README.md
, which will be the first working example.
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b0...0.0.1.b1
Published by alvarobartt 8 months ago
This is just a beta release with the basic features to upload the 🤗transformers
models to Google Cloud Storage, to register those in Vertex AI, and later to deploy those as an endpoint, everything running a Custom Prediction Routine (CPR) built based on the specified input parameters.
Still under testing, and needs WAY more testing!
Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/commits/0.0.1.b0