vertex-ai-huggingface-inference-toolkit

v0.0.1

🎉 First release of vertex-ai-huggingface-inference-toolkit!

[!WARNING]
This is still very at a very early stage and subject to major changes.

Features

• 🤗 Straight forward way of deploying models from the Hugging Face Hub in Vertex AI
• 🐳 Automatically build Custom Prediction Routines (CPR) for Hugging Face Hub models using transformers.pipeline
• 📦 Everything is packaged within a single method, providing more flexibility and ease of usage than the former google-cloud-aiplatform SDK for custom models
• 🔌 Seamless integration for running inference on top of any model from the Hugging Face Hub in Vertex AI thanks to transformers
• 🔍 Includes custom logging messages for better monitoring and debugging via Google Cloud Logging

0.0.1.b12

🤞🏻

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b11...0.0.1.b12

0.0.1.b11

This release adds more logging messages, automatic generation for both instance_schema_uri and prediction_schema_uri only for HF_TASK='zero-shot-classification', as well as some other minor fixes.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b10...0.0.1.b11

0.0.1.b10

🤞🏻

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b9...0.0.1.b10

0.0.1.b9

This release adds more logging messages.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b8...0.0.1.b9

0.0.1.b8

This release adds the re.compile check back when falling back from ValueError on device_map, and also adds some more logging messages to better monitor the issue.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b7...0.0.1.b8

0.0.1.b7

This release skips the re.compile check on the device_map="auto" issue, in order to better debug the Vertex AI Endpoint, in order to fallback to device=get_device() by default.

This release is mainly for debugging purposes.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b6...0.0.1.b7

0.0.1.b6

This release changes the default logging messages from DEBUG to INFO, and removes the escaped characters from re.compile pattern.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b5...0.0.1.b6

0.0.1.b5

This release fixes an issue related to some model architectures not supporting device_map="auto" from 🤗accelerate, so on, a fallback mechanism has been implemented in order to use the available device via a get_device function if the device_map is not supported, otherwise an exception is raised normally.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b4...0.0.1.b5

0.0.1.b4

This release adds the transformers extra installing both transformers and accelerate as otherwise the TransformersPredictor won't work. Besides that the Dockerfile.cpu and Dockerfile.gpu files have been updated so that a specific version of vertex_ai_huggingface_inference_toolkit is installed, as well as the transformers extras (which are the default ones atm).

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b3...0.0.1.b4

0.0.1.b3

This release fixes a bug within the tarfile compression of the model.tar.gz, as well as in upload_file_to_gcs to return the URI to the directory in Google Cloud Storage instead of the URI to the file, as aiplatform.Model.upload expects the directory not the file.

Besides that, the build_docker_image has also been fixed since it was not using the platform=linux/amd64 as it was being provided as build-arg instead of as a Docker argument.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b2...0.0.1.b3

0.0.1.b2

This release fixes a bug within the previous release where shutil.rmtree was being called with a file instead of a directory, so it has been replaced now with pathlib.Path.unlink instead, also to keep everything within pathlib without having to use shutil at all.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b0...0.0.1.b1

0.0.1.b1

This is an iteration on top of 0.0.1.b0 to fix some existing issues when trying to run the example within the README.md, which will be the first working example.

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/compare/0.0.1.b0...0.0.1.b1

0.0.1.b0

This is just a beta release with the basic features to upload the 🤗transformers models to Google Cloud Storage, to register those in Vertex AI, and later to deploy those as an endpoint, everything running a Custom Prediction Routine (CPR) built based on the specified input parameters.

Still under testing, and needs WAY more testing!

Full Changelog: https://github.com/alvarobartt/vertex-ai-huggingface-inference-toolkit/commits/0.0.1.b0