Included are examples which can deploy and serve stable diffusion / runwayml / midjourney or any of the Hugging face diffusers models in seconds. Or you can use the new mitsua diffusion option which is trained on public data.
We've observed about a 2x speed up across the example schema vs a FastAPI/Ariadne Python GraphQL server with identical schema. This is because although the model code is in Python, the API is actually a Rust process running TechEmpower no.5 fastest web framework ActiX.
Please note
pip install fastqlapi
You can call any callback with each GraphQL query recieved by the server, the callback must return a dict with values for each field you wish to return. The returned type must match the type defined for the field in the fields dict given to the start method (see below).
For example:
from fastqlapi import fastql_server
def infer(**kwargs):
print (kwargs['input'])
return {
'output': "test response",
}
fastql_server.start(callback=infer, query_name="Model", args={"input": { "type": "String", "description": "this is my input field"}}, fields={"output": { "type": "String"}})
Will spin up the below GraphQL Playground ready for requests on localhost:8000/graphiql or you can make a GraphQL request to localhost:8000 like:
{
Model(input="to send"){
output
}
}
NB. these examples are just toy examples not suitable for production
EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id || die \"wget instance-id has failed: $?\"`"
export PUBLIC_IP=$(aws ec2 describe-instances --instance-ids $EC2_INSTANCE_ID --query 'Reservations[*].Instances[*].PublicIpAddress' --output text)
echo $PUBLIC_IP
and make a note of this public ip you will use to connect to the APIdocker run -p 8000:8000 -p 8080:8080 \
-e PUBLIC_IP=$PUBLIC_IP \
-e MODEL_ID="stabilityai/stable-diffusion-2" \
--gpus all danfreshbc/fastqlapi-diffusers:latest
Once Docker has finished installing...
{
Model1(prompt: "a handsome man holding a baby goat") {
images
}
}
You can change MODEL_ID to one of the following to try a different diffusers model for example:
"Mitsua/mitsua-diffusion-cc0"
This is recommended as it trained on totally public images, there is some controversy over the legality of the models below (be warned!)
"CompVis/stable-diffusion-v1-4"
"stabilityai/stable-diffusion-2"
"runwayml/stable-diffusion-v1-5"
The example API surfaces prompt, number_of_images, guidance_scale, number_inference_steps and seed and returns an array of images or a seed.
The example uses a ruby webserver to serve up the content of an images directory which it will deposit generated images into on port 8080.
NB. Mutation support coming soon.
This example allows you to download a folder of images from google drive or individual gdrive image ids. It is suggested to upload 10-12 images to a Gdrive folder and make sure you set sharing to 'Anyone with the link' and copy the link. The toy API will then finetune the model on these images so you can run inference against them. This example will overwrite the model each time you upload new files.
docker run -p 8000:8000 -p 8080:8080 \
-e PUBLIC_IP=$PUBLIC_IP \
-e MODEL_ID="stabilityai/stable-diffusion-2" \
--gpus all danfreshbc/fastqlapi-diffusers:latest bash start_finetune.sh
To start finetuning use a query like below:
{
Model(
fine_tune_photo_description: "Photo of happymachine"
gdrive_folder_of_images_link: "https://drive.google.com/drive/folders/17g_m3eaBA6SJQP-xppGkAHXYoKCLObZg"
) {
images
}
}
This query will take around five minutes while the model is tuned and prepared for inference.
To infer:
{
Model(prompt: "happymachine standing in front of an F1 McLaren") {
images
}
}
FastQL currently implements Int, Float, String, ID and array and required versions of those types but not currently required subtypes (an element of a list).
Under the hood FastQL uses the ActiX Rust web server which is currently no.5 fastest web framework according to Tech Empower. By comparison, Python's FastAPI is at no.93. We've observed about a 2x speed up across the example schema here vs a FastAPI/Ariadne Python GraphQL server with the same schema.
GRAPHQL_HOST Default localhost
GRAPHQL_PORT Default 8000
ENABLE_GRAPHIQL Will enable GraphiQL | default to true unless RUST_ENV is set to production (in which case setting ENABLE_GRAPHIQL=true will still overide)
CORS_PERMISSIVE Should be used in Development/Test only, allows all headers and origins, credentials supported, maximum age 1 hour does not send wildcard | default true
ALLOW_AUTHORIZATION_HEADER default false
ALLOW_ORIGIN_HEADER default false
ALLOW_CONTENT_TYPE_HEADER default false
MAX_AGE_HEADER default 3600
RUST_ENV default 'development'
RUST_LOG Rust log level | default 'debug'
RUST_BACKTRACE Add Rust backtrace to log | default 1
RUST_QUIET No Rust logs | default false
TRACING Turn on Apollo tracing | default false
The code in this repo is released under the MIT license.
The folks at Hugging Face The team at Actix Sunli @ Async-graphql.rs The team at Maturin