🦀 Rust server plugin for deploying deep learning models with batched prediction
APACHE-2.0 License
Bot releases are hidden (Show)
An adapter for futures, which chunks up elements and flushes them after a timeout — or when the b...
Small crate to batch inferences of ONNX models using ort (onnxruntime)
A blazing fast inference solution for text embeddings models
`select!` multiplex asynchronous futures simultaneously
