ort-batcher

Small crate to batch inferences of ONNX models using ort (onnxruntime)

APACHE-2.0 License

Downloads

Stars

Ecosystems: Rust

Package Rankings

Rust crate where we implement vectorized versions of fast pseudo-random number generators.

Deep learning in Rust, with shape checked tensors and neural networks

A fast llama2 decoder in pure Rust.

An adapter for futures, which chunks up elements and flushes them after a timeout — or when the b...

🦀 Rust server plugin for deploying deep learning models with batched prediction

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)