batched-fn

🦀 Rust server plugin for deploying deep learning models with batched prediction

APACHE-2.0 License

Downloads
16.2K
Stars
18
Committers
2
View Code on GitHub Visit Website
Ecosystems: Rust

Bot releases are visible (Hide)

batched-fn - v0.2.5 Latest Release

Published by github-actions[bot] 7 months ago

What's new

Fixed ✅

  • Removed unnecessary Mutex in BatchedFn.
  • Fixed issue where thread would crash when request from calling thread disconnects.

Commits

7196d9d Fix thread crash on disconnect (#26)
ac6c530 remove unnecessary Mutex (#22)
2cd40dd generate release notes before cargo release

batched-fn - v0.2.4

Published by github-actions[bot] over 2 years ago

What's new

Added 🎉

  • Added more documentation about channel_cap.

Commits

c86329d add more documentation about "channel_cap" (#19)
cdc54a3 fix release notes script

Package Rankings
Top 24.11% on Crates.io
Related Projects
async-select

`select!` multiplex asynchronous futures simultaneously

09 May 2024 0
text-embeddings-inference

A blazing fast inference solution for text embeddings models

13 Oct 2023 2,039
ort-batcher

Small crate to batch inferences of ONNX models using ort (onnxruntime)

17 Nov 2023 2
futures-batch

An adapter for futures, which chunks up elements and flushes them after a timeout — or when the b...

24 Oct 2017 76