py-txi
A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
APACHE-2.0 License
Bot releases are hidden (Show)
What's Changed
- optional colored logs and speculative decoding by @IlyasMoutawwakil in https://github.com/IlyasMoutawwakil/py-txi/pull/7
- revert by @IlyasMoutawwakil in https://github.com/IlyasMoutawwakil/py-txi/pull/8
Full Changelog: https://github.com/IlyasMoutawwakil/py-txi/compare/v0.7.0...v0.8.0
Published by IlyasMoutawwakil 6 months ago
What's Changed
- Updates for TGI 2.0 by @IlyasMoutawwakil in https://github.com/IlyasMoutawwakil/py-txi/pull/6
Full Changelog: https://github.com/IlyasMoutawwakil/py-txi/compare/v0.6.0...v0.7.0
Published by IlyasMoutawwakil 6 months ago
What's Changed
- Added semaphores and event loop management by @IlyasMoutawwakil in https://github.com/IlyasMoutawwakil/py-txi/pull/4
Full Changelog: https://github.com/IlyasMoutawwakil/py-txi/compare/v0.5.0...v0.6.0
Published by IlyasMoutawwakil 8 months ago
What's Changed
TEI + TGI, dataclass configs, asyncio instead of threads
- refacored + syncio by @IlyasMoutawwakil in https://github.com/IlyasMoutawwakil/py-txi/pull/3
Full Changelog: https://github.com/IlyasMoutawwakil/py-txi/compare/v0.2.0...v0.5.0
Published by IlyasMoutawwakil 8 months ago
What's Changed
- Text embedding inference by @IlyasMoutawwakil in https://github.com/IlyasMoutawwakil/py-tgi/pull/2
Full Changelog: https://github.com/IlyasMoutawwakil/py-tgi/compare/v0.1.3...v0.2.0
Published by IlyasMoutawwakil 8 months ago
What's Changed
- Rocm and custom devices support
- PyPI at https://pypi.org/project/py-tgi/
Published by IlyasMoutawwakil 8 months ago
- CI testing on Ubuntu (cpu only).
- vLLM like interface.
- PyPI release 🥳
