willow-inference-server

Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS

APACHE-2.0 License

Stars
375
Committers
7
View Code on GitHub View on X
Ecosystems: WebRTC, Whisper, Llama, Cuda

Commit Statistics

Past Year

All Time

Total Commits
11
421
Total Committers
4
10
Avg. Commits Per Committer
2.75
42.1
Bot Commits
0
0

Issue Statistics

Past Year

All Time

Total Pull Requests
1
61
Merged Pull Requests
0
51
Total Issues
3
134
Time to Close Issues
N/A
28 days
Related Projects
talkGPT4All

A voice chatbot based on GPT4All and talkGPT, running on your local pc!

01 Apr 2023 140
pytldr-oss

An open source, Gradio-based chatbot app that combines the best of retrieval augmented generation...

20 Jul 2024 29
lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable fo...

22 Jul 2023 1,967
kobold_assistant

Like ChatGPT's voice conversations with an AI, but entirely offline/private/trade-secret-friendly...

21 May 2023 129
autoshow

End-to-end scripting workflow to automatically generate show notes from audio/video transcripts w...

17 Apr 2024 28
cortex

Simplify and accelerate AI-powered application development with structured interfaces to models a...

24 Feb 2023 43
AI.Labs

openai chatgpt or local llm(llama.cpp gguf format)+TTS+STT+Word+Excel

10 Dec 2023 83
mlx_gguf_server

This is a FastAPI based LLM server. Load multiple LLM models (MLX or llama.cpp) simultaneously us...

22 Mar 2024 6
BambooAI

A lightweight library that leverages Language Models (LLMs) to enable natural language interactio...

07 May 2023 439
voice-chat-ai

🎙️ Speak with AI - Run locally using ollama or OpenAI - XTTS or OpenAI Speech or ElevenLabs

13 Jun 2024 78
Speech-AI-Forge

🍦 Speech-AI-Forge is a project developed around TTS generation model, implementing an API Server ...

01 Jun 2024 770
wllama

WebAssembly binding for llama.cpp - Enabling in-browser LLM inference

13 Mar 2024 371
botality-ii

telegram bot for self-hosted local inference of stable diffusion, text-to-speech and large langua...

11 Mar 2023 37
DocPOI_repo

A local chatbot for managing docs

23 Aug 2024 13
lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

20 Oct 2023 1,529