llm-demo

This repository demonstrates how to do inference with llama-2-7b-chat using llama.cpp on a machine with minimal specs.

Stars
3
Committers
1
View Code on GitHub
Ecosystems: Llama

Commit Statistics

Past Year

All Time

Total Commits
17
17
Total Committers
1
1
Avg. Commits Per Committer
17.0
17.0
Bot Commits
0
0

Issue Statistics

Past Year

All Time

Total Pull Requests
0
0
Merged Pull Requests
0
0
Total Issues
0
0
Time to Close Issues
N/A
N/A
Related Projects
Llama-2-Open-Source-LLM-CPU-Inference

Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

06 Jul 2023 945
LLaMA-2-hf-Chatbot

Chatbot from pretrained LLaMA-2 LLM model, fine-tuned with medical research papers using RAG (Ret...

06 Jun 2024 2
CASALIOY

♾️ toolkit for air-gapped LLMs on consumer-grade hardware

10 May 2023 229
lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable fo...

22 Jul 2023 1,967
llama.go

llama.go is like llama.cpp in pure Golang!

19 Mar 2023 1,245
botality-ii

telegram bot for self-hosted local inference of stable diffusion, text-to-speech and large langua...

11 Mar 2023 37
llm-api

Run any Large Language Model behind a unified API

02 Apr 2023 159
codellama

Inference code for CodeLlama models

24 Aug 2023 15,934
Enhancing-LLM-with-Jenkins-Knowledge

🚀 this project aims to develop an app using an existing open-source LLM with data collected for d...

20 May 2024 10
inferflow

Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

26 Dec 2023 235
llama.cpp

LLM inference in C/C++

10 Mar 2023 55,160
LlamaChat

Chat with your favourite LLaMA models in a native macOS app

26 Mar 2023 1,453
llama.mmengine

Training LLaMA language model with MMEngine! It supports LoRA fine-tuning!

02 Apr 2023 40
airllm

AirLLM 70B inference with single 4GB GPU

12 Jun 2023 4,536
llama

Inference code for Llama models

14 Feb 2023 53,568