catai
Run AI ✨ assistant locally! with simple API for Node.js 🚀
MIT License
Bot releases are visible (Hide)
catai
- v3.2.0
Published by github-actions[bot] 5 months ago
3.2.0 (2024-05-10)
Features
- better api support for node-llama-cpp@beta (0ecce70)
Development API + Node-llama-cpp@beta integration
You can use the model with node-llama-cpp@beta
CatAI enables you to easily manage the models and chat with them.
import {downloadModel, getModelPath} from 'catai';
// download the model, skip if you already have the model
await downloadModel(
"https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q2_K.gguf?download=true",
"llama3"
);
// get the model path with catai
const modelPath = getModelPath("llama3");
const llama = await getLlama();
const model = await llama.loadModel({
modelPath
});
const context = await model.createContext();
const session = new LlamaChatSession({
contextSequence: context.getSequence()
});
const a1 = await session.prompt("Hi there, how are you?");
console.log("AI: " + a1);
catai
- v3.1.0-beta.2
Published by github-actions[bot] 10 months ago
catai
- v3.1.0-beta.1
Published by github-actions[bot] 10 months ago
Package Rankings
Top 21.12% on Npmjs.org
Top 6.75% on Proxy.golang.org
Badges
Extracted from project README
