3.1.1 (2024-10-06)

Features

• minor: reference common classes on the Llama instance (#360) (8145c94)

Shipped with llama.cpp release b3889

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.1.0 (2024-10-05)

Bug Fixes

• improve metadata read times (#351) (4ee10a9)
• hide internal type (#351) (4ee10a9)

Features

• resolveModelFile method (#351) (4ee10a9)
• hf: URI support (#351) (4ee10a9)

Shipped with llama.cpp release b3887

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

✨ node-llama-cpp 3.0 is here! ✨

Read about the release in the blog post

3.0.3 (2024-09-25)

Bug Fixes

• adapt to llama.cpp breaking change (#344) (2e751c8)

Shipped with llama.cpp release b3825

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

✨ node-llama-cpp 3.0 is here! ✨

Read about the release in the blog post

3.0.2 (2024-09-25)

Bug Fixes

• node template: bug (#342) (1291b97)
• use a compressed logo image for README.md (#340) (8ab983b)

Shipped with llama.cpp release b3821

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

✨ node-llama-cpp 3.0 is here! ✨

Read about the release in the blog post

3.0.1 (2024-09-24)

Bug Fixes

• deploy docs website (#337) (ec45bbf)
• release create-command package (#335) (51f4622)

Shipped with llama.cpp release b3808

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

✨ node-llama-cpp 3.0 is here! ✨

Read about the release in the blog post

3.0.0 (2024-09-24)

Features

• function calling (#139) (5fcdf9b)
• get embedding for text (#144) (4cf1fba)
• async model and context loading (#178) (315a3eb)
• token biases (#196) (3ad4494)
• automatic batching (#104) (4757af8)
• prompt completion engine (#225) (95f4645)
• model compatibility warnings (#225) (95f4645)
• Vulkan support (#171) (d161bcd)
• Windows on Arm prebuilt binary (#181) (f3b7f81)
• change the default log level to warn (#191) (b542b53)
• pull command (#214) (453c162)
• inspect gpu command (#175) (5a70576)
• inspect gguf command (#182) (35e6f50)
• inspect estimate command (#309) (4b3ad61)
• inspect measure command (#182) (35e6f50)
• init command to scaffold a new project from a template (with node-typescript and electron-typescript-react templates) (#217) (d6a0f43)
• move download, build and clear commands to be subcommands of a source command (#309) (4b3ad61)
• move seed option to the prompt level (#309) (4b3ad61)
• TemplateChatWrapper: custom history template for each message role (#309) (4b3ad61)
• Llama 3.1 support (#273) (e3e0994)
• Mistral chat wrapper (#309) (4b3ad61)
• Functionary v3 support (#309) (4b3ad61)
• Phi-3 support (#273) (e3e0994)
• extract all prebuilt binaries to external modules (#309) (4b3ad61)
• parallel function calling (#225) (95f4645)
• preload prompt (#225) (95f4645)
• onTextChunk option (#273) (e3e0994)
• flash attention (#264) (c2e322c)
• debug mode (#217) (d6a0f43)
• load LoRA adapters (#217) (d6a0f43)
• split gguf files support (#214) (453c162)
• stopOnAbortSignal and customStopTriggers on LlamaChat and LlamaChatSession (#214) (453c162)
• Llama 3 support (#205) (ef501f9)
• --gpu flag in generation CLI commands (#205) (ef501f9)
• specialTokens parameter on model.detokenize (#205) (ef501f9)
• interactively select a model from CLI commands (#191) (b542b53)
• automatically adapt to current free VRAM state (#182) (35e6f50)
• GGUF file metadata info on LlamaModel (#182) (35e6f50)
• use the tokenizer.chat_template header from the gguf file when available - use it to find a better specialized chat wrapper or use JinjaTemplateChatWrapper with it as a fallback (#182) (35e6f50)
• simplify generation CLI commands: chat, complete, infill (#182) (35e6f50)
• gguf parser (#168) (bcaab4f)
• use the best compute layer available by default (#175) (5a70576)
• more guardrails to prevent loading an incompatible prebuilt binary (#175) (5a70576)
• completion and infill (#164) (ede69c1)
• support configuring more options for getLlama when using "lastBuild" (#164) (ede69c1)
• get VRAM state (#161) (46235a2)
• chatWrapper getter on a LlamaChatSession (#161) (46235a2)
• minP support (#162) (47b476f)
• chat syntax aware context shifting (#139) (5fcdf9b)
• stateless LlamaChat (#139) (5fcdf9b)
• LlamaText util (#139) (5fcdf9b)
• show llama.cpp release in GitHub releases (#142) (36c779d)
• model metadata overrides (#273) (e3e0994)

Shipped with llama.cpp release b3808

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.0.0-beta.47 (2024-09-23)

Bug Fixes

• improve model downloader CI logs (#329) (4b7ef5b)

Features

• resetChatHistory function on a LlamaChatSession (#327) (ebc4e83)

Shipped with llama.cpp release b3804

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.0.0-beta.46 (2024-09-20)

Bug Fixes

• no thread limit when using a GPU (#322) (2204e7a)
• improve defineChatSessionFunction types and docs (#322) (2204e7a)
• format numbers printed in the CLI (#322) (2204e7a)
• revert electron-builder version used in Electron template (#323) (6c644ff)

Shipped with llama.cpp release b3787

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.0.0-beta.45 (2024-09-19)

Bug Fixes

• improve performance of parallel evaluation from multiple contexts (#309) (4b3ad61)
• Llama 3.1 chat wrapper standard chat history (#309) (4b3ad61)
• adapt to llama.cpp sampling refactor (#309) (4b3ad61)
• Llama 3 Instruct function calling (#309) (4b3ad61)
• don't preload prompt in the chat command when using --printTimings or --meter (#309) (4b3ad61)
• more stable Jinja template matching (#309) (4b3ad61)

Features

• inspect estimate command (#309) (4b3ad61)
• move seed option to the prompt level (#309) (4b3ad61)
• Functionary v3 support (#309) (4b3ad61)
• Mistral chat wrapper (#309) (4b3ad61)
• improve Llama 3.1 chat template detection (#309) (4b3ad61)
• change autoDisposeSequence default to false (#309) (4b3ad61)
• move download, build and clear commands to be subcommands of a source command (#309) (4b3ad61)
• simplify TokenBias (#309) (4b3ad61)
• better threads default value (#309) (4b3ad61)
• make LlamaEmbedding an object (#309) (4b3ad61)
• HF_TOKEN env var support for reading GGUF file metadata (#309) (4b3ad61)
• TemplateChatWrapper: custom history template for each message role (#309) (4b3ad61)
• more helpful inspect gpu command (#309) (4b3ad61)
• all tokenizer tokens iterator (#309) (4b3ad61)
• failed context creation automatic remedy (#309) (4b3ad61)
• abort generation support in CLI commands (#309) (4b3ad61)
• --gpuLayers max and --contextSize max flag support for inspect estimate command (#309) (4b3ad61)
• extract all prebuilt binaries to external modules (#309) (4b3ad61)
• updated docs (#309) (4b3ad61)
• combine model downloaders (#309) (4b3ad61)
• feat(electron example template): update badge, scroll anchoring, table support (#309) (4b3ad61)

Shipped with llama.cpp release b3785

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

2.8.16 (2024-09-03)

Bug Fixes

• bump llama.cpp release used in prebuilt binaries (#305) (660651a)
• update documentation website URL (#306) (51265c8)

3.0.0-beta.44 (2024-08-10)

Bug Fixes

• revert to the latest stable Metal llama.cpp release (#297) (bf12e9c)

Shipped with llama.cpp release b3543

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.43 (2024-08-09)

Bug Fixes

• more cases of unknown characters in generation streaming (#295) (ecaef63)

Shipped with llama.cpp release b3560

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.42 (2024-08-07)

Bug Fixes

• unkown characters in generation streaming (#293) (097b3ec)

Shipped with llama.cpp release b3541

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.15 (2024-08-06)

Bug Fixes

• adapt to llama.cpp breaking changes (#291) (c4b5d80)

3.0.0-beta.41 (2024-08-02)

Bug Fixes

• CUDA context creation (#285) (a2b2bc3)
• detokenizer unpredictable text length (#285) (a2b2bc3)

Shipped with llama.cpp release b3504

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.40 (2024-07-30)

Bug Fixes

• update model recommendations (#276) (826334b)

Features

• model downloader: use HF_TOKEN when needed (#276) (826334b)

Shipped with llama.cpp release b3488

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.39 (2024-07-28)

Bug Fixes

• Gemma chat wrapper bug (#273) (e3e0994)
• GGUF metadata nested key conflicts (#273) (e3e0994)
• adapt to llama.cpp breaking changes (#273) (e3e0994)
• preserve function calling chunks (#273) (e3e0994)
• format JSON objects like models expect (#273) (e3e0994)

Features

• Llama 3.1 support (#273) (e3e0994)
• Phi-3 support (#273) (e3e0994)
• model metadata overrides (#273) (e3e0994)
• use LoRA on a context instead of on a model (#273) (e3e0994)
• onTextChunk option (#273) (e3e0994)

Shipped with llama.cpp release b3479

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.14 (2024-07-26)

Bug Fixes

• prebuilt binaries (#272) (e539e5b)

2.8.13 (2024-07-26)

Bug Fixes

• adapt to llama.cpp breaking changes (#271) (7f15823)

3.0.0-beta.38 (2024-07-09)

Bug Fixes

• adapt to llama.cpp breaking changes (#266) (c35ff5a)
• Llama 3 Instruct function calling (#266) (c35ff5a)

Features

• flash attention in model selection (#266) (c35ff5a)

Shipped with llama.cpp release b3347

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)