3.0.0-beta.19 (2024-05-12)

Bug Fixes

• adapt to llama.cpp changes (#208) (29e8c67)

Features

• improve grammar support (#215) (d321fe3)
• improve JSON schema grammar (#215) (d321fe3)

Shipped with llama.cpp release b2861

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.18 (2024-05-09)

Bug Fixes

• more efficient max context size finding algorithm (#214) (453c162)
• make embedding-only models work correctly (#214) (453c162)
• perform context shift on the correct token index on generation (#214) (453c162)
• make context loading work for all models on Electron (#214) (453c162)

Features

• split gguf files support (#214) (453c162)
• pull command (#214) (453c162)
• stopOnAbortSignal and customStopTriggers on LlamaChat and LlamaChatSession (#214) (453c162)
• checkTensors parameter on loadModel (#214) (453c162)
• improve Electron support (#214) (453c162)

Shipped with llama.cpp release b2834

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.10 (2024-04-27)

Bug Fixes

• adapt to llama.cpp changes (#208) (29e8c67)

3.0.0-beta.17 (2024-04-24)

Bug Fixes

• FunctionaryChatWrapper bugs (#205) (ef501f9)
• function calling syntax bugs (#205) ([ef501f9]
• show GPU layers in the Model line in CLI commands (#205) ([ef501f9]
• refactor: rename LlamaChatWrapper to Llama2ChatWrapper

Features

• Llama 3 support (#205) (ef501f9)
• --gpu flag in generation CLI commands (#205) (ef501f9)
• specialTokens parameter on model.detokenize (#205) (ef501f9)

Shipped with llama.cpp release b2717

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.16 (2024-04-13)

Bug Fixes

• fallback to general chat wrapper (#197) (7878c8a)

Features

• inspect gpu command: print device names (#198) (5ca33c7)
• inspect gpu command: print env info (#202) (d332b77)
• download models using the CLI (#191) (b542b53)
• interactively select a model from CLI commands (#191) (b542b53)
• change the default log level to warn (#191) (b542b53)
• token biases (#196) (3ad4494)

Shipped with llama.cpp release b2665

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.15 (2024-04-04)

Bug Fixes

• create a context with no parameters (#188) (6267778)
• improve chat wrappers tokenization (#182) (35e6f50)
• use the new llama.cpp CUDA flag (#182) (35e6f50)
• adapt to breaking llama.cpp changes (#183) (6b012a6)

Features

• automatically adapt to current free VRAM state (#182) (35e6f50)
• inspect gguf command (#182) (35e6f50)
• inspect measure command (#182) (35e6f50)
• readGgufFileInfo function (#182) (35e6f50)
• GGUF file metadata info on LlamaModel (#182) (35e6f50)
• JinjaTemplateChatWrapper (#182) (35e6f50)
• use the tokenizer.chat_template header from the gguf file when available - use it to find a better specialized chat wrapper or use JinjaTemplateChatWrapper with it as a fallback (#182) (35e6f50)
• simplify generation CLI commands: chat, complete, infill (#182) (35e6f50)
• Windows on Arm prebuilt binary (#181) (f3b7f81)

Shipped with llama.cpp release b2608

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.9 (2024-03-21)

Bug Fixes

• adapt to breaking llama.cpp changes (#183) (6b012a6)

3.0.0-beta.14 (2024-03-16)

Bug Fixes

• DisposedError was thrown when calling .dispose() (#178) (315a3eb)
• adapt to breaking llama.cpp changes (#178) (315a3eb)

Features

• async model and context loading (#178) (315a3eb)
• automatically try to resolve Failed to detect a default CUDA architecture CUDA compilation error (#178) (315a3eb)
• detect cmake binary issues and suggest fixes on detection (#178) (315a3eb)

Shipped with llama.cpp release b2440

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.13 (2024-03-03)

Bug Fixes

• adapt to llama.cpp breaking change (#175) (5a70576)
• return user-defined llama tokens (#175) (5a70576)

Features

• gguf parser (#168) (bcaab4f)
• use the best compute layer available by default (#175) (5a70576)
• more guardrails to prevent loading an incompatible prebuilt binary (#175) (5a70576)
• inspect command (#175) (5a70576)
• GemmaChatWrapper (#175) (5a70576)
• TemplateChatWrapper (#175) (5a70576)

Shipped with llama.cpp release b2329

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.12 (2024-02-24)

Bug Fixes

• adapt to llama.cpp breaking changes (#166) (7450aae)
• asset links (#170) (d841fff)

Features

• Vulkan support (#171) (d161bcd)

Shipped with llama.cpp release b2254

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.8 (2024-02-23)

Bug Fixes

• asset links (#170) (d841fff)

2.8.7 (2024-02-18)

Bug Fixes

• adapt to llama.cpp breaking changes (#166) (7450aae)

3.0.0-beta.11 (2024-02-18)

Features

• completion and infill (#164) (ede69c1)
• support configuring more options for getLlama when using "lastBuild" (#164) (ede69c1)
• export resolveChatWrapperBasedOnWrapperTypeName (#165) (624fa30)

Shipped with llama.cpp release b2174

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.10 (2024-02-11)

Features

• get VRAM state (#161) (46235a2)
• chatWrapper getter on a LlamaChatSession (#161) (46235a2)
• minP support (#162) (47b476f)

Shipped with llama.cpp release b2127

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.6 (2024-02-06)

3.0.0-beta.9 (2024-02-05)

Bug Fixes

• don't block a node process from exiting (#157) (74fb35c)
• fix: respect logLevel and logger params when using "lastBuild" (#157) (74fb35c)
• fix: print logs on the same event loop cycle (#157) (74fb35c)

Shipped with llama.cpp release b2074

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.8 (2024-02-05)

Bug Fixes

• clone from bundle (#156) (d4a39f5)

Shipped with llama.cpp release b2060

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.7 (2024-02-05)

Bug Fixes

• build and postinstall bug (#155) (9b9677f)

Shipped with llama.cpp release b2060

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.6 (2024-02-04)

Bug Fixes

• release config (#154) (cc78961)

Features

• manual binding loading (#153) (0e4b8d2)

Shipped with llama.cpp release b2060

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.5 (2024-01-24)

Bug Fixes

• build Metal by default for Apple silicone devices (#150) (187627e)

Features

• minor: add more flags to the chat command (#149) (cab617a)

Shipped with llama.cpp release b1961

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)