3.0.0-beta.37 (2024-07-05)

Features

• flash attention (#264) (c2e322c)
• manually exclude specific GPU types from gpu: "auto" (#264) (c2e322c)

Shipped with llama.cpp release b3322

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.36 (2024-06-30)

Bug Fixes

• macOS prebuilt binaries (#259) (81e0575)

Shipped with llama.cpp release b3267

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.35 (2024-06-30)

Bug Fixes

• Linux prebuilt binaries (#257) (a7a2517)

Shipped with llama.cpp release b3266

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.34 (2024-06-30)

Bug Fixes

• Windows and CUDA bindings (#254) (da00a17)
• Windows build (#255) (37c4b26)

Shipped with llama.cpp release b3265

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.33 (2024-06-29)

Bug Fixes

• long LlamaText tokenization (#249) (9cab784)
• improve GGUF metadata reading performance (#249) (9cab784)
• adapt to breaking llama.cpp changes (#249) (9cab784)

Features

• move CUDA prebuilt binaries to dependency modules (#250) (8a92e31)

Shipped with llama.cpp release b3265

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.12 (2024-06-21)

Bug Fixes

• bump llama.cpp release used in prebuilt binaries (#247) (2137c46)

3.0.0-beta.32 (2024-06-18)

Bug Fixes

• preload error on chat command (#245) (c89178f)

Shipped with llama.cpp release b3166

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.31 (2024-06-17)

Bug Fixes

• remove CUDA binary compression for Windows (#243) (0b85800)
• improve inspect gpu command output (#243) (0b85800)

Shipped with llama.cpp release b3166

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.30 (2024-06-17)

Bug Fixes

• avoid duplicate context shifts (#241) (1e7c5d0)
• onProgress on ModelDownloader (#241) (1e7c5d0)
• re-enable CUDA binary compression (#241) (1e7c5d0)
• more thorough tests before loading a binary (#241) (1e7c5d0)
• increase compatibility of prebuilt binaries (#241) (1e7c5d0)

Shipped with llama.cpp release b3166

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.29 (2024-06-16)

Bug Fixes

• remove CUDA binary compression for now (#238) (0d40ffc)

Shipped with llama.cpp release b3153

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.28 (2024-06-15)

Features

• compress CUDA prebuilt binaries (#236) (b89ad2d)
• automatically solve more CUDA compilation errors (#236) (b89ad2d)

Shipped with llama.cpp release b3153

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.27 (2024-06-12)

Features

• render markdown in the Electron example (#234) (23012d1)

Shipped with llama.cpp release b3135

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.26 (2024-06-11)

Bug Fixes

• async gpu info getters (#232) (56749c7)

Shipped with llama.cpp release b3135

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.25 (2024-06-10)

Bug Fixes

• Electron example build (#230) (73cabcd)

Shipped with llama.cpp release b3091

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.24 (2024-06-09)

Bug Fixes

• Electron example build (#228) (a5f3596)
• rename llamaBins directory to bins (#228) (a5f3596)

Shipped with llama.cpp release b3091

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.23 (2024-06-09)

Bug Fixes

• improve CUDA detection on Windows (#225) (95f4645)
• performance improvements (#225) (95f4645)

Features

• parallel function calling (#225) (95f4645)
• preload prompt (#225) (95f4645)
• prompt completion engine (#225) (95f4645)
• chat wrapper based system message support (#225) (95f4645)
• add prompt completion to the Electron example (#225) (95f4645)
• model compatibility warnings (#225) (95f4645)
• Functionary v2.llama3 support (#225) (95f4645)
• parallel function calling with plain Llama 3 Instruct (#225) (95f4645)
• improve function calling support for default chat wrapper (#225) (95f4645)
• parallel model downloads (#225) (95f4645)
• improve the electron example (#225) (95f4645)
• customStopTriggers for LlamaCompletion (#225) (95f4645)
• improve loading status in the Electron example (#226) (4ea0c3c)

Shipped with llama.cpp release b3091

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

2.8.11 (2024-05-24)

Bug Fixes

• bump llama.cpp release used in prebuilt binaries (#223) (81a203e)

3.0.0-beta.22 (2024-05-19)

Bug Fixes

• templates bugs (#221) (6619b28)

Shipped with llama.cpp release b2929

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.21 (2024-05-19)

Bug Fixes

• include templates in npm package (#219) (ba03ca9)

Shipped with llama.cpp release b2929

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)

3.0.0-beta.20 (2024-05-19)

Bug Fixes

• improve binary compatibility detection on Linux (#217) (d6a0f43)

Features

• init command to scaffold a new project from a template (with node-typescript and electron-typescript-react templates) (#217) (d6a0f43)
• debug mode (#217) (d6a0f43)
• load LoRA adapters (#217) (d6a0f43)
• improve Electron support (#217) (d6a0f43)

Shipped with llama.cpp release b2928

To use the latest llama.cpp release available, run npx --no node-llama-cpp download --release latest. (learn more)