ad-llama

What's Changed

• Solidjs demo by @gsuuon in https://github.com/gsuuon/ad-llama/pull/6

Adds the helper along with a and makes preword configurable. the defaults to a question ("What is the ...") and a defaults to generate ("Generate a ...") - the is meant for information extraction like getting the name of a company from a listing, while a is meant for generating something from scratch. You can change preword at the context level to overwrite 'Generate' and 'What is' for all helpers. prompt ignores all of this and feeds in exactly what's provided.

This also exports a couple more types - types will probably be moved to a 'ad-llama/types' import path eventually.

Full Changelog: https://github.com/gsuuon/ad-llama/compare/v0.2.0...v0.2.1

I made a whoopsie with an automatic LSP rename and just letting tsc take the wheel. This fixes that!

This release adds the ability to modify logits before the sampling phase for each expression. You can now enforce that a field is only numbers, or that it's far more likely to be 'nun-chucks'.

The naive way of implementing this didn't add much accuracy to my desired inference outcomes - it turns out that we can't just encode our desired strings and modify the logits of those tokens, we have to encode them as they would appear in the existing (and upcoming) completion. I figured there were two main uses for modifying logits - one is to modify the odds of specific sequences, the other is to target specific character classes (eg. numbers). These require slightly different handling in terms of figuring out the desired tokens -- I'm not sure I got it right in this first pass, especially with character classes, but I'll keep improving it as I bump into bugs.

There's also an auto-generated docs site now: https://gsuuon.github.io/ad-llama

Hope it makes the API easier to understand!

This adds streaming and basic generation configuration, plus a slightly prettier demo page

https://github.com/gsuuon/ad-llama/assets/6422188/cefc22a8-6c4f-4c9d-bce3-21b6be7821d6

HMR the LLM - now this is prompt engineering

https://github.com/gsuuon/ad-llama/assets/6422188/b825a697-72bd-4322-826c-a56b6ec8a365