Structured inference with Llama 2 in your browser
MIT License
Adds the
helper along with a
and makes preword
configurable. the
defaults to a question ("What is the ...") and a
defaults to generate ("Generate a ...") - the
is meant for information extraction like getting the name of a company from a listing, while a
is meant for generating something from scratch. You can change preword
at the context
level to overwrite 'Generate' and 'What is' for all helpers. prompt
ignores all of this and feeds in exactly what's provided.
This also exports a couple more types - types will probably be moved to a 'ad-llama/types' import path eventually.
Full Changelog: https://github.com/gsuuon/ad-llama/compare/v0.2.0...v0.2.1
Published by gsuuon about 1 year ago
I made a whoopsie with an automatic LSP rename and just letting tsc take the wheel. This fixes that!
Published by gsuuon about 1 year ago
This release adds the ability to modify logits before the sampling phase for each expression. You can now enforce that a field is only numbers, or that it's far more likely to be 'nun-chucks'.
The naive way of implementing this didn't add much accuracy to my desired inference outcomes - it turns out that we can't just encode our desired strings and modify the logits of those tokens, we have to encode them as they would appear in the existing (and upcoming) completion. I figured there were two main uses for modifying logits - one is to modify the odds of specific sequences, the other is to target specific character classes (eg. numbers). These require slightly different handling in terms of figuring out the desired tokens -- I'm not sure I got it right in this first pass, especially with character classes, but I'll keep improving it as I bump into bugs.
There's also an auto-generated docs site now: https://gsuuon.github.io/ad-llama
Hope it makes the API easier to understand!
Published by gsuuon about 1 year ago
This adds streaming and basic generation configuration, plus a slightly prettier demo page
https://github.com/gsuuon/ad-llama/assets/6422188/cefc22a8-6c4f-4c9d-bce3-21b6be7821d6
Published by gsuuon about 1 year ago
HMR the LLM - now this is prompt engineering
https://github.com/gsuuon/ad-llama/assets/6422188/b825a697-72bd-4322-826c-a56b6ec8a365