Completion After Prompt Probability. Make your LLM make a choice
APACHE-2.0 License
Bot releases are hidden (Show)
Published by kddubey about 1 month ago
None
cappr.huggingface.classify
doesn't copy data if batch_size=1
. Instead, it repeats a view of the data. This change saves memory for tasks where there are many completions. For example, in the Banking 77 demo, peak VRAM usage decreases by ~5 GB.None
Published by kddubey 3 months ago
None
agg_log_probs
function can return a numpy array instead of list if there are a constant number of completions.None
Published by kddubey 3 months ago
None
agg_log_probs
function applies func
instead of np.exp ∘ func
. So if you want average token log-probabilities, set func=np.mean
None
Published by kddubey 4 months ago
setup.py
file, in case you were relying on that.None
None
Published by kddubey 9 months ago
pip install cappr
will no longer install openai, tiktoken
. Install them yourself, or install them using pip install "cappr[openai]"
. For previous versions of cappr
, if you needed to install cappr
without these dependencies, you had to run:python -m pip install \
"numpy>=1.21.0" \
"tqdm>=4.27.0" && \
python -m pip install --no-deps cappr
cappr.openai.api.Model
no longer includes the deprecated text-*
models
None
None
Published by kddubey 11 months ago
Published by kddubey 11 months ago
logits_all=True
None
Published by kddubey 11 months ago
past
attribute of the cache to None
now will cause an error to be raised if you try to use it again. Please use the original model insteadrepr
for cached modelNone
Published by kddubey 11 months ago
None
None
Published by kddubey 11 months ago
None
cappr.huggingface.classify.cache_model
and cappr.llama_cpp.classify.cache_model
. See the demonstration in the Craigslist Bargains demo
None
Published by kddubey 12 months ago
None
None
openai>=1.2.0
, and backwards compatibility fix for openai<1.0.0
Published by kddubey 12 months ago
None
None
_examples
functions now workPublished by kddubey 12 months ago
end_of_prompt
must be a whitespace or empty string. (This was intended since v0.4.7, but I forgot to add a check for it)cappr.openai
is now compatible with OpenAI v1.0.0 (and is backwards compatible with previous versions). You can input a client
objectcappr.huggingface.classify.token_logprobs
now has an option to add a BOS token or not. Previously, it always added it if applicable, which is wrong for the (still highly experimental) discount featurePublished by kddubey 12 months ago
None
cappr.llama_cpp.classify
now takes an end_of_prompt=" "
kwarg, which makes it compatible with models which use BPE tokenizers. (This change is backward compatible with SentencePiece tokenizers because they always separate the prompt and completion with a whitespace.)None
Published by kddubey 12 months ago
cappr.huggingface.classify_no_batch
and cappr.huggingface.classify_no_cache_no_batch
have been deprecated. Instead, set batch_size_completions=1
in cappr.huggingface.classify
and cappr.huggingface.classify_no_cache
, respectively. Apologies for releasing these half measurescappr.huggingface.classify*
can batch over completions to save memory. Set the keyword argument batch_size_completions
cappr.huggingface.classify
now allows for sub-prompt caching to reduce runtime. Cache shared instructions or exemplars for prompts using the new context manager. See this functionality in action in the Banking 77 demo
None
Published by kddubey 12 months ago
None
The prior
can now be any sort of sequence, not just a Sequence
or numpy array. So you can supply something like the following (from a pandas DataFrame):
prior = (
df_tr["correct_completion"]
.value_counts(normalize=True)
[completions]
)
token_logprobs
functions now allow a str
input for texts
None
Published by kddubey 12 months ago
None
None
Published by kddubey 12 months ago
batch_size
in cappr.huggingface
is now 2
, not 32
cappr.huggingface.classify_no_batch
is now in cappr.huggingface.classify_no_batch_no_cache
cappr.huggingface.classify_no_batch
now caches the prompt, which makes it much faster. It can also cache shared instructions or exemplars for prompts using the new context manager. See this functionality in action in the Banking 77 demo
None
Published by kddubey 12 months ago
None
cappr.llama_cpp.classify
, cache shared instructions or exemplars for many prompts using the new context manager.None
Published by kddubey about 1 year ago
None
Install all extra dependencies to run any model format using:
pip install "cappr[all]"
None