cappr

Breaking changes

None

New features

• cappr.huggingface.classify doesn't copy data if batch_size=1. Instead, it repeats a view of the data. This change saves memory for tasks where there are many completions. For example, in the Banking 77 demo, peak VRAM usage decreases by ~5 GB.

Bug fixes

None

Breaking changes

None

New features

• The agg_log_probs function can return a numpy array instead of list if there are a constant number of completions.

Bug fixes

None

Breaking changes

None

New features

• The agg_log_probs function applies func instead of np.exp ∘ func. So if you want average token log-probabilities, set func=np.mean

Bug fixes

None

Breaking changes

• There's no setup.py file, in case you were relying on that.

New features

None

Bug fixes

None

Breaking changes

• pip install cappr will no longer install openai, tiktoken. Install them yourself, or install them using pip install "cappr[openai]". For previous versions of cappr, if you needed to install cappr without these dependencies, you had to run:

python -m pip install \
    "numpy>=1.21.0" \
    "tqdm>=4.27.0" && \
python -m pip install --no-deps cappr

• cappr.openai.api.Model no longer includes the deprecated text-* models

New features

None

Bug fixes

None

Breaking changes

None

New features

• The axis of posterior_prob defaults to the last one

Bug fixes

None

Breaking changes

• None

New features

• You no longer need to instantiate your Llama CPP model with logits_all=True

Bug fixes

None

Breaking changes

• Setting the internal past attribute of the cache to None now will cause an error to be raised if you try to use it again. Please use the original model instead

New features

• Support LongLLaMA
• repr for cached model
• Don't check logits from Llama CPP

Bug fixes

None

Breaking changes

None

New features

• Slightly better docstrings

Bug fixes

None

Breaking changes

None

New features

• For online applications, you can cache the model once and use it later. See cappr.huggingface.classify.cache_model and cappr.llama_cpp.classify.cache_model. See the demonstration in the Craigslist Bargains demo

Bug fixes

None

Breaking changes

None

New features

None

Bug fixes

• Now works for openai>=1.2.0, and backwards compatibility fix for openai<1.0.0

Breaking changes

None

New features

None

Bug fixes

• For older versions of numpy, _examples functions now work

Breaking changes

• end_of_prompt must be a whitespace or empty string. (This was intended since v0.4.7, but I forgot to add a check for it)

New features

• cappr.openai is now compatible with OpenAI v1.0.0 (and is backwards compatible with previous versions). You can input a client object

Bug fixes

• cappr.huggingface.classify.token_logprobs now has an option to add a BOS token or not. Previously, it always added it if applicable, which is wrong for the (still highly experimental) discount feature

Breaking changes

None

New features

• cappr.llama_cpp.classify now takes an end_of_prompt=" " kwarg, which makes it compatible with models which use BPE tokenizers. (This change is backward compatible with SentencePiece tokenizers because they always separate the prompt and completion with a whitespace.)

Bug fixes

None

Breaking changes

• cappr.huggingface.classify_no_batch and cappr.huggingface.classify_no_cache_no_batch have been deprecated. Instead, set batch_size_completions=1 in cappr.huggingface.classify and cappr.huggingface.classify_no_cache, respectively. Apologies for releasing these half measures

New features

• cappr.huggingface.classify* can batch over completions to save memory. Set the keyword argument batch_size_completions
• cappr.huggingface.classify now allows for sub-prompt caching to reduce runtime. Cache shared instructions or exemplars for prompts using the new context manager. See this functionality in action in the Banking 77 demo

Bug fixes

None

Breaking changes

None

New features

• The prior can now be any sort of sequence, not just a Sequence or numpy array. So you can supply something like the following (from a pandas DataFrame):

  prior = (
      df_tr["correct_completion"]
      .value_counts(normalize=True)
      [completions]
  )
  

• token_logprobs functions now allow a str input for texts

Bug fixes

None

Breaking changes

None

New features

None

Bug fixes

• Previously, it was possible for an HF or Llama CPP function call to modify the inputted model/tokenizer if an exception were raised. (Technically, the cause is that my context managers didn't wrap the context in a try-finally block. Now they do.)

Breaking changes

• The default batch_size in cappr.huggingface is now 2, not 32
• The implementation for cappr.huggingface.classify_no_batch is now in cappr.huggingface.classify_no_batch_no_cache

New features

• cappr.huggingface.classify_no_batch now caches the prompt, which makes it much faster. It can also cache shared instructions or exemplars for prompts using the new context manager. See this functionality in action in the Banking 77 demo

Bug fixes

None

Breaking changes

None

New features

• For cappr.llama_cpp.classify, cache shared instructions or exemplars for many prompts using the new context manager.

Bug fixes

None

Breaking changes

None

New features

• Install all extra dependencies to run any model format using:

  pip install "cappr[all]"
  

Bug fixes

None