Supplementary code for Stealing Part of a Production Language Model, Carlini et al., 2024
optimize_logit_queries
Implements logprob-free attacks on a known vector of logits (no API calls).
Use run_attacks.py to access all implemented attacks, described in the paper in varying levels of detail.
Running run_attacks.py without modification runs several methods on a small random vector of logits.
This directory is useful as a starting point for further research on logprob-free attacks.
Other, less directly useful code
query_logprobs_emulator.
Emulates the OpenAI API to test any logprob attack on a local model, as well as possible mitigation strategies.
(Could also use it on the OpenAI API before March 3 2024, but not anymore.)
Defaults to the logprob attack that uses top_logprobs - 1 tokens per query.
May be useful for research on mitigations and ways to bypass them.
distribution_logits
Very unpolished script investigating the distribution of logits over the vocabulary for various open-source models. Pythia seems to be an outlier with very low probabilities on the long tail of the vocabulary.
openai_api_intricacies
Verifying undocumented properties of logit_bias in the OpenAI API that are necessary for the attack to work as described in the paper.
Disclaimer
Note: this repo does not contain any parameters of OpenAI or Google proprietary models, nor any code that can directly extract weights from any API known to the authors.