👩🏫 Typed LLM Outputs in PHP
MIT License
Best in class LLMs are able to output JSON following a schema you provide, usually JSON-Schema. This significantly expands the ways you can leverage LLMs in your application!
Think of the input as:
And the output/outcome is whichever structure best matches your use case and domain.
The python instructor cookbook has interesting examples.
Instructrice is a PHP library that simplifies working with structured output from LLMs in a type-safe manner.
Features:
A Symfony Bundle is also available.
$ composer require adrienbrault/instructrice:@dev
use AdrienBrault\Instructrice\InstructriceFactory;
use AdrienBrault\Instructrice\LLM\Provider\Ollama;
use AdrienBrault\Instructrice\LLM\Provider\OpenAi;
use AdrienBrault\Instructrice\LLM\Provider\Anthropic;
$instructrice = InstructriceFactory::create(
defaultLlm: Ollama::HERMES2THETA_LLAMA3_8B,
apiKeys: [ // Unless you inject keys here, api keys will be fetched from environment variables
OpenAi::class => $openAiApiKey,
Anthropic::class => $anthropicApiKey,
],
);
use AdrienBrault\Instructrice\Attribute\Prompt;
class Character
{
// The prompt annotation lets you add instructions specific to a property
#[Prompt('Just the first name.')]
public string $name;
public ?string $rank = null;
}
$characters = $instructrice->getList(
Character::class,
'Colonel Jack O\'Neil walks into a bar and meets Major Samanta Carter. They call Teal\'c to join them.',
);
/*
dump($characters);
array:3 [
0 => Character^ {
+name: "Jack"
+rank: "Colonel"
}
1 => Character^ {
+name: "Samanta"
+rank: "Major"
}
2 => Character^ {
+name: "Teal'c"
+rank: null
}
]
*/
$character = $instructrice->get(
type: Character::class,
context: 'Colonel Jack O\'Neil.',
);
/*
dump($character);
Character^ {
+name: "Jack"
+rank: "Colonel"
}
*/
$label = $instructrice->get(
type: [
'type' => 'string',
'enum' => ['positive', 'neutral', 'negative'],
],
context: 'Amazing great cool nice',
prompt: 'Sentiment analysis',
);
/*
dump($label);
"positive"
*/
You can also use third party json schema libraries like goldspecdigital/oooas to generate the schema:
https://github.com/adrienbrault/instructrice/assets/611271/da69281d-ac56-4135-b2ef-c5e306a56de2
Provider | Environment Variables | Enum | API Key Creation URL |
---|---|---|---|
Ollama | OLLAMA_HOST |
Ollama | |
OpenAI | OPENAI_API_KEY |
OpenAi | API Key Management |
Anthropic | ANTHROPIC_API_KEY |
Anthropic | API Key Management |
Mistral | MISTRAL_API_KEY |
Mistral | API Key Management |
Fireworks AI | FIREWORKS_API_KEY |
Fireworks | API Key Management |
Groq | GROQ_API_KEY |
Groq | API Key Management |
Together AI | TOGETHER_API_KEY |
Together | API Key Management |
Deepinfra | DEEPINFRA_API_KEY |
Deepinfra | API Key Management |
Perplexity | PERPLEXITY_API_KEY |
Perplexity | API Key Management |
Anyscale | ANYSCALE_API_KEY |
Anyscale | API Key Management |
OctoAI | OCTOAI_API_KEY |
OctoAI | API Key Management |
The supported providers are Enums, which you can pass to the llm
argument of InstructriceFactory::create
:
use AdrienBrault\Instructrice\InstructriceFactory;
use AdrienBrault\Instructrice\LLM\Provider\OpenAi;
$instructrice->get(
...,
llm: OpenAi::GPT_4T, // API Key will be fetched from the OPENAI_API_KEY environment variable
);
Strategy | 📄 Text | 🧩 JSON | 🚀 Function |
---|
Commercial usage 💼 | ✅ Yes | ⚠️ Yes, but | ❌ Nope |
---|
💼 | ctx | Ollama | Mistral | Fireworks | Groq | Together | DeepInfra | Perplexity | Anyscale | OctoAI | |
---|---|---|---|---|---|---|---|---|---|---|---|
Mistral 7B | ✅ | 32k | 🧩 | 🧩 68/s | 📄 98/s | 📄 88/s !ctx=16k! | 🧩 | 🧩 | |||
Mixtral 8x7B | ✅ | 32k | 🧩 | 🧩 44/s | 🧩 237/s | 🚀 560/s | 🚀 99/s | 📄 119/s !ctx=16k! | 🧩 | 🧩 | |
Mixtral 8x22B | ✅ | 65k | 🧩 | 🧩 77/s | 🧩 77/s | 📄 52/s | 🧩 40/s | 📄 62/s !ctx=16k! | 🧩 | 🧩 | |
Phi-3-Mini-4K | ✅ | 4k | 🧩 | ||||||||
Phi-3-Mini-128K | ✅ | 128k | 🧩 | ||||||||
Phi-3-Medium-4K | ✅ | 4k | 🧩 | ||||||||
Phi-3-Medium-128K | ✅ | 128k | 🧩 | ||||||||
Qwen2 0.5B | ✅ | 32k | 🧩 | ||||||||
Qwen2 1.5B | ✅ | 32k | 🧩 | ||||||||
Qwen2 7B | ✅ | 128k | 🧩 | ||||||||
Llama3 8B | ⚠️ | 8k | 📄 | 🧩 280/s | 🚀 800/s | 📄 194/s | 🧩 133/s | 📄 121/s | 🧩 | 🧩 | |
Llama3 70B | ⚠️ | 8k | 🧩 | 🧩 116/s | 🚀 270/s | 📄 105/s | 🧩 26/s | 📄 42/s | 🧩 | 🧩 | |
Llama 3.1 8B | [⚠️][llama31_license] | 128k | 🧩 | 🧩 | 📄 | 📄 | 🧩 | 📄 | |||
Llama 3.1 70B | [⚠️][llama31_license] | 128k | 🧩 | 🧩 | 📄 | 📄 | 🧩 | 📄 | |||
Llama 3.1 405B | [⚠️][llama31_license] | 128k | 🧩 | 🧩 | 📄 | 📄 | 📄 | ||||
Gemma 7B | ⚠️ | 8k | 🚀 800/s | 📄 118/s | 🧩 64/s | 🧩 | |||||
DBRX | ⚠️ | 32k | 🧩 50/s | 📄 72/s | 🧩 | ||||||
Qwen2 72B | ⚠️ | 128k | 🧩 | ||||||||
Qwen1.5 32B | ⚠️ | 32k | 📄 | 🧩 | |||||||
Command R | ❌ | 128k | 📄 | ||||||||
Command R+ | ❌ | 128k | 📄 |
Throughputs from https://artificialanalysis.ai/leaderboards/providers .
💼 | ctx | Base | Ollama | Fireworks | Together | DeepInfra | OctoAI | |
---|---|---|---|---|---|---|---|---|
Hermes 2 Pro Mistral 7B | ✅ | Mistral 7B | 🧩 | 🧩 | 🧩 | |||
FireFunction V1 | ✅ | Mixtral 8x7B | 🚀 | |||||
WizardLM 2 7B | ✅ | Mistral 7B | 🧩 | |||||
WizardLM 2 8x22B | ✅ | Mixtral 8x7B | 📄 | 🧩 | 🧩 | |||
Capybara 34B | ✅ | 200k | Yi 34B | 🧩 | ||||
Hermes 2 Pro Llama3 8B | ⚠️ | Llama3 8B | 📄 | |||||
Hermes 2 Theta Llama3 8B | ⚠️ | Llama3 8B | 📄 | |||||
Dolphin 2.9 | ⚠️ | 8k | Llama3 8B | 🧩 | 📄 | 🧩 |
Provider | Model | ctx | |
---|---|---|---|
Mistral | Large | 32k | ✅ 26/s |
OpenAI | GPT-4o | 128k | 🚀 83/s |
OpenAI | GPT-4o mini | 128k | 🚀 140/s |
OpenAI | GPT-4 Turbo | 128k | 🚀 28/s |
OpenAI | GPT-3.5 Turbo | 16k | 🚀 72/s |
Anthropic | Claude 3 Haiku | 200k | 📄 88/s |
Anthropic | Claude 3 Sonnet | 200k | 📄 59/s |
Anthropic | Claude 3 Opus | 200k | 📄 26/s |
Gemini 1.5 Flash | 1000k | 🧩 136/s | |
Gemini 1.5 Pro | 1000k | 🧩 57/s | |
Perplexity | Sonar Small Chat | 16k | 📄 |
Perplexity | Sonar Small Online | 12k | 📄 |
Perplexity | Sonar Medium Chat | 16k | 📄 |
Perplexity | Sonar Medium Online | 12k | 📄 |
Throughputs from https://artificialanalysis.ai/leaderboards/providers .
Automate updating these tables by scraping https://artificialanalysis.ai , along with chatboard arena elo.? Would be a good use case / showcase of this library/cli?
If you want to use an Ollama model that is not available in the enum, you can use the Ollama::create
static method:
use AdrienBrault\Instructrice\LLM\LLMConfig;
use AdrienBrault\Instructrice\LLM\Cost;
use AdrienBrault\Instructrice\LLM\OpenAiJsonStrategy;
use AdrienBrault\Instructrice\LLM\Provider\Ollama;
$instructrice->get(
...,
llm: Ollama::create(
'codestral:22b-v0.1-q5_K_M', // check its license first!
32000,
),
);
You can also use any OpenAI compatible api by passing an LLMConfig:
use AdrienBrault\Instructrice\LLM\LLMConfig;
use AdrienBrault\Instructrice\LLM\Cost;
use AdrienBrault\Instructrice\LLM\OpenAiJsonStrategy;
$instructrice->get(
...,
llm: new LLMConfig(
uri: 'https://api.together.xyz/v1/chat/completions',
model: 'meta-llama/Llama-3-70b-chat-hf',
contextWindow: 8000,
label: 'Llama 3 70B',
provider: 'Together',
cost: Cost::create(0.9),
strategy: OpenAiJsonStrategy::JSON,
headers: [
'Authorization' => 'Bearer ' . $apiKey,
]
),
);
You may configure the LLM using a DSN:
openai
, openai-http
, anthropic
, google
model
is the model namecontext
is the context windowstrategy
is the strategy to use:
json
for json mode with the schema in the prompt onlyjson_with_schema
for json mode with probably the completion perfectly constrained to the schematool_any
tool_auto
tool_function
Examples:
use AdrienBrault\Instructrice\InstructriceFactory;
$instructrice = InstructriceFactory::create(
defaultLlm: 'openai://:[email protected]/v1/chat/completions?model=gpt-3.5-turbo&strategy=tool_auto&context=16000'
);
$instructrice->get(
...,
llm: 'openai-http://localhost:11434?model=adrienbrault/nous-hermes2theta-llama3-8b&strategy=json&context=8000'
);
$instructrice->get(
...,
llm: 'openai://:[email protected]/inference/v1/chat/completions?model=accounts/fireworks/models/llama-v3-70b-instruct&context=8000&strategy=json_with_schema'
);
$instructrice->get(
...,
llm: 'google://:[email protected]/v1beta/models?model=gemini-1.5-flash&context=1000000'
);
$instructrice->get(
...,
llm: 'anthropic://:[email protected]?model=claude-3-haiku-20240307&context=200000'
);
You may also implement LLMInterface.
Obviously inspired by instructor-php and instructor.
How is it different from instructor php?
Both libraries essentially do the same thing:
However, instructice differs with:
instructor-php
.Things to look into:
$client->request('GET', 'https://r.jina.ai/' . $url)
DSPy is very interesting. There are great ideas to be inspired by.
Ideally this library is good to prototype with, but can support more advanced extraction workflows with few shot examples, some sort of eval system, generating samples/output like DSPy, etc
Would be cool to have a CLI, that accepts a FQCN and a context.
instructrice get "App\Entity\Customer" "$(cat some_email_body.md)"
Autosave all input/schema/output in sqlite db. Like llm? Leverage that to test examples, add few shots, evals?