The TypeScript framework for AI development
MIT License
Bot releases are visible (Hide)
import { StreamToIterable } from '@axflow/models/shared';
import { GoogleGenerateContent } from '@axflow/models/google/generate-content';
const stream = await GoogleGenerateContent.stream(
{
model: 'gemini-pro',
contents: [
{
parts: [
{
text: 'Write a two sentence story about a magic backpack',
},
],
},
],
},
{
apiKey: process.env.GOOGLE_API_KEY,
}
);
for await (const chunk of StreamToIterable(stream)) {
console.log(chunk);
}
Published by benjreinhart 10 months ago
For example, we can run Llama2 70B on Together's inference endpoint with:
import { StreamToIterable } from '@axflow/models/shared';
import { TogetherAIInference } from '@axflow/models/togetherai/inference';
const stream = await TogetherAIInference.stream(
{
model: 'togethercomputer/llama-2-70b-chat',
prompt: '[INST] Using no more than 20 words, what is the Eiffel tower? [/INST] ',
max_tokens: 250,
},
{
apiKey: process.env.TOGETHERAI_API_KEY,
}
);
for await (const chunk of StreamToIterable(stream)) {
console.log(chunk);
}
Published by nichochar 11 months ago
Support openAI tools:
toolCallsAccessor
callbackPublished by nichochar 12 months ago
OpenAI made some changes to their SDK during their 2023 dev day, Axflow now supports them:
functions
to tools
json_mode
seed
and deterministic callsPublished by benjreinhart 12 months ago
Published by benjreinhart 12 months ago
Published by benjreinhart 12 months ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Introduce a streamJsonResponse
function that can pipe a ReadableStream
through a Node.js ServerResponse
object, enabling seamless integration for environments using the Node.js standard library, like Express.js.
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago
Published by benjreinhart about 1 year ago