The TypeScript library for building AI applications.
MIT License
Bot releases are visible (Hide)
Published by lgrammel 10 months ago
Tool call support for chat prompts. Assistant messages can contain tool calls, and tool messages can contain tool call results. Tool calls can be used to implement e.g. agents:
const chat: ChatPrompt = {
system: "You are ...",
messages: [ChatMessage.user({ text: instruction })],
};
while (true) {
const { text, toolResults } = await useToolsOrGenerateText(
openai
.ChatTextGenerator({ model: "gpt-4-1106-preview" })
.withChatPrompt(),
tools, // array of tools
chat
);
// add the assistant and tool messages to the chat:
chat.messages.push(
ChatMessage.assistant({ text, toolResults }),
ChatMessage.tool({ toolResults })
);
if (toolResults == null) {
return; // no more actions, break loop
}
// ... (handle tool results)
}
streamText
returns a text
promise when invoked with fullResponse: true
. After the streaming has finished, the promise resolves with the full text.
const { text, textStream } = await streamText(
openai.ChatTextGenerator({ model: "gpt-3.5-turbo" }).withTextPrompt(),
"Write a short story about a robot learning to love:",
{ fullResponse: true }
);
// ... (handle streaming)
console.log(await text); // full text
Published by lgrammel 10 months ago
[Text/MultiModal]InstructionPrompt
is now InstructionPrompt
, and [Text/MultiModalChatPrompt]
is now ChatPrompt
.Published by lgrammel 10 months ago
finishReason
support for generateText
.
The finish reason can be stop
(the model stopped because it generated a stop sequence), length
(the model stopped because it generated the maximum number of tokens), content-filter
(the model stopped because the content filter detected a violation), tool-calls
(the model stopped because it triggered a tool call), error
(the model stopped because of an error), other
(the model stopped for another reason), or unknown
(the model stop reason is not know or the model does not support finish reasons).
You can extract it from the full response when using fullResponse: true
:
const { text, finishReason } = await generateText(
openai
.ChatTextGenerator({ model: "gpt-3.5-turbo", maxGenerationTokens: 200 })
.withTextPrompt(),
"Write a short story about a robot learning to love:",
{ fullResponse: true }
);
Published by lgrammel 10 months ago
You can specify numberOfGenerations
on image generation models and create multiple images by using the fullResponse: true
option. Example:
// generate 2 images:
const { images } = await generateImage(
openai.ImageGenerator({
model: "dall-e-3",
numberOfGenerations: 2,
size: "1024x1024",
}),
"the wicked witch of the west in the style of early 19th century painting",
{ fullResponse: true }
);
breaking change: Image generation models use a generalized numberOfGenerations
parameter (instead of model specific parameters) to specify the number of generations.
Published by lgrammel 10 months ago
Published by lgrammel 10 months ago
ollama.ChatTextGenerator
model that calls the Ollama chat API.ollama.ChatMessage
and ollama.ChatPrompt
openai.ChatMessage
and openai.ChatPrompt
mistral.ChatMessage
and mistral.ChatPrompt
ollama.TextGenerator
to ollama.CompletionTextGenerator
mistral.TextGenerator
to mistral.ChatTextGenerator
Published by lgrammel 10 months ago
You can now specify numberOfGenerations
on text generation models and access multiple generations by using the fullResponse: true
option. Example:
// generate 2 texts:
const { texts } = await generateText(
openai.CompletionTextGenerator({
model: "gpt-3.5-turbo-instruct",
numberOfGenerations: 2,
maxGenerationTokens: 1000,
}),
"Write a short story about a robot learning to love:\n\n",
{ fullResponse: true }
);
breaking change: Text generation models now use a generalized numberOfGenerations
parameter (instead of model specific parameters) to specify the number of generations.
maxCompletionTokens
text generation model setting to maxGenerationTokens
.Published by lgrammel 10 months ago
breaking change: responseType
option was changed into fullResponse
option and now uses a boolean value to make discovery easy. The response values from the full response have been renamed for clarity. For base64 image generation, you can use the imageBase64
value from the full response:
const { imageBase64 } = await generateImage(model, prompt, {
fullResponse: true,
});
n:2
or higher now returns only the stream from the first choice.Published by lgrammel 10 months ago
breaking change: Ollama image (vision) support. This changes the Ollama prompt format. You can add .withTextPrompt()
to existing Ollama text generators to get a text prompt like before.
Vision example:
import { ollama, streamText } from "modelfusion";
const textStream = await streamText(
ollama.TextGenerator({
model: "bakllava",
maxCompletionTokens: 1024,
temperature: 0,
}),
{
prompt: "Describe the image in detail",
images: [image], // base-64 encoded png or jpeg
}
);
Published by lgrammel 10 months ago
Published by lgrammel 10 months ago
Published by lgrammel 10 months ago
Published by lgrammel 10 months ago
Published by lgrammel 10 months ago
Published by lgrammel 11 months ago
breaking change: setting global function observers and global logging has changed.
You can now call methods on a modelfusion
import:
import { modelfusion } from "modelfusion";
modelfusion.setLogFormat("basic-text");
Cleaned output when using detailed-object
log format.
Published by lgrammel 11 months ago
Whisper.cpp
transcription (speech-to-text) model support.
import { generateTranscription, whispercpp } from "modelfusion";
const data = await fs.promises.readFile("data/test.wav");
const transcription = await generateTranscription(whispercpp.Transcriber(), {
type: "wav",
data,
});
Published by lgrammel 11 months ago
Published by lgrammel 11 months ago
Published by lgrammel 11 months ago
maxValuesPerCall
setting for OpenAITextEmbeddingModel
to enable different configurations, e.g. for Azure. Thanks @nanotronic for the contribution!