modelfusion

Added

• Ollama model provider for text embeddings.

Fixed

• Llama.cpp embeddings are invoked sequentially to avoid rejection by the server.

Added

• Ollama model provider for text generation and text streaming.

Adding experimental ModelFusion server, flows, and browser utils.

Added

• ModelFusion server (separate export 'modelfusion/server') with a Fastify plugin for running ModelFusion flows on a server.
• ModelFusion flows.
• ModelFusion browser utils (separate export 'modelfusion/browser') for dealing with audio data and invoking ModelFusion flows on the server (invokeFlow).

Changed

• breaking change: readEventSource and readEventSourceStream are now part of 'modelfusion/browser'.

Added

• Prompt callback option for streamStructure

Improved

• Inline JSDoc comments for the model functions.

Fixed

• Abort signals and errors during streaming are caught and forwarded correctly.

Added

• executeFunction utility function for tracing execution time, parameters, and result of composite functions and non-ModelFusion functions.

Changed

• Streaming results and AsyncQueue objects can be used by several consumers. Each consumer will receive all values. This means that you can e.g. forward the same text stream to speech generation and the client.

ElevenLabs improvements.

Added

• ElevenLabs model settings outputFormat and optimizeStreamingLatency.

Fixed

• Default ElevenLabs model is eleven_monolingual_v1.

Added

• parentCallId event property
• Tracing for useTool, useToolOrGenerateText, upsertIntoVectorIndex, and guard

Changed

• breaking change: rename embedding event type to embed
• breaking change: rename image-generation event type to generate-image
• breaking change: rename speech-generation event type to generate-speech
• breaking change: rename speech-streaming event type to stream-speech
• breaking change: rename structure-generation event type to generate-structure
• breaking change: rename structure-or-text-generation event type to generate-structure-or-text
• breaking change: rename structure-streaming event type to stream-structure
• breaking change: rename text-generation event type to generate-text
• breaking change: rename text-streaming event type to stream-text
• breaking change: rename transcription event type to generate-transcription

Added

• Speech synthesis streaming supports string inputs.
• Observability for speech synthesis streaming.

Changes

• breaking change: split synthesizeSpeech into generateSpeech and streamSpeech functions
• breaking change: renamed speech-synthesis event to speech-generation
• breaking change: renamed transcribe to generateTranscription
• breaking change: renamed LmntSpeechSynthesisModel to LmntSpeechModel
• breaking change: renamed ElevenLabesSpeechSynthesisModel to ElevenLabsSpeechModel
• breaking change: renamed OpenAITextGenerationModel to OpenAICompletionModel

Removed

• breaking change: describeImage model function. Use generateText instead (with e.g. HuggingFaceImageDescriptionModel).

Added

• Duplex streaming for speech synthesis.
• Elevenlabs duplex streaming support.

Changed

• Schema is using data in return type (breaking change for tools).