langstream

What's Changed

• Remove agent context from Python API by @cbornet in https://github.com/LangStream/langstream/pull/441
• [docker run] Fix loading of OpenAI Async client - use the Netty provider instead of okhttp by @eolivelli in https://github.com/LangStream/langstream/pull/442
• [vector-db-sink] Ensure that we load the latest Cassandra driver by @eolivelli in https://github.com/LangStream/langstream/pull/443

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.20...v0.0.21

What's Changed

• Fix typo in error message by @cdbartholomew in https://github.com/LangStream/langstream/pull/424
• Add gRPC processor agent by @cbornet in https://github.com/LangStream/langstream/pull/406
• Introduce deletion-mode for assets by @nicoloboschi in https://github.com/LangStream/langstream/pull/421
• [tests] Add ai-chat-completions e2e test by @nicoloboschi in https://github.com/LangStream/langstream/pull/430
• WebCrawler: improvements by @eolivelli in https://github.com/LangStream/langstream/pull/422
• Fix robots.txt WebCrawler with real websites (with default http port) by @eolivelli in https://github.com/LangStream/langstream/pull/433
• OpenAI: use the default url if the url is empty by @eolivelli in https://github.com/LangStream/langstream/pull/434
• Build: do not download Kafka and Minio from docker build (speed up build on M1) by @eolivelli in https://github.com/LangStream/langstream/pull/435
• Fix topics runtime classloader by @nicoloboschi in https://github.com/LangStream/langstream/pull/437
• Move to the OpenAI Async client by @eolivelli in https://github.com/LangStream/langstream/pull/436
• Improved streaming chat output by @cdbartholomew in https://github.com/LangStream/langstream/pull/438

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.19...v0.0.20

What's Changed

• Update needs-doc.yml by @mendonk in https://github.com/LangStream/langstream/pull/415
• [gateway] Load the Topic Runtime from nar files in order to not pollute the classpath by @eolivelli in https://github.com/LangStream/langstream/pull/414
• CLI: local run without passing secrets by @nicoloboschi in https://github.com/LangStream/langstream/pull/416
• Split topic/assets deploy in a dedicated job by @nicoloboschi in https://github.com/LangStream/langstream/pull/412
• Fix CLI dependencies download and enable Cassandra Sink e2e test by @eolivelli in https://github.com/LangStream/langstream/pull/400
• Introduce deletion-mode for topics by @nicoloboschi in https://github.com/LangStream/langstream/pull/418
• Upgrade OpenAI client by @eolivelli in https://github.com/LangStream/langstream/pull/419
• [docker run] Some improvements by @eolivelli in https://github.com/LangStream/langstream/pull/420

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.18...v0.0.19

What's Changed

• [cli] "langstream run docker" command: start the webservices and other improvements by @eolivelli in https://github.com/LangStream/langstream/pull/410
• [docker run] Make the version configurable by @eolivelli in https://github.com/LangStream/langstream/pull/411
• [kafka] Support writing messages with different schemas by @eolivelli in https://github.com/LangStream/langstream/pull/413

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.17...v0.0.18

What's Changed

• [cli] Add a tool to run locally an application on docker by @eolivelli in https://github.com/LangStream/langstream/pull/393
• [cli] docker run, make Minio work by @eolivelli in https://github.com/LangStream/langstream/pull/394
• Internal refactor for Gateway by @nicoloboschi in https://github.com/LangStream/langstream/pull/397
• Refactor: move Kafka and Pulsar clients to nar files by @eolivelli in https://github.com/LangStream/langstream/pull/395
• Enhance the PGSQL example by @eolivelli in https://github.com/LangStream/langstream/pull/399
• Change the default provider to OpenAI by @cdbartholomew in https://github.com/LangStream/langstream/pull/401
• Improve helm install example by @nicoloboschi in https://github.com/LangStream/langstream/pull/405
• [agentrunner] Split the docker image and clean up the logs by @eolivelli in https://github.com/LangStream/langstream/pull/404
• Use least privilege service account for the runtime pod by @nicoloboschi in https://github.com/LangStream/langstream/pull/403
• Ensure existing agents are not affected by deployer config changes by @nicoloboschi in https://github.com/LangStream/langstream/pull/396
• Add workflow for needs-doc label by @mendonk in https://github.com/LangStream/langstream/pull/407
• Introduce new gateway type: 'chat' by @nicoloboschi in https://github.com/LangStream/langstream/pull/398
• Code storage: Azure Blob integration by @nicoloboschi in https://github.com/LangStream/langstream/pull/408

New Contributors

• @mendonk made their first contribution in https://github.com/LangStream/langstream/pull/407

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.16...v0.0.17

What's Changed

• Add serialization of dict and list to json bytes in python Kafka by @cbornet in https://github.com/LangStream/langstream/pull/388
• [resources] Introduce planner side validation of resources by @eolivelli in https://github.com/LangStream/langstream/pull/387
• Add support for Kafka UUID Serdes in Python by @cbornet in https://github.com/LangStream/langstream/pull/390
• Allow to return tuples instead of Records in Python agents by @cbornet in https://github.com/LangStream/langstream/pull/391
• Resource: add static validation for all the known resource types by @eolivelli in https://github.com/LangStream/langstream/pull/392
• Enable test mode on API gateway by @nicoloboschi in https://github.com/LangStream/langstream/pull/389

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.15...v0.0.16

What's Changed

• Fix flaky Python tests by @cbornet in https://github.com/LangStream/langstream/pull/385
• Add /java/lib to the unignored files by @cbornet in https://github.com/LangStream/langstream/pull/384
• [cli] Fix recursion in case of local parse error and the astra.yaml template by @eolivelli in https://github.com/LangStream/langstream/pull/386

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.14...v0.0.15

What's Changed

• Cli: allow to pass remote file - HTTPS by @nicoloboschi in https://github.com/LangStream/langstream/pull/381
• [cli] Resolve ENV placeholders in secrets.yaml with a bash-like syntax by @eolivelli in https://github.com/LangStream/langstream/pull/380
• Cli: allow to pass remote file - Github by @nicoloboschi in https://github.com/LangStream/langstream/pull/382
• Add support for a .langstreamignore file by @cbornet in https://github.com/LangStream/langstream/pull/383

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.13...v0.0.14

What's Changed

• [new feature] Implement Assets - part 1 by @eolivelli in https://github.com/LangStream/langstream/pull/359
• Replace serializer by one inferred from type if it's ByteArraySerializer by @cbornet in https://github.com/LangStream/langstream/pull/365
• Update langchain example by @cbornet in https://github.com/LangStream/langstream/pull/367
• Support consumer deserializer config in Python kafka connector by @cbornet in https://github.com/LangStream/langstream/pull/364
• Fix Java Kafka connector commit by @cbornet in https://github.com/LangStream/langstream/pull/369
• Reformat Python examples by @cbornet in https://github.com/LangStream/langstream/pull/370
• Implement gateway events by @nicoloboschi in https://github.com/LangStream/langstream/pull/363
• Default Kafka producer serializer to ByteArraySerializer by @cbornet in https://github.com/LangStream/langstream/pull/372
• Cli: add profiles by @nicoloboschi in https://github.com/LangStream/langstream/pull/371
• Implement Cassandra Table Assets by @eolivelli in https://github.com/LangStream/langstream/pull/366
• Cli: use global config file by default by @nicoloboschi in https://github.com/LangStream/langstream/pull/373
• Cli: ensure JDK11 compatibility by @nicoloboschi in https://github.com/LangStream/langstream/pull/376
• Cassandra Assets: improve examples and had astra-keyspace asset by @eolivelli in https://github.com/LangStream/langstream/pull/375
• Vector DB Support: download automatically the AstraDB secure bundle by @eolivelli in https://github.com/LangStream/langstream/pull/377
• Astra support: Upgrade Astra SDK to 0.6.9 by @eolivelli in https://github.com/LangStream/langstream/pull/378
• Fix cli using jdk11 by @nicoloboschi in https://github.com/LangStream/langstream/pull/379

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.12...v0.0.13

What's Changed

• Fix CLI test creating a new file in the source dir by @nicoloboschi in https://github.com/LangStream/langstream/pull/338
• [agents] compute-embeddings: Implement batch and async execution by @eolivelli in https://github.com/LangStream/langstream/pull/337
• Set default unit memory to 512M by @nicoloboschi in https://github.com/LangStream/langstream/pull/339
• Add possibility to commit on Kafka consumer out of order by @cbornet in https://github.com/LangStream/langstream/pull/340
• [cli] Allow to refer to local files in secrets.yaml and in instance.yaml using file:xxx syntax by @eolivelli in https://github.com/LangStream/langstream/pull/341
• Cassandra Sink: set 2 resource units in all the examples by @eolivelli in https://github.com/LangStream/langstream/pull/342
• Add Kafka consumer commit info by @cbornet in https://github.com/LangStream/langstream/pull/349
• Fix Python Kafka offsets and add tests by @cbornet in https://github.com/LangStream/langstream/pull/343
• Add test about Kafka topic retention by @eolivelli in https://github.com/LangStream/langstream/pull/350
• [agents] TextNormalizer: remove repeated sequences of space + new line by @eolivelli in https://github.com/LangStream/langstream/pull/348
• Upgrade only executors actually impacted by the app changes by @nicoloboschi in https://github.com/LangStream/langstream/pull/347
• [gateway][cli][controlplane][api] Improvements on startup and new GET Application API without stats by @eolivelli in https://github.com/LangStream/langstream/pull/344
• Handle Python Kafka producer errors by @cbornet in https://github.com/LangStream/langstream/pull/356
• Implement fully asynchronous Sinks and graceful pod shutdown (Java runtime) by @eolivelli in https://github.com/LangStream/langstream/pull/354
• TextNormalizer: improvements by @eolivelli in https://github.com/LangStream/langstream/pull/358
• Do not build the schema registry client if not needed by @cbornet in https://github.com/LangStream/langstream/pull/361
• Support producer serializer config in Python kafka connector by @cbornet in https://github.com/LangStream/langstream/pull/357
• Add Python logging with same format as Java by @cbornet in https://github.com/LangStream/langstream/pull/360

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.11...v0.0.12

What's Changed

• Agents: document-to-json: handle copy-properties while reading directly from a topic by @eolivelli in https://github.com/LangStream/langstream/pull/320
• [agents] Chat Completion: allow to override the completion-field for the streaming response (streaming-response-completion-field) by @eolivelli in https://github.com/LangStream/langstream/pull/321
• Deployer: add application resource limits per tenant by @nicoloboschi in https://github.com/LangStream/langstream/pull/314
• Chat completion: fix the demo apps and make streaming response work with CompositeAgentProcessor by @eolivelli in https://github.com/LangStream/langstream/pull/322
• CI: split runtime tests by @nicoloboschi in https://github.com/LangStream/langstream/pull/323
• Control plane: add application resource limits per tenant (part2) by @nicoloboschi in https://github.com/LangStream/langstream/pull/316
• Agent pods: set resource limits by @nicoloboschi in https://github.com/LangStream/langstream/pull/328
• Read resource max units from tenant metadata by @nicoloboschi in https://github.com/LangStream/langstream/pull/326
• Fix Python Kafka commit with empty list by @cbornet in https://github.com/LangStream/langstream/pull/331
• Rename python-function to python-processor by @cbornet in https://github.com/LangStream/langstream/pull/332
• Add example using llama-index by @cbornet in https://github.com/LangStream/langstream/pull/327
• Fix application status while deploying by @nicoloboschi in https://github.com/LangStream/langstream/pull/330
• Add retries on admin http client by @nicoloboschi in https://github.com/LangStream/langstream/pull/329
• Fix tenants creation while deleting is in progress by @nicoloboschi in https://github.com/LangStream/langstream/pull/305
• Expose metrics endpoint for all the services by @nicoloboschi in https://github.com/LangStream/langstream/pull/333
• [configuration] Allow to use global variables for topic names by @eolivelli in https://github.com/LangStream/langstream/pull/335
• Add CLI version (langstream -V) by @nicoloboschi in https://github.com/LangStream/langstream/pull/336
• Deployer: configure leader election by @nicoloboschi in https://github.com/LangStream/langstream/pull/334

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.10...v0.0.11

What's Changed

• Fix docker/build.sh script by @nicoloboschi in https://github.com/LangStream/langstream/pull/319
• WebCrawler enhancements: forbidden users and auto-reindex by @eolivelli in https://github.com/LangStream/langstream/pull/317

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.9...v0.0.10

What's Changed

• Allow asynchronous processing in AgentProcessor and in AgentSinks by @eolivelli in https://github.com/LangStream/langstream/pull/286
• Enable Spotless Maven Plugin by @eolivelli in https://github.com/LangStream/langstream/pull/293
• Agent runtime images should be always in-sync with the deployer config by @nicoloboschi in https://github.com/LangStream/langstream/pull/295

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.7...v0.0.8

What's Changed

• Fix OpenID discovery on GCP by @nicoloboschi in https://github.com/LangStream/langstream/pull/288
• Fix python release action by @cbornet in https://github.com/LangStream/langstream/pull/290
• Add error handling in Python Sinks by @cbornet in https://github.com/LangStream/langstream/pull/289
• Fix OpenID discovery on GCP - part2 by @nicoloboschi in https://github.com/LangStream/langstream/pull/291

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.6...v0.0.7

What's Changed

• Remove support for Pulsar IO Compute Cluster by @eolivelli in https://github.com/LangStream/langstream/pull/249
• Add cassio and llama-index deps by @cbornet in https://github.com/LangStream/langstream/pull/250
• Enable e2e tests on CI by @nicoloboschi in https://github.com/LangStream/langstream/pull/243
• Control plane/CLI: add endpoint to download application's code by @nicoloboschi in https://github.com/LangStream/langstream/pull/253
• Package system agents with isolated classpath (using NiFi Nar) by @eolivelli in https://github.com/LangStream/langstream/pull/252
• Add CLI autocomplete instructions for Bash by @nicoloboschi in https://github.com/LangStream/langstream/pull/257
• AgentCodeRegistry/NarFileHandler: do not unpack unecessary nar files by @eolivelli in https://github.com/LangStream/langstream/pull/259
• Update README.md with links to Slack, docs, and code extension by @cdbartholomew in https://github.com/LangStream/langstream/pull/260
• Python libs and PYTHONPATH cleanup by @cbornet in https://github.com/LangStream/langstream/pull/264
• Create Java Admin client module by @nicoloboschi in https://github.com/LangStream/langstream/pull/255
• Bump python runtime deps (llama_index to 0.8.8) by @cbornet in https://github.com/LangStream/langstream/pull/266
• Text Normaliser: improvements by @eolivelli in https://github.com/LangStream/langstream/pull/268
• Fix Kafka Connect after loader refactor and E2E test with Kafka Connect by @eolivelli in https://github.com/LangStream/langstream/pull/262
• Cli installation: get latest version by default by @nicoloboschi in https://github.com/LangStream/langstream/pull/272
• Runtime images should be always in-sync with the deployer config by @nicoloboschi in https://github.com/LangStream/langstream/pull/265
• Vector DB Sink: implement for Apache Cassandra and Astra DB by @eolivelli in https://github.com/LangStream/langstream/pull/270
• Add /app-code-download/python/lib to the PYTHONPATH by @cbornet in https://github.com/LangStream/langstream/pull/273
• Move runtime pods from args to env by @nicoloboschi in https://github.com/LangStream/langstream/pull/267
• Import utility classes into top-level module by @cbornet in https://github.com/LangStream/langstream/pull/274
• Code cleanup by @cbornet in https://github.com/LangStream/langstream/pull/277
• Update DataStax AI functions to 3.1.8 by @eolivelli in https://github.com/LangStream/langstream/pull/279
• Use SLASHSTAR style for Java copyright by @cbornet in https://github.com/LangStream/langstream/pull/276
• test python ci release by @cbornet in https://github.com/LangStream/langstream/pull/282
• Revert "test python ci release" by @cbornet in https://github.com/LangStream/langstream/pull/283
• WebCrawler: improvements by @eolivelli in https://github.com/LangStream/langstream/pull/280
• Move runtime pod to download code using the control plane by @nicoloboschi in https://github.com/LangStream/langstream/pull/263
• Add GH action to publish to pypi by @cbornet in https://github.com/LangStream/langstream/pull/285
• Sinks: handle errors by @eolivelli in https://github.com/LangStream/langstream/pull/284
• Fixes using OIDC on AWS by @nicoloboschi in https://github.com/LangStream/langstream/pull/287

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.5...v0.0.6

What's Changed

• Parse JAAS config in Python runtime to configure SASL by @cbornet in https://github.com/LangStream/langstream/pull/247
• Vector Database Support: vector-db-sink agent + support for Pinecone by @eolivelli in https://github.com/LangStream/langstream/pull/248

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.4...v0.0.5

What's Changed

• Improve CLI gateway error messages by @nicoloboschi in https://github.com/LangStream/langstream/pull/236
• Merge readme updates from PR 235 by @cdbartholomew in https://github.com/LangStream/langstream/pull/241
• Fix Python runtime module import by @nicoloboschi in https://github.com/LangStream/langstream/pull/239
• Move docker images to Github Container Registry by @nicoloboschi in https://github.com/LangStream/langstream/pull/244
• query-vector-db: Pinecone and AstraDB support by @eolivelli in https://github.com/LangStream/langstream/pull/240
• Deployer: set infinite retries by @nicoloboschi in https://github.com/LangStream/langstream/pull/245

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.3...v0.0.4

What's Changed

• Kubernetes Compute Runtime: allow to compose GenAI Toolkit agents with the other agents by @eolivelli in https://github.com/LangStream/langstream/pull/178
• Ensure that agents with different errors/resources spec cannot be merged by @eolivelli in https://github.com/LangStream/langstream/pull/179
• Fix deployer helm chart so that runtime picks the correct docker image by @dave2wave in https://github.com/LangStream/langstream/pull/180
• Update DataStax AI to 3.1.6 to pick Query filtering features by @eolivelli in https://github.com/LangStream/langstream/pull/182
• Add examples directory to sga-cli docker image by @dave2wave in https://github.com/LangStream/langstream/pull/186
• Cli: always include downloaded dependencies by @nicoloboschi in https://github.com/LangStream/langstream/pull/193
• Validate application/agent id to be compliant with k8s resources by @nicoloboschi in https://github.com/LangStream/langstream/pull/196
• Bump org.eclipse.jetty:jetty-server from 11.0.3 to 11.0.14 by @dependabot in https://github.com/LangStream/langstream/pull/197
• Bump com.google.guava:guava from 31.0.1-jre to 32.0.0-jre by @dependabot in https://github.com/LangStream/langstream/pull/198
• Fix gateway concurrent consumers by @nicoloboschi in https://github.com/LangStream/langstream/pull/200
• Fix null agentId in composite agents status by @nicoloboschi in https://github.com/LangStream/langstream/pull/201
• Make agent/deployer pod template configurable (node pool selector) by @nicoloboschi in https://github.com/LangStream/langstream/pull/205
• skip/fail/dead-letter in Kafka Connect Sinks by @dlg99 in https://github.com/LangStream/langstream/pull/190
• Refactor helm chart to avoid redundant values by @nicoloboschi in https://github.com/LangStream/langstream/pull/199
• Add error handler in python by @cbornet in https://github.com/LangStream/langstream/pull/206
• Rename python module to langstream by @cbornet in https://github.com/LangStream/langstream/pull/208
• Report the definition of the runners even when the pods are down by @eolivelli in https://github.com/LangStream/langstream/pull/203
• Finalise the Get application API by @eolivelli in https://github.com/LangStream/langstream/pull/209
• Remove instance and secrets from the app zip by @nicoloboschi in https://github.com/LangStream/langstream/pull/119
• Align the CLI to the new API by @eolivelli in https://github.com/LangStream/langstream/pull/210
• Rename some package to ai.langstream by @eolivelli in https://github.com/LangStream/langstream/pull/215
• Move WebService and Gateway packages to ai.langstream by @eolivelli in https://github.com/LangStream/langstream/pull/216
• AgentRunner: Fix behaviour of fail by @eolivelli in https://github.com/LangStream/langstream/pull/214
• Rename more packages to ai.langstream by @eolivelli in https://github.com/LangStream/langstream/pull/217
• Rename API and Core to ai.langstream by @eolivelli in https://github.com/LangStream/langstream/pull/218
• Add support for DLQ in Python by @cbornet in https://github.com/LangStream/langstream/pull/211
• Fix: make Source commit() implementation optional again by @cbornet in https://github.com/LangStream/langstream/pull/219
• Rename more packages to ai.langstream by @eolivelli in https://github.com/LangStream/langstream/pull/220
• Rename Operator packages to ai.langstream by @eolivelli in https://github.com/LangStream/langstream/pull/221
• Final rename by @eolivelli in https://github.com/LangStream/langstream/pull/222
• Rename directories, modules and maven artifacts by @eolivelli in https://github.com/LangStream/langstream/pull/223
• WebCrawler source - prototype by @eolivelli in https://github.com/LangStream/langstream/pull/224
• Improve WebCrawler and README by @eolivelli in https://github.com/LangStream/langstream/pull/226
• WebCrawler: first stable version by @eolivelli in https://github.com/LangStream/langstream/pull/228
• Add files to build a python wheel with the API by @cbornet in https://github.com/LangStream/langstream/pull/225
• Cli: default config must point to conf/cli.yaml by @nicoloboschi in https://github.com/LangStream/langstream/pull/231
• Add cli download instructions by @nicoloboschi in https://github.com/LangStream/langstream/pull/232
• Refactor readme to explain how to run server and deploy an app by @nicoloboschi in https://github.com/LangStream/langstream/pull/233
• Add doc to the Python API by @cbornet in https://github.com/LangStream/langstream/pull/230
• Remove local helm chart references by @nicoloboschi in https://github.com/LangStream/langstream/pull/234

New Contributors

• @dependabot made their first contribution in https://github.com/LangStream/langstream/pull/197

Full Changelog: https://github.com/LangStream/langstream/compare/v0.0.1...v0.0.3

Docker images:
public.ecr.aws/y3i6u2n7/datastax-public/langstream-runtime 0.0.2-alpha c5806ee8b4cc
public.ecr.aws/y3i6u2n7/datastax-public/langstream-control-plane 0.0.2-alpha 0cdffe00a613
public.ecr.aws/y3i6u2n7/datastax-public/langstream-api-gateway 0.0.2-alpha dcc9589e82f5
public.ecr.aws/y3i6u2n7/datastax-public/langstream-cli 0.0.2-alpha 3f4807c7d2eb
public.ecr.aws/y3i6u2n7/datastax-public/langstream-deployer 0.0.2-alpha ab34ec9a5a85