Release v0.2.84

We're excited to announce the release of version 0.2.84, which brings new features, bug fixes, and improvements to enhance your experience with R2R.

New Features

• Migration Support: Introduced Alembic for database migration support.
• Enhanced Logging: Added CompletionRecord to logs for richer observability.
• Completion Scoring: Users can now provide feedback on the responses they receive.
• User Tracking: Added user tracking to logs for improved analytics.

Bug Fixes

• Resolved issues related to CONFIG_PATH variables.
• Fixed incorrect providers in default configs.
• Various other minor bug fixes and improvements.

Breaking Changes

• Database Migration Required: Clone the repository and r2r migrate to update your local SQLite logging database. Note: this is a beta feature, and you may need to adjust the migration script to point to your local logs.
• User ID in Logs: User IDs are now included in logs. Logging without user ID is deprecated but will be supported until v0.3.0.

Configuration Updates

• Adjustments made to r2r/examples/configs/local_llm_neo4j_kg.toml.

Other Changes

• Multiple pull requests for patches and updates.
• Temporary changes that were reverted.

Documentation Updates

• Added instructions on how to score completions with the Python SDK.

Contributors

We'd like to thank all the contributors who made this release possible, with a special shoutout to @smig23 for their first contribution!

Release v0.2.83

We're excited to announce the release of version 0.2.83, which brings new features, bug fixes, and improvements to enhance your experience with R2R.

New Features

• Migration Support: Introduced Alembic for database migration support.
• Enhanced Logging: Added CompletionRecord to logs for richer observability.
• Completion Scoring: Users can now provide feedback on the responses they receive.
• User Tracking: Added user tracking to logs for improved analytics.

Bug Fixes

• Resolved issues related to CONFIG_PATH variables.
• Fixed incorrect providers in default configs.
• Various other minor bug fixes and improvements.

Breaking Changes

• Database Migration Required: Run r2r migrate to update your local SQLite logging database.
• User ID in Logs: User IDs are now included in logs. Logging without user ID is deprecated but will be supported until v0.3.0.

Configuration Updates

• Adjustments made to r2r/examples/configs/local_llm_neo4j_kg.toml.

Other Changes

• Multiple pull requests for patches and updates.
• Temporary changes that were reverted.

Documentation Updates

• Added instructions on how to score completions with the Python SDK.

Contributors

We'd like to thank all the contributors who made this release possible, with a special shoutout to @smig23 for their first contribution!

Summary of Changes

This release includes several significant updates and improvements:

• Bug fixes:

  • Resolved a PostgreSQL issue (#811)
  • Addressed PDF parsing problems with multilingual Unicode characters (#785)

• Feature enhancements:

  • Added support for external LiteLLM proxy (#815)
  • Implemented server stats (#823)
  • Introduced dynamic port selection (#833)

• Infrastructure improvements:

  • Refactored configuration to use TOML and JSON (#817)
  • Added Unstructured Docker support (#824, #826)
  • Implemented additional registry (#828)

• Developer experience:

  • Extended CLI provider checks (#816)
  • Improved test runner setup (#827, #829)
  • Split build flows for optimization (#835)

• User experience:

  • Enabled automatic navigation to dashboard on launch (#819)
  • Completed renaming to "agent" for consistency (#830)

• Performance enhancements:

  • Prevented Docker timeouts during report generation (#821)
  • Implemented caching improvements (#831)

• Flexibility improvements:

  • Made parsing and chunking providers configurable (#820, #832)

New Contributors

• @igor-pechersky made their first contribution (#815)

0.2.74 - 2024-07-30

Added

• Docker Dev Image for CI/CD
• No-postgres mode
• Completion yield functionality
• Entity types and relationships for Knowledge Graph search

Changed

• Refactored CLI
• Updated concurrency and various configs
• Renamed 'local_ollama' to 'local_llm'
• Separated tool stream and LLM outputs

Fixed

• Health check dependency in Dockerfile
• Backoff for stream chat
• Runtime warning about version reading
• host.docker.internal connectivity issues

Improved

• R2R image configuration
• Local LLM configurations

Removed

• Postgres dependency from Docker (optional exclusion)

Contributors

• @houjibofa2050 made their first contribution

Here's a compressed version of the changelog:

Key Changes

• Introduction of assistants an streaming assistants (#759, #763, #765, #774, #777)
• Improvements to PDF, XLSX, and CSV parsing (#754, #764)
• Package updates and fixes (PRs #741, #743, #744, #753)
• Refactoring and improvements to request handling (#756, #757)
• Renaming KG agent to KG search (#766)
• Integration tests added (#761)
• CLI and performance enhancements (#773, #776)
• Various bug fixes and improvements (#745, #746, #752, #755, #769, #768, #772, #775, #780, #787)

New Contributors

• @hhjin
• @gaoyifan

Full Changelog: v2.0.64...v0.2.73

What's Changed

• bundle ollama by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/723
• Dev by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/727
• cleanup print cruft by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/737

Full Changelog: https://github.com/SciPhi-AI/R2R/compare/v0.2.61...v2.0.64

v0.2.61 - Adding Auth 🚀 🚀 🚀

Summary:

Added authentication, crypto, and database providers, and updated routes, client, and tests accordingly

Key points:

• Hard-coded limits for entities and relations in r2r/base/providers/kg.py update_kg_prompt function.
• Added auth, crypto, and database configurations in r2r.json.
• Introduced new classes for user management in r2r/base/abstractions/user.py.
• Added AuthProvider, CryptoProvider, and DatabaseProvider classes in r2r/base/providers.
• Updated R2RClient in r2r/main/api/client.py to handle authentication.
• Added authentication routes in r2r/main/api/routes/auth.py.
• Updated ingestion, management, and retrieval routes to include authentication checks.
• Updated R2RBuilder and R2RProviderFactory to support new providers.
• Added tests for new authentication and database functionalities.
• Updated documentation and configuration files to reflect changes.

v0.2.60

###v 0.2.58

Summary:

This PR renames configuration options, updates prompts, enhances knowledge graph extraction, and improves Docker and CLI functionalities across the codebase.

Key points:

• Renamed CONFIG_OPTION to CONFIG_NAME in README.md, compose.yaml, and r2r/main/app_entry.py.
• Updated Docker volumes in compose.neo4j.yaml and compose.yaml.
• Changed default prompts in r2r/base/providers/kg_provider.py and r2r/prompts/local/defaults.jsonl.
• Enhanced knowledge graph extraction logic in r2r/pipes/ingestion/kg_extraction_pipe.py.
• Modified CLI commands in r2r/cli/cli.py to support new configuration options and improved error handling.
• Improved knowledge graph inspection and analytics in r2r/main/services/management_service.py.
• Refactored vector handling in r2r/vecs/collection.py to use a custom Vector type.

Summary:

Enhanced ingestion process, added new tests, and improved logging and configuration handling across the codebase.

Key points:

• Added POSTGRES_VECS_COLLECTION to .env.example and ci.yml.
• Updated Python version to 3.12.4 in ci.yml.
• Added Ollama installation and server setup steps in ci.yml.
• Modified Dockerfile to use multi-stage builds and updated CMD to use uvicorn.
• Added CLIENT_MODE to compose.yaml.
• Updated pyproject.toml to ignore specific warnings and set asyncio_mode to auto.
• Added new exceptions R2RDocumentProcessingError and R2RException in r2r/base/abstractions/exception.py.
• Updated r2r/base/abstractions/document.py to include DocumentStatus and set initial status to processing.
• Updated document ingestion process to mark progressive status throughout ingestion process.
• Refactored ingestion and update document methods in r2r/main/services/ingestion_service.py.
• Added new tests for ingestion service in tests/test_ingestion_service.py.
• Added async fixtures and cleanup tasks in multiple test files to handle asyncio event loops and task cleanup.
• Updated embedding providers to handle retries and backoff in r2r/providers/embeddings/ollama/ollama_base.py.
• Added new end-to-end tests in tests/test_end_to_end.py.
• Refactored logging tests to include async fixtures and handle multiple logging providers in tests/test_logging.py.

Add r2r CLI

e.g. r2r ingest or r2r search --query='who is aristotle' --do-hybrid-search

Extending docker support

• First steps towards adding Postgres+pgvector directly to the Docker.
• Adding nginx to docker buildout
• Adding r2r dashboard to docker buildout

Developer experience

• clarified errors around dimension mismatches and missing pgvector installs
• improved configuration options, e.g. support for hardcoding postgres connection directly into the config
• made it trivial to do basic functionality via curl commands
• return search results with RAG output
• improved deletion filter options

v0.2.39

v2.0.38

Release Notes:

1. Document Management:

   • Enhanced document ingestion and search functionalities
   • Added UUID generation based on data hash
   • Improved handling of document updates and file ingestion

2. API and Endpoints:

   • Added health check endpoint
   • Updated search result format and handling
   • Improved KG agent prompt updating

3. Configuration and Dependencies:

   • Set Ollama as default local embedding provider
   • Updated dependencies and added CodeCov integration
   • Refined Docker documentation

4. Code Refactoring:

   • Streamlined pipeline initialization
   • Renamed functions in quickstart example for consistency
   • Removed unused clients (ExaClient, IonicClient)

5. Testing and CI:

   • Added end-to-end tests for document operations
   • Integrated CodeCov with CI workflow

6. Documentation:

   • Reorganized and expanded README.md
   • Created Jupyter notebook for examples
   • Updated quickstart demo

7. Bugfixes:

   • Addressed SQLite AND/OR logic bug
   • Fixed Ollama serve command error
   • Patched quickstart bug

8. Misc:

   • Updated logging format
   • Added debug print statement in R2RApp initialization

Version v2.0.3

Integrates v0 of the knowledge graph agent and finalizes v1 API, check out the latest docs here

• https://r2r-docs.sciphi.ai/introduction.

• Refactor application logic to have ingestion / retrieval / management services which are connected to the FastAPI.

• Finalize v1 API spec

• Add knowledge graph agent to the R2R pipeline

• Refactor R2R into singleton

Summary:

Refactored R2R application into modular services, integrated knowledge graph agent, updated API to v1, and enhanced embedding providers.

Key points:

• Refactored application logic into ingestion, retrieval, and management services (r2r/main/services/*).
• Integrated knowledge graph agent into R2R pipeline (r2r/pipes/kg_agent_search_pipe.py).
• Refactored R2R into a singleton (r2r/main/app.py).
• Updated API to v1 with new routes (r2r/main/api/routes/*).
• Added VectorSearchSettings and KGSearchSettings for search configurations (r2r/core/abstractions/search.py).
• Enhanced embedding providers (r2r/providers/embeddings/*).
• Updated tests to reflect new architecture (tests/*).
• Updated documentation to reflect new API and architecture (docs/*).

What's Changed

• Fix broken relative links by @NolanTrem in https://github.com/SciPhi-AI/R2R/pull/448
• Feature/add missing abstractions by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/449
• Move from whitelisting parsers to blacklisting by @NolanTrem in https://github.com/SciPhi-AI/R2R/pull/445
• Add better handling for ingesting duplicates by @NolanTrem in https://github.com/SciPhi-AI/R2R/pull/450
• Add basic telemetry by @NolanTrem in https://github.com/SciPhi-AI/R2R/pull/451
• Feature/improve kg docs rebased by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/453
• refine the kg example by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/454
• fix node linkage by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/455
• Feature/improve kg docs 2 by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/456
• Add telemetry to docs by @NolanTrem in https://github.com/SciPhi-AI/R2R/pull/457
• tweak by @emrgnt-cmplxty in https://github.com/SciPhi-AI/R2R/pull/458
• Reapply changes from merge conflict by @NolanTrem in https://github.com/SciPhi-AI/R2R/pull/459

Full Changelog: https://github.com/SciPhi-AI/R2R/compare/v0.2.17...v2.0.18

• Transport latest neo4j integration in LlamaIndex for node metadata

• Add automatic knowledge graph generation and knowledge graph cookbook.
• Finalize analytics
• Add dashboard cookbook
• Update docs

• Add automatic knowledge graph generation and knowledge graph cookbook.
• Finalize analytics
• Add dashboard cookbook
• Update docs

Fixing OPENAI_API_KEY error stemming from default image parser activation.