Welcome to ArXiv ChatGuru. This tool harnesses LangChain and Redis to make ArXiv's vast collection of scientific papers more interactive. Through this approach, we aim to make accessing and understanding research easier and more engaging, but also just to teach about how Retrieval Augmented Generation (RAG) systems work.
This diagram shows the process how ArXiv ChatGuru works. The user submits a topic, which is used to retrieve relevant papers from ArXiv. These papers are then chunked into smaller pieces, for which embeddings are generated. These embeddings are stored in Redis, which is used as a vector database. The user can then ask questions about the papers retrieved by the topic they submitted, and the system will return the most relevant answer.
redisvl
, Langchain
, Streamlit
, etcNote: This is not a production application. It's a learning tool more than anything. We're using Streamlit to make it easy to interact with, but it's not meant to be a scalable application. It's meant to be a learning tool for understanding how RAG systems work, and how they can be used to make scientific literature more interactive. We will continue to make this better over time.
🌟 If you love what we're doing, give us a star! Contributions and feedback are always welcome. 🌌ðŸ”
What we want to do next (ideas welcome!):
First, clone this repo and cd into it.
$ git clone https://github.com/RedisVentures/ArxivChatGuru.git && cd ArxivChatGuru
Create your env file:
$ cp .env.template .env
fill out values, most importantly, your OPENAI_API_KEY
.
Install dependencies: You should have Python 3.7+ installed and a virtual environment set up.
$ pip install -r requirements.txt
Run the app:
$ streamlit run App.py
Navigate to:
http://localhost:8501/
First, clone the repo like above.
Create your env file:
$ cp .env.template .env
fill out values, most importantly, your OPENAI_API_KEY
.
Run with docker compose:
$ docker compose up
add -d
option to daemonize the processes to the background if you wish.
Issues with dependencies? Try force-building with no-cache:
$ docker compose build --no-cache
Navigate to:
http://localhost:8080/