asktube

Demo & Screenshot

Watch "AskTube First Demo" on YouTube

https://github.com/user-attachments/assets/610ec00b-e25a-4ac5-900c-145c8485675f

Features

• Work even with unsubtitle video
• No limit video time
• Support multiple AI vendors
• Focus on RAG implemetation
• Fully run on your local machine

Why does this project exist?

• Ive seen several GitHub repositories offering AI-powered summaries for YouTube videos, but none include Q&A
  functionality.
• I want to implement a more comprehensive solution while also gaining experience with AI to build my own RAG application.

Technology

• Language: Python, JS
• Server: [email protected], Bun@v1
• Framework/Lib: Sanic, Peewee, Pytubefix, Sentence Transformers, Sqlite, Chroma, NuxtJs/DaisyUI, etc.
• Embedding Provider (Analysis Provider):
  • OpenAI
  • Gemini
  • VoyageAI
  • Mistral
  • Sentence Transformers (Local)
• AI Provider:
  • OpenAI
  • Claude
  • Gemini
  • Mistral
  • Ollama (Local)
• Speech To Text:
  • Faster-Whisper (Local)
  • AssemblyAI
  • OpenAI
  • Gemini

Next Todo Tasks

• Implement Speech To Text for cloud models
  • AssemblyAI
  • OpenAI
  • Gemini
• Enhance
  • Skip using RAG for short videos
  • Chat prompts, chat messages by context limit
  • RAG: Implement Query Translation
    • Multiquery
    • Fusion
    • Decomposition
    • Step back
    • HyDE

How to run ?

For the first time running, the program maybe a bit slow due they need to install local models.

Run on your machine

• Ensure you installed:

  • Python 3.10

    • Windows User, please download here
    • Linux, MacOS User, please use homebrew or your install package command (apt, dnf, etc)
    • Or use conda

  • Poetry

    • Windows User open Powershell and run:

    (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -
    

    • Linux, MacOS User open Terminal and run:

    curl -sSL https://install.python-poetry.org | python3 -
    

  • Bun

  • ffmpeg

    • MacOS User

    brew install ffmpeg
    

    • Linux User

    # Ubuntu
    sudo apt install ffmpeg
    # Fedora
    sudo dnf install -y ffmpeg
    

    • Windows, please follow this tutorial Install ffmpeg for Windows

• Clone repostiory

  git clone https://github.com/jonaskahn/asktube.git
  

• Create file .env in asktube/engine directory:

  • Locally
  • Free with some limitations

• Run program

  • You may need to run first:

  poetry env use python
  

  • Open terminal/cmd/powershell in asktube/engine directory, then run:

  poetry install && poetry run python engine/server.py
  

  • Open terminal/cmd/powershell in asktube/web directory, then run:

  bun install && bun run dev
  

• Open web: http://localhost:3000

With docker (In process)

Before You Start

1. I built these services to docker images, but if you want to build local images, please run build.local.bat for Windows or build.local.amd64.sh or build.local.aarch64.sh for MacOS, Linux
2. If you have a GPU (cuda or rocm), please refer ENV settings above, change params like above

Locally

• Use local.yaml compose file to start
• Open terminal/cmd/powershell in asktube directory

docker compose -f compose/local.yaml pull && docker compose -f compose/local.yaml up -d

• After run, you need install Ollama model qwen2 and llama3.1 for QA

docker run ollama ollama run qwen2
docker run ollama ollama run llama3.1

Free (with rate limit)

• You need to go Google Gemini and VoyageAI to register account and generate your own API keys:
  • Gemini is free with your Google Account
  • VoyageAI (recommended by Claude) gives you free 50M tokens (a huge amount) but you need to add your credit card first.
• Replace your ENV setting in docker file free and start docker
• Open terminal/cmd/powershell in asktube directory

docker compose -f compose/free.yaml pull && docker compose -f compose/free.yaml up -d

Ideal

• Using VoyageAI for embedding texts
• Using OpenAI and Claude for QA, register account and generate your own API keys
• Replace your ENV setting in docker file ideal and start docker
• Open terminal/cmd/powershell in asktube directory

docker compose -f compose/ideal.yaml pull && docker compose -f compose/ideal.yaml up -d

Result

• Open web: http://localhost:8080

Architecture

The real implementation might differ from this art due to its complexity.

1 Extract data from given URL

2 Storing embedding chapter subtitles

3 Asking (included enrich question)

Notice

1. Do not use this for production. This aimed for end-users on their local machines.
2. Do not request any advanced features for management.

For development

• Backend
• Frontend

FAQ and Troubleshooting

• FQA