Bringing TED experiences to every topic with Gen AI
Bringing TED experiences to every topic with Gen AI.
extrapolaTED is a pioneering application designed to craft TED-like lectures on any conceivable topic on demand. Through a fusion of AI and human genius, we're pushing the boundaries of education and information dissemination. Our mission is to democratize knowledge, making every recent scientific achievement known and inspiring to all.
Embark on a technological odyssey encompassing retrieval, embedding, story, and image generation:
Dive deep into the heart of extrapolaTED with a wide array of datasets that serve as the bedrock of our content generation:
unum-cloud/ann-arxiv-2m
): A treasure trove of 2 million vectorized abstracts summarizing the latest strides in scientific research.wikipedia
): The 6 million abstracts in this dataset are a solid foundation for textual content, providing ground-truth retrieval of factual information.Get started with extrapolaTED in a breeze:
Environment Setup:
conda env create -f conda.yml
conda activate extrapolaTED
conda create -n extrapolated python=3.10
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip3 install -r requirements.txt
Data Preparation:
./download_arxiv_texts.sh
./download_wiki_images.sh
python prepare_arxiv_texts.ipynb
python prepare_ted_texts.ipynb
python prepare_wiki_images.py
python prepare_wiki_texts.ipynb
Server Startup:
python server.py
Exploration:
Your journey toward creating insightful and illuminating TED-like lectures begins now!
Navigate through the well-organized directory structure to explore the different facets of extrapolaTED:
├── README.md
├── requirements.txt
├── conda.yml
├── mount_disks.sh
├── download_arxiv_texts.sh
├── download_wiki_images.py
├── download_wiki_images.sh
├── prepare_embeddings.py
├── prepare_arxiv_texts.ipynb
├── prepare_ted_texts.ipynb
├── prepare_wiki_images.py
├── prepare_wiki_texts.ipynb
├── server.py
├── data
│ ├── ann-arxiv-2m
│ │ ├── abstract.e5-base-v2.fbin
│ │ ├── abstract.e5-base-v2.usearch
│ │ ├── title_abstract.parquet
│ │ └── title_abstract.tsv
│ ├── ann-wiki-6m
│ │ ├── abstract.e5-base-v2.fbin
│ │ ├── abstract.e5-base-v2.usearch
│ │ ├── downloads
│ │ ├── title_abstract.parquet
│ │ └── wikipedia
│ └── ann-wiki-images-3m
│ ├── abstract.e5-base-v2.fbin
│ ├── abstract.e5-base-v2.usearch
│ ├── abstract.uform-vl-english.fbin
│ ├── images.uform-vl-english.fbin
│ └── title_abstract.parquet
├── article_generation
│ ├── add_images_to_transcript.ipynb
│ ├── betterfy_prompt.txt
│ ├── example_input_1_french_architecture.json
│ ├── example_transcript_1_french_architecture.json
│ ├── learn_to_generate_articles.ipynb
│ ├── learn_to_generate_images.ipynb
│ └── raw_articles
│ ├── architecture_of_paris.txt
│ ├── french_architecture.txt
│ ├── grand_palais.txt
│ ├── jean_nouvel.txt
│ └── paris_architecture_of_the_belle_epoque.txt
└── use_wordware_ouput
├── grav_wave_astronomy.json
├── make_silence.sh
├── make_video.py
└── superconductors.json
extrapolaTED: Where the quest for knowledge never ends.