Large Language Models for Developers - from zero to your first LLM application

• 🍿 Talk - 1st edition - Open Source Day 2024
• 🍿 Talk - 2nd edition - PyCon Italy 2024
• 🧑‍🏫 Slide deck

Material for the homonymous talk.

This session provides a developer-centric introduction to LLMs, focused on practical applications. No pre-existing knowledge of LLMs and NLP is required.

You will gain insights into: using closed and open-source models, how to effectively prompt LLMs, vector databases, implementing Retrieval Augmented Generation applications (answer generation based on your data), building more complex applications.

Through a hands-on approach, I will show code examples using open-source tools: Haystack LLM framework, Hugging Face Transformers, Ollama, and more. I will also show how you can switch from proprietary to open models.

📚 Resources and code ‍💻

• Haystack LLM framework
• Start from a proprietary model
• Switch to local open LLMs with Ollama
  • Chat with Mistral
• Prompt Engineering
• RAG
  • Naive RAG
  • Web RAG
• Retrieval
  • Document Stores and Retrievers
  • Keyword-based Retrieval (BM25)
  • Embedding/vector Retrieval
  • Hybrid Retrieval
• From Demo to Production
  • Choose a good Language Model
  • Choose a LLM inference solution
  • Evaluation
  • Deployment
• Beyond RAG...
• There is much more!

Haystack LLM framework

• Website
• GitHub
• Main concepts

Start from a proprietary model

• Code snippet
• OpenAI Generator docs

Switch to local open LLMs with Ollama

• Ollama-Haystack integration
• Code snippets: run Ollama; use Ollama in Haystack

Chat with Mistral

• ChatMessage docs
• Code snippet

Prompt Engineering

• OpenAI guide
• promptingguide.ai

RAG

• Introduction to RAG with code examples

Naive RAG

• Code snippet

Web RAG

• Docs

Retrieval

Document Stores and Retrievers

• Document Store abstraction
• Choosing a Document Store
• Retrievers

Keyword-based Retrieval (BM25)

• Introduction to keyword-based retrieval: Bag of Words and TF-IDF; BM25
• BM25 Indexing Pipeline - code snippet
• BM25 RAG Pipeline - code snippet

Embedding/vector Retrieval

• Introduction to vector retrieval: From sparse representations to Language Models; Dense Passage Retrieval; Sentence Transformers for Dense Retrieval
• Embedding Indexing Pipeline - code snippet
• Embedding RAG Pipeline - code snippet

Hybrid Retrieval

• Retrieve and Re-Rank
• Hybrid Retrieval Tutorial

From Demo to Production

Choose a good Language Model

• LLM Evals and Benchmarking - Great blog post by Omar San Seviero
• Open LLM Leaderboard
• LMSYS Chatbot Arena Leaderboard
• Open ITA LLM Leaderboard

Choose a LLM inference solution

• Docs

Evaluation

• Docs

Deployment

• Serialization of Pipelines
• Hayhooks
• Deployment guides

Beyond RAG...

• Haystack cookbook
• Multilingual RAG from a podcast
• Hacker News summarizer
• Information extraction via LLMs
• AutoQuizzer

There is much more!

• File converters

• Observability: logging and tracing

• Function calling: docs and tutorial

• Conclusion: "The Shift from Models to Compound AI Systems" - blog post