My Digital Palace - A Personal Journal for Reflection

Introduction

I created my Digital Palace as a public platform for self-expression, reflection, and sharing my memories. It is where I openly document my thoughts, experiences, milestones, creative works - my personal journey.

My Digital Palace allows me to write journal entries, upload multimedia, record audio, and categorize content by date and themes. The tools help me capture toughts to share publicly.

With my palace being public, anyone can read my entries, view my content, and leave comments to connect with me and others in my community. I enjoy fostering this open dialogue and discussion around my shared stories.

In summary, my Digital Palace provides an engaging platform for public self-expression, reflection, and sharing memories.

Raphaël MANSUY

Today I Learned (TIL)

The goal of a "Today I Learned" (TIL) journal is to document and reflect upon something new or interesting that you have learned on a given day.

It serves as a personal knowledge base where I can record and reflect on my daily learning experiences.

TIL journals are often used by programmers, developers, and other professionals to share tips, tricks, and lessons learned in their respective fields.

They can also be a valuable resource for others who are looking to learn and expand their knowledge in a specific domain.

👉 Todays I Learned

Tutorial Series: "For the Impatient"

The "For the Impatient" tutorial series is designed to provide a fast-paced and efficient way for learners to master programming languages and other technical subjects. The core idea behind this series is to cater to the needs of individuals who are eager to gain proficiency quickly, without getting bogged down in lengthy, comprehensive tutorials.

The typical learner in the "Impatient" target audience might be someone who:

• Has a strong motivation to learn a new skill quickly, perhaps for a time-sensitive project or job requirement
• Prefers a more streamlined, concise learning approach that focuses on the most essential concepts and practical applications
• Wants to get up and running with a new language or technology as rapidly as possible, without spending too much time on theoretical background or peripheral topics

👉 The impatient series

Mental models

A mental model is a compression of how something works

Mental models

Generative AI

• Generative AI Business Use Cases
• Comprehensive Guide to Large Language Model Engineering

Courses and lectures

• AI Courses
• Prompt engineering
• System, Database, Operating System
• 🤗 GenAI Cookbook
• Generative AI for Beginners Microsoft

Blog

👉 Top Quality Blogs

Cloud Providers

👉 Top Quality Cloud Providers

High Quality Datasets for Fine-Tuning

👉 Top Quality Datasets

AI Engineering

Articles

• Demystifying Classifiers and Embeddings

• Design and Architecture Patterns for LLM application

• Frameworks for building LLM applications

• Bridging the Gap Between Thinking and Doing: FaR an Effective Prompting Framework inspired from Theory of Mind

• Beyond Prompt Engineering: Modular and Optimized LM Programs with DSPy

• Mastering the Art of Training Large Language Models from Scratch

LLMs implementation

• LLMs from scratch Implementing a ChatGPT-like LLM from scratch, step by step by from Sebastian Raschka

Prompt Engineering

• A list of prompt engineering techniques
• Mastering the art of prompt engineering in French
• Mastering the art of prompt engineering

Embeddings

Articles

• What are embeddings and how do they work? A book from Vicki Boykis

  • Github
  • PDF

• Fine-tuning language models improves performance by enhancing existing mechanisms rather than creating new ones, as evidenced by consistent circuit functionality in entity tracking tasks

• Introduction to Matryoshka Embedding Models

• Binary Embeddings Cohere

LLM Applications

What are the common use cases of LLM Applications ?

• ottogrid

Front End

Bionic GPT BionicGPT is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality BionicGPT can run on your laptop or scale into the data center. Lobe Chat 🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system. One-click FREE deployment of your private ChatGPT/ Claude application.

Toolbox

• QLLM QLLM: A powerful CLI for seamless interaction with multiple Large Language Models. Simplify AI workflows, streamline development, and unlock the potential of cutting-edge language models. by Quantalogic

• ell A language model programming framework.

• LangChain Framework to construct LLMs application documentation 🦜

• LLamaIndexTurn your enterprise data into production-ready LLM applications 🦙

• HuggingFace Model, Datasets, Inference Space, the GitHub of AI models. 🤗

  • Hub Your starting point to HuggingFace
  • cli Command Line Interface (CLI)

• instructor Instructor makes it easy to reliably get structured data like JSON from Large Language Models (LLMs) like GPT-3.5, GPT-4, GPT-4-Vision, including open source models like Mistral/Mixtral from Together, Anyscale, Ollama, and llama-cpp-python.

• instructor_ex (Elixir version of Instructor)

• marvin Marvin is a lightweight AI toolkit for building natural language interfaces that are reliable, scalable, and easy to trust.

• Vercel AI SDK (Stream React Components from LLMs to deliver richer user experiences)

• easyllm EasyLLM is an open source project that provides helpful tools and methods for working with large language models (LLMs), both open source and closed source.

• Flowneum A toolkit for controllable, private AI on consumer hardware in rust

• Kalosm Kalosm is an open source framework for private language, audio, and image models in Rust

• PhiData Phidata is a toolkit for building AI Assistants using function calling.

• Microsoft AutoDev 🧙‍AutoDev: The AI-powered coding wizard with multilingual support

• Cognee Memory management for the AI Applications and AI Agents

• MelloTTS High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean.

• MemGPT Enable Next-Gen Large Language Model Applications

• lumentis Generate beautiful docs from your transcripts and unstructured information with a single command.

• Wantdb/OpenUI OpenUI let's you describe UI using your imagination, then see it rendered live.

• FireCrawl Turn entire websites into LLM-ready markdown

• Sammo A library for prompt engineering and optimization (SAMMO = Structure-aware Multi-Objective Metaprompt Optimization)

AI Assisted coding

• Plandex An AI coding engine for complex tasks
• [Aider][https://github.com/paul-gauthier/aider] aider is AI pair programming in your terminal

RAG

• RagFlow RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
• RAG Techniques This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. RAG systems combine information retrieval with generative models to provide accurate and contextually rich responses.
• byaldi Use late-interaction multi-modal models such as ColPali in just a few lines of code.

Knowledge Graph

-MindGraph proof of concept prototype for generating and querying against an ever-expanding knowledge graph with ai

Controlling Inference

• Transformer Head Toolkit for attaching, training, saving and loading of new heads for transformer models
• AICI AICI: Prompts as (Wasm) Programs. Controlling inference using Wasm programs.

Quantization

• Aimet AIMET is a library that provides advanced quantization and compression techniques for trained neural network models from Qualcomm Innovation Center.
• hqq Official implementation of Half-Quadratic Quantization (HQQ). HQQ is a fast and accurate model quantizer that skips the need for calibration data. It's super simple to implement (just a few lines of code for the optimizer). It can crunch through quantizing the Llama2-70B model in only 4 minutes! 🚀

Training Models

• OLMo Modeling, training, eval, and inference code for OLMo

AI Command tools

• llms A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine. By Simon Willison
• Open Interpreter A natural language interface for computers
• Documentation Open Interpreter A new way to use computers
• whisper Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.
• plock From anywhere you can type, query and stream the output of an LLM or any other script
• Screen shot to code Drop in a screenshot and convert it to clean code (HTML/Tailwind/React/Vue)
• Code2prompt Convert a codebase to an AI prompt
• Claude Engineer Claude Engineer is an interactive command-line interface (CLI) that leverages the power of Anthropic's Claude-3.5-Sonnet model to assist with software development tasks. This tool combines the capabilities of a large language model with practical file system operations and web search functionality.

Controlling the generation of LLMs during decoding

• Guidance A guidance language for controlling large language models.
• AICI AICI: Prompts as (Wasm) Programs
• Representation Ingeneering Representation Engineering Mistral-7B an Acid Trip 💊

Voice

• VoiceCraft Zero-Shot Speech Editing and Text-to-Speech in the Wild

Mac App

• Enchanted Enchanted is iOS and macOS app for chatting with private self hosted language models such as Llama2, Mistral or Vicuna using Ollama.
• BoltAI BoltAI is a beautiful and powerful AI chat app for Mac. It integrates ChatGPT and other large language models (LLMs) deeply into your workflows so you don’t have to go back and forth with the ChatGPT web UI.

Serving LLMS

Tools

• llamaC++ LLM inference in C/C++
• vLLM Easy, fast, and cheap LLM serving for everyone, documentation
• nm-vllm A high-throughput and memory-efficient inference and serving engine for LLMs (sparse compressing)
• LLama Cpp Python Binding OpenAI compatible web server
• SkyPilot Run LLMs and AI on Any Cloud documentation
• Ollama Go program that encapsulate llamac++. documentation
• fastassert Dockerized LLM inference server with constrained output (JSON mode), built on top of vLLM and outlines.
• luminal Deep learning at the speed of light coded in Rust. The aim for 0.3 is to achieve SOTA performance on an M1 pro (50 tok/s), and near SOTA on single nvidia gpus (>100 tok/s)
• LLamafile Turning a LLM model into a Multiplatform executable
• candle Minimalist ML framework for Rust. Run and Serve Models in Rust.
• Mlx Server This Python library is the easist way to begin building on top of Apple's machine learning library MLX
• AICI AICI: Prompts as (Wasm) Programs
• List of tools that serves AI locally An awesome repository of local AI tools
• SkypilotSkyPilot is a framework for running LLMs, AI, and batch jobs on any cloud, offering maximum cost savings, highest GPU availability, and managed execution.
• Jan Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
• LoraX Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs (Documentation)
• Candle Minimalist ML framework for Rust from HuggingFace to run models in production.
• ZML High performance AI inference stack. Built for production. @ziglang / @openxla / MLIR / @bazelbuild

Agents

• MegaGPTThe Multi-Agent Framework: Given one line Requirement, return PRD, Design, Tasks, Repo
• CrewAI Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
• Devika Devika is an Agentic AI Software Engineer that can understand high-level human instructions, break them down into steps, research relevant information, and write code to achieve the given objective. Devika aims to be a competitive open-source alternative to Devin by Cognition AI.
• AgentStudio AgentStudio is an open toolkit covering the entire lifespan of building virtual agents that can interact with everything on digital worlds
• AIOS AIOS: LLM Agent Operating System
• AutoGen Enable Next-Gen Large Language Model Applications
• Screen Agents ScreenAgent: A Computer Control Agent Driven by Visual Language Large Model
• Synapse Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control
• Cradles Towards General Computer Control: A Multimodal Agent for Red Dead Redemption II as a Case Study
• kwaiagents A generalized information-seeking agent system with Large Language Models (LLMs).
• LLocalSearch LLocalSearch is a completely locally running search aggregator using LLM Agents. The user can ask a question and the system will use a chain of LLMs to find the answer. The user can see the progress of the agents and the final answer. No OpenAI or Google API keys are needed.
• SWE Agents SWE-agent: Agent Computer Interfaces Enable Software Engineering Language Models
• Presentation of Agentic Models from Angrew NG
• Zep Long term memory for Agent Zep: Long-Term Memory for ‍AI Assistants.
• OpenSSA OpenSSA: Small Specialist Agents—Enabling Efficient, Domain-Specific Planning + Reasoning for AI

Prompt Engineering

• Summoning the Magic of Prompts: A Deep Dive into Prompt Engineering Patterns
• Claude Anthropic prompts Library Explore optimized prompts for a breadth of business and personal tasks
• Navigating the Prompt Engineering Landscape: A Comprehensive Survey for NLP Practitioners

Interesting projects

• MLX Swift Examples Examples using MLX Swift

Models

• Gorilla OpenFunction 🦍 Gorilla: Large Language Model Connected with Massive APIs
• Models list Open Sources model list
• NousResearch/Nous-Hermes-llama-2-7b
• [Ollama version of Nous-Hermes2Pro][https://ollama.com/adrienbrault/nous-hermes2pro]
• LLavaNext LLaVA-NeXT is recently merged to transformers and it outperforms many of the proprietary models like Gemini on various benchmarks!
• Startling LM 7B Beta The best 7B Models - Finetuned from model: Openchat-3.5-0106 (based on Mistral-7B-v0.1) - From Berkeley https://starling.cs.berkeley.edu/

Vector Database

• pgvectorscale A complement to pgvector for high performance, cost efficient vector search on large workloads.

LLM Leaderboard

• Berkeley Function-Calling Leaderboard LeaderBoard for function calling
• OpenLLMs Leaderboard 📐 The 🤗 Open LLM Leaderboard aims to track, rank and evaluate open LLMs and chatbots.
• HumanEval Leaderboard # 🏆 EvalPlus Leaderboard 🏆 EvalPlus evaluates AI Coders with rigorous tests

Control generation

• instructor structured outputs for llms
• fructose LLM calls as strongly-typed functions

Synthetic data generation

Tools

• Bonito A lightweight library for generating synthetic instruction tuning datasets for your data without GPT.

Fine tuning

Tools

• unsloth 5X faster 60% less memory QLoRA finetuning
• Documentation from Premai about Finetuning
• LLama-Factory Unify Efficient Fine-tuning of 100+ LLMs
• peft 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
• Tochtune A Native-PyTorch Library for LLM Fine-tuning
• LLMTuner Tune LLM in few lines of code
• LMFlow An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All. (Include Lisa Finetuning)
• # Efficient finetuning of Llama 3 with FSDP QDoRA A blog article that explains how to use the stat of the art QDoRA fine tuning method on LLAMA3

Computing

Lighting thunder Source to source compiler for PyTorch. It makes PyTorch programs faster on single accelerators and distributed.

AI and Ethics

How to document an AI/Machine Learning Model: Improving Transparency in AI through Model Cards

AI Devices

• Friend AI wearable with 24h+ battery
• Whomane An open source wearable with camera

Development

VSCode

• Useful VSCode extensions
• Open source alternatives to popular products Discover Open Source Alternatives to Popular Software

UX Libraries

• React Aria
• TaillwindCSS
• shadcn

Data Engineering

• Fluvio Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.
• Unstract No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents

Useful tools

• Shot Scraper A command-line utility for taking automated screenshots of websites from Simon Willison
• ttok Count and truncate text based on tokens from Simon Willison
• github2files Create a big files from a GitHub files

Articles

Arrticles list

LinkedIn Posts

LinkedIn Posts

Medium publications

Forget the Cloud ☁️ — A MacBook Pro with M1/M2 is the Only AI Brain You Need … 🧠

---

mindmap
	root((AI))
		Applications
			LLM Applications
			Serving LLMs
		Engineering
			AI Engineering
				Framework
					LangChain
					LLamaIndex
				Vector Databases
			Prompt Engineering
			Fine Tuning
			RAG
		Models
			AI Models
			Embeddings
			
		Ethics
			AI and Ethics