EasyContext

LongMamba

Some preliminary explorations of Mamba's context scaling.

Video-LLaVA

【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

MoE-LLaVA

Mixture-of-Experts for Large Vision-Language Models

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vi...

long_llama

LongLLaMA is a large language model capable of handling long contexts. It is based on OpenLLaMA a...

LLMZoo

⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language mod...

LongLoRA

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

yarn

YaRN: Efficient Context Window Extension of Large Language Models

PixArt-alpha

PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

LWM

flash-attention

Fast and memory-efficient exact attention

Vary

Official code implementation of Vary: Scaling Up the Vision Vocabulary of Large Vision Language M...

attention_sinks

Extend existing LLMs way beyond the original training length with constant memory usage, without ...