RWKV-v2-RNN-Pile

pyllama

LLaMA: Open and Efficient Foundation Language Models

TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

ToolkenGPT

ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings - NeurIPS 20...

starcoder

Home of StarCoder: fine-tuning & inference!

LongMamba

Some preliminary explorations of Mamba's context scaling.

car-racing-attention-agent

Reproduce the results of "Neuroevolution of Self-Interpretable Agents" paper

language-modeling

Pipeline for training Language Models using PyTorch.

MAE-pytorch

Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners

two_hot_encoding

bilm-tf

Tensorflow implementation of contextualized word representations from bi-directional language models

starcoder2

Home of StarCoder2!

minimal-gpt-neox-20b

xlm-v-experiments

Experiments for XLM-V Transformers Integeration

BertSum

Code for paper Fine-tune BERT for Extractive Summarization

minGPT

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training