YaLM-100B

Code-LMs

Guide to using pre-trained large language models of source code

ChatLM-mini-Chinese

中文对话0.2B小模型（ChatLM-Chinese-0.2B），开源所有数据集来源、数据清洗、tokenizer训练、模型预训练、SFT指令微调、RLHF优化等流程的全部代码。支持下游任务sf...

languagemodels

Explore large language models in 512MB of RAM

Megatron-LM

Ongoing research training transformer models at scale

minGPT

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

minGPT-flax

GPT implementation in Flax

language-modeling

Pipeline for training Language Models using PyTorch.

AnyText

Official implementation code of the paper <AnyText: Multilingual Visual Text Generation And Editing>

two_hot_encoding

sentiment-discovery

Unsupervised Language Modeling at scale for robust sentiment classification

yarn

YaRN: Efficient Context Window Extension of Large Language Models

GLM

GLM (General Language Model)