n-grammer-pytorch

electra-pytorch

A simple and working implementation of Electra, the fastest way to pretrain language models from ...

PPLM

Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.

two_hot_encoding

RQ-Transformer

Implementation of RQ Transformer, proposed in the paper "Autoregressive Image Generation using Re...

language-modeling

Pipeline for training Language Models using PyTorch.

ali-pytorch

Adversarially Learned Inference in Pytorch

hourglass-transformer-pytorch

Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI

nanotron

Minimalistic large language model 3D-parallelism training

minGPT

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

transformers-from-scratch

Modular Python implementation of encoder-only, decoder-only and encoder-decoder transformer archi...

fast-transformer-pytorch

Implementation of Fast Transformer in Pytorch

h-transformer-1d

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

mlm-pytorch

An implementation of masked language modeling for Pytorch, made as concise and simple as possible

feedback-transformer-pytorch

Implementation of Feedback Transformer in Pytorch

transformer-smaller-training-vocab

Temporary remove unused tokens during training to save ram and speed.