Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
MIT License
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Re...
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
An implementation of local windowed attention for language modeling
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 min...
Implementation of MagViT2 Tokenizer in Pytorch
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
An implementation of Performer, a linear attention-based transformer, in Pytorch
Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and ...
Pytorch implementation of Compressive Transformers, from Deepmind
Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch
Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"