Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind
MIT License
A variant of Transformer-XL where the memory is updated not with a queue, but with attention
Implementation of MagViT2 Tokenizer in Pytorch
An implementation of Performer, a linear attention-based transformer, in Pytorch
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
Modular Python implementation of encoder-only, decoder-only and encoder-decoder transformer archi...
Implementation of MusicLM, Google's new SOTA model for music generation using attention networks,...
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Py...
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with ...
Implementation of Agent Attention in Pytorch
Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Fun...
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 min...
An implementation of local windowed attention for language modeling
Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch