Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
MIT License
Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and...
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch
Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 min...
Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-E...
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...
Implementation of MagViT2 Tokenizer in Pytorch
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Re...
Fast and memory-efficient exact attention
An implementation of Performer, a linear attention-based transformer, in Pytorch
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
An implementation of local windowed attention for language modeling