Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
MIT License
Fast and memory-efficient exact attention
Implementation of a Transformer, but completely in Triton
Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and...
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 min...
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
Some personal experiments around routing tokens to different autoregressive attention, akin to mi...
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Re...
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...
An implementation of Performer, a linear attention-based transformer, in Pytorch