A Transformer made of Rotation-equivariant Attention using Vector Neurons
MIT License
Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-E...
Explorations into the recently proposed Taylor Series Linear Attention
Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Fun...
A variant of Transformer-XL where the memory is updated not with a queue, but with attention
An implementation of local windowed attention for language modeling
Implementation of Agent Attention in Pytorch
Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and...
Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repo...
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Py...
Implementation of Lie Transformer, Equivariant Self-Attention, in Pytorch
Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...
An implementation of Performer, a linear attention-based transformer, in Pytorch
Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch
Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.