Graph neural network message passing reframed as a Transformer with local attention
MIT License
Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch
Implementation of the Point Transformer layer, in Pytorch
Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repo...
An implementation of local windowed attention for language modeling
Implementation of Lie Transformer, Equivariant Self-Attention, in Pytorch
A Transformer made of Rotation-equivariant Attention using Vector Neurons
Implementation of an Attention layer where each head can attend to more than just one token, usin...
Implementation of Bottleneck Transformer in Pytorch
(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000...
Implementation of Block Recurrent Transformer - Pytorch
Pytorch reimplementation of Molecule Attention Transformer, which uses a transformer to tackle th...
Implementation of Agent Attention in Pytorch
Transformers with Arbitrarily Large Context
Exploring an idea where one forgets about efficiency and carries out attention across each edge o...
An implementation of (Induced) Set Attention Block, from the Set Transformers paper