Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI
MIT License
Implementation of Feedback Transformer in Pytorch
An implementation of Performer, a linear attention-based transformer, in Pytorch
Implementation of Long-Short Transformer, combining local and global inductive biases for attenti...
An implementation of masked language modeling for Pytorch, made as concise and simple as possible
Implementation of the Point Transformer layer, in Pytorch
An implementation of local windowed attention for language modeling
Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classif...
Implementation of RQ Transformer, proposed in the paper "Autoregressive Image Generation using Re...
Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning
Implementation of TabTransformer, attention network for tabular data, in Pytorch
Implementation of Fast Transformer in Pytorch
Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"
Implementation of Bottleneck Transformer in Pytorch
Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with ...