Implementation of Agent Attention in Pytorch
MIT License
An implementation of Performer, a linear attention-based transformer, in Pytorch
Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch
An implementation of local windowed attention for language modeling
Implementation of the transformer proposed in "Building Blocks for a Complex-Valued Transformer A...
Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.
Some personal experiments around routing tokens to different autoregressive attention, akin to mi...
Implementation of MagViT2 Tokenizer in Pytorch
Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-E...
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Py...
Explorations into the recently proposed Taylor Series Linear Attention
Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Fun...
Exploring an idea where one forgets about efficiency and carries out attention across each edge o...
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and...