Projects
NWT-pytorch
Implementation of NWT, audio-to-video generation, in Pytorch
Python - Released: 09 Jun 2021 - 87
anymal-belief-state-encoder-decoder-pytorch
Implementation of the Belief State Encoder / Decoder in the new breakthrough robotics paper from ETH Zürich
Python - Released: 26 Jan 2022 - 62
uformer-pytorch
Implementation of Uformer, Attention-based Unet, in Pytorch
Python - Released: 17 Jun 2021 - 93
hamburger-pytorch
Pytorch implementation of the hamburger module from the ICLR 2021 paper "Is Attention Better Than Matrix Decomposition"
Python - Released: 11 Nov 2020 - 98
marge-pytorch
Implementation of Marge, Pre-training via Paraphrasing, in Pytorch
Python - Released: 24 Aug 2020 - 75
memory-compressed-attention
Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"
Python - Released: 25 Jul 2020 - 71
geometric-vector-perceptron
Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021
Python - Released: 13 Jan 2021 - 72
n-grammer-pytorch
Implementation of N-Grammer, augmenting Transformers with latent n-grams, in Pytorch
Python - Released: 28 Nov 2021 - 72
ponder-transformer
Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper
Python - Released: 25 Aug 2021 - 78
complex-valued-transformer
Implementation of the transformer proposed in "Building Blocks for a Complex-Valued Transformer Architecture"
Python - Released: 06 Oct 2023 - 57
HTM-pytorch
Implementation of Hierarchical Transformer Memory (HTM) for Pytorch
Python - Released: 14 Sep 2021 - 72
mogrifier
Usable implementation of Mogrifier, a circuit for enhancing LSTMs and potentially other networks, from Deepmind
Python - Released: 05 Jul 2020 - 15
isab-pytorch
An implementation of (Induced) Set Attention Block, from the Set Transformers paper
Python - Released: 26 Oct 2020 - 53
JEPA-pytorch
Implementation of JEPA, Yann LeCun's vision of how AGI would be built, in Pytorch
Python - Released: 21 Aug 2022 - 90
ddpm-ipa-protein-generation
Implementation of the DDPM + IPA (invariant point attention) for protein generation, as outlined in the paper "Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models"
Python - Released: 31 May 2022 - 86
pause-transformer
Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount of time on any token
Python - Released: 18 Oct 2023 - 42
omninet-pytorch
Implementation of OmniNet, Omnidirectional Representations from Transformers, in Pytorch
Python - Released: 02 Mar 2021 - 53
multistream-transformers
Implementation of Multistream Transformers in Pytorch
Python - Released: 29 Jul 2021 - 54
molecule-attention-transformer
Pytorch reimplementation of Molecule Attention Transformer, which uses a transformer to tackle the graph-like structure of molecules
Python - Released: 30 Nov 2020 - 58
rela-transformer
Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012
Python - Released: 10 Jan 2022 - 49
cross-transformers-pytorch
Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch
Python - Released: 11 Dec 2020 - 51
compositional-attention-pytorch
Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process with disentangled search and retrieval head aggregation, in Pytorch
Python - Released: 09 May 2022 - 50