Projects
muse-maskgit-pytorch
Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch
Python - Released: 03 Jan 2023 - 860
RETRO-pytorch
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
Python - Released: 15 Jan 2022 - 849
CoCa-pytorch
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
Python - Released: 05 May 2022 - 977
CoLT5-attention
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
Python - Released: 20 Mar 2023 - 223
st-moe-pytorch
Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
Python - Released: 26 Mar 2023 - 285
autoregressive-diffusion-pytorch
Implementation of Autoregressive Diffusion in Pytorch
Python - Released: 23 Jul 2024 - 265
x-unet
Implementation of a U-net complete with efficient attention as well as the latest research findings
Python - Released: 23 Mar 2022 - 259
x-clip
A concise but complete implementation of CLIP with various experimental improvements from recent papers
Python - Released: 01 Dec 2021 - 686
PaLM-pytorch
Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways
Python - Released: 04 Apr 2022 - 822
lumiere-pytorch
Implementation of Lumiere, SOTA text-to-video generation from Google Deepmind, in Pytorch
Python - Released: 05 Feb 2024 - 243
mmdit
Implementation of a single layer of the MMDiT, proposed in Stable Diffusion 3, in Pytorch
Python - Released: 04 May 2024 - 244
block-recurrent-transformer-pytorch
Implementation of Block Recurrent Transformer - Pytorch
Python - Released: 07 Feb 2023 - 212
memorizing-transformers-pytorch
Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch
Python - Released: 21 Mar 2022 - 623
simple-hierarchical-transformer
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
Python - Released: 06 Apr 2023 - 204
mixture-of-experts
A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models
Python - Released: 13 Jul 2020 - 624
En-transformer
Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network
Python - Released: 27 Feb 2021 - 208
TimeSformer-pytorch
Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification
Python - Released: 11 Feb 2021 - 689
CALM-pytorch
Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind
Python - Released: 09 Jan 2024 - 167
egnn-pytorch
Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch
Python - Released: 26 Feb 2021 - 414
rectified-flow-pytorch
Implementation of rectified flow and some of its followup research / improvements in Pytorch
Python - Released: 17 Jul 2024 - 161
nuwa-pytorch
Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch
Python - Released: 28 Nov 2021 - 540
bottleneck-transformer-pytorch
Implementation of Bottleneck Transformer in Pytorch
Python - Released: 28 Jan 2021 - 670
parti-pytorch
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
Python - Released: 22 Jun 2022 - 522
siren-pytorch
Pytorch implementation of SIREN - Implicit Neural Representations with Periodic Activation Function
Python - Released: 19 Jun 2020 - 465
bidirectional-cross-attention
A simple cross attention that updates both the source and target in one step
Python - Released: 27 Mar 2022 - 145