Phil Wang

Working with Attention. It's all we need

Ecosystems: Python, PyTorch, Crystal, Cuda

Projects

vit-pytorch

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Python - Released: 03 Oct 2020 - 20,058

DALLE2-pytorch

Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch

Python - Released: 07 Apr 2022 - 11,101

imagen-pytorch

Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

Python - Released: 23 May 2022 - 8,031

denoising-diffusion-pytorch

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

Python - Released: 26 Aug 2020 - 7,187

x-transformers

A simple but complete full-attention transformer with a set of promising experimental features from various papers

Python - Released: 24 Oct 2020 - 4,226

stylegan2-pytorch

Simplest working implementation of Stylegan2, state of the art generative adversarial network, in Pytorch. Enabling everyone to experience disentanglement

Python - Released: 09 Jan 2020 - 3,705

lion-pytorch

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

Python - Released: 15 Feb 2023 - 2,018

DALLE-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Python - Released: 05 Jan 2021 - 5,563

PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Python - Released: 09 Dec 2022 - 7,595

toolformer-pytorch

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

Python - Released: 10 Feb 2023 - 1,948

make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

Python - Released: 29 Sep 2022 - 1,852

soundstorm-pytorch

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

Python - Released: 17 May 2023 - 1,370

deep-daze

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun

Python - Released: 17 Jan 2021 - 4,371

audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

Python - Released: 09 Sep 2022 - 2,399

alphafold3-pytorch

Implementation of Alphafold 3 in Pytorch

Python - Released: 08 May 2024 - 881

video-diffusion-pytorch

Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch

Python - Released: 08 Apr 2022 - 1,235

vector-quantize-pytorch

Vector (and Scalar) Quantization, in Pytorch

Python - Released: 09 Jun 2020 - 2,165

rotary-embedding-torch

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

Python - Released: 29 Jun 2021 - 547

musiclm-pytorch

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

Python - Released: 27 Jan 2023 - 3,139

mlp-mixer-pytorch

An All-MLP solution for Vision, from Google AI

Python - Released: 05 May 2021 - 992

gigagan-pytorch

Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

Python - Released: 10 Mar 2023 - 1,813

ema-pytorch

A simple way to keep track of an Exponential Moving Average (EMA) version of your pytorch model

Python - Released: 20 Jun 2022 - 497

reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

Python - Released: 09 Jan 2020 - 2,105

big-sleep

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun

Python - Released: 18 Jan 2021 - 2,564

meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

Python - Released: 29 Nov 2023 - 642