PEER-pytorch

Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind

MIT License

Downloads
1.3K
Stars
109
Committers
1
View Code on GitHub
Ecosystems: Python

Commit Statistics

Past Year

All Time

Total Commits
26
26
Total Committers
1
1
Avg. Commits Per Committer
26.0
26.0
Bot Commits
0
0

Issue Statistics

Past Year

All Time

Total Pull Requests
0
0
Merged Pull Requests
0
0
Total Issues
4
4
Time to Close Issues
N/A
N/A
Package Rankings
Top 35.35% on Pypi.org
Related Projects
mirasol-pytorch

Implementation of 🌻 Mirasol, SOTA Multimodal Autoregressive model out of Google Deepmind, in Pytorch

18 Nov 2023 84
graph-transformer-pytorch

Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2

18 Jun 2021 197
mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the param...

13 Jul 2020 624
remixer-pytorch

Implementation of the Remixer Block from the Remixer paper, in Pytorch

24 Sep 2021 35
sinkhorn-router-pytorch

Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or other...

23 Aug 2024 31
feedback-transformer-pytorch

Implementation of Feedback Transformer in Pytorch

02 Feb 2021 104
MEGABYTE-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Py...

15 May 2023 620
ali-pytorch

Adversarially Learned Inference in Pytorch

24 Mar 2017 30
g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

18 May 2021 422
bottleneck-transformer-pytorch

Implementation of Bottleneck Transformer in Pytorch

28 Jan 2021 670
fast-transformer-pytorch

Implementation of Fast Transformer in Pytorch

23 Aug 2021 171
mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101
soft-moe-pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

04 Aug 2023 239
st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

26 Mar 2023 285
gamengen-pytorch

Implementation of a framework for Gamengen in Pytorch

28 Aug 2024 89