PEER-pytorch

Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind

MIT License

Downloads
1.3K
Stars
109
Committers
1
View Code on GitHub
Ecosystems: Python
PEER-pytorch - 0.0.4 Latest Release

Published by lucidrains 4 months ago

PEER-pytorch - 0.0.2

Published by lucidrains 4 months ago

PEER-pytorch - 0.0.1

Published by lucidrains 4 months ago

Package Rankings
Top 35.35% on Pypi.org
Related Projects
sinkhorn-router-pytorch

Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or other...

23 Aug 2024 31
fast-transformer-pytorch

Implementation of Fast Transformer in Pytorch

23 Aug 2021 171
remixer-pytorch

Implementation of the Remixer Block from the Remixer paper, in Pytorch

24 Sep 2021 35
st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

26 Mar 2023 285
ali-pytorch

Adversarially Learned Inference in Pytorch

24 Mar 2017 30
MEGABYTE-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Py...

15 May 2023 620
feedback-transformer-pytorch

Implementation of Feedback Transformer in Pytorch

02 Feb 2021 104
mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101
graph-transformer-pytorch

Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2

18 Jun 2021 197
mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the param...

13 Jul 2020 624
g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

18 May 2021 422
bottleneck-transformer-pytorch

Implementation of Bottleneck Transformer in Pytorch

28 Jan 2021 670
gamengen-pytorch

Implementation of a framework for Gamengen in Pytorch

28 Aug 2024 89
soft-moe-pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

04 Aug 2023 239
mirasol-pytorch

Implementation of 🌻 Mirasol, SOTA Multimodal Autoregressive model out of Google Deepmind, in Pytorch

18 Nov 2023 84