st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

MIT License

Downloads
1.5K
Stars
285
Committers
1
View Code on GitHub
Ecosystems: Python
st-moe-pytorch - 0.0.17

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.16

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.15

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.14

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.12

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.11

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.10

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.9

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.8

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.7

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.6

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.5

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.3

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.2

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.1

Published by lucidrains about 1 year ago

Package Rankings
Top 21.44% on Pypi.org
Related Projects
fastmoe

A fast MoE impl for PyTorch

25 Jan 2021 1,534
long_llama

LongLLaMA is a large language model capable of handling long contexts. It is based on OpenLLaMA a...

06 Jul 2023 1,448
diffengine

Diffusers training with mmengine

28 Jul 2023 79
sinkhorn-router-pytorch

Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or other...

23 Aug 2024 31
PEER-pytorch

Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen...

09 Jul 2024 109
the-incredible-pytorch

The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relat...

11 Feb 2017 11,389
soft-moe-pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

04 Aug 2023 239
mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the param...

13 Jul 2020 624
MoE-LLaVA

Mixture-of-Experts for Large Vision-Language Models

14 Dec 2023 1,932
mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101
MoA

Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models

04 Jun 2024 2,554
minichatgpt

minichatgpt - To Train ChatGPT In 5 Minutes

23 Feb 2023 155
diffusers-torchao

End-to-end recipes for optimizing diffusion models with torchao and diffusers (inference and FP8 ...

05 Aug 2024 213
OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

08 Aug 2023 1,368
megablocks

MegaBlocks

26 Jan 2023 1,076