st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

MIT License

Downloads

1.5K

Stars

285

Committers

View Code on GitHub

Ecosystems: Python

st-moe-pytorch - 0.1.8 Latest Release

Published by lucidrains 5 months ago

Full Changelog: https://github.com/lucidrains/st-moe-pytorch/compare/0.1.7...0.1.8

st-moe-pytorch - 0.1.7

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/st-moe-pytorch/compare/0.1.6...0.1.7

st-moe-pytorch - 0.1.6

Published by lucidrains 9 months ago

Full Changelog: https://github.com/lucidrains/st-moe-pytorch/compare/0.1.5...0.1.6

st-moe-pytorch - 0.1.5

Published by lucidrains 10 months ago

st-moe-pytorch - 0.1.4

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.1.2

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.1.1

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.1.0

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.30

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.29

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.28

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.27

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.25

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.24

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.23

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.22

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.21

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.20

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.19

Published by lucidrains about 1 year ago

st-moe-pytorch - 0.0.18

Published by lucidrains about 1 year ago

Package Rankings

Top 21.44% on Pypi.org

Related Projects

MoE-LLaVA

Mixture-of-Experts for Large Vision-Language Models

14 Dec 2023 1,932

megablocks

MegaBlocks

26 Jan 2023 1,076

fastmoe

A fast MoE impl for PyTorch

25 Jan 2021 1,534

minichatgpt

minichatgpt - To Train ChatGPT In 5 Minutes

23 Feb 2023 155

diffengine

Diffusers training with mmengine

the-incredible-pytorch

The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relat...

11 Feb 2017 11,389

long_llama

LongLLaMA is a large language model capable of handling long contexts. It is based on OpenLLaMA a...

06 Jul 2023 1,448

sinkhorn-router-pytorch

Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or other...

diffusers-torchao

End-to-end recipes for optimizing diffusion models with torchao and diffusers (inference and FP8 ...

05 Aug 2024 213

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101

OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

08 Aug 2023 1,368

MoA

Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models

04 Jun 2024 2,554

PEER-pytorch

Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen...

09 Jul 2024 109

soft-moe-pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

04 Aug 2023 239

mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the param...

13 Jul 2020 624