ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

MIT License

Downloads
5.4K
Stars
457
Committers
1
View Code on GitHub
Ecosystems: Python
ring-attention-pytorch - 0.1.33

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.32...0.1.33

ring-attention-pytorch - 0.1.32

Published by lucidrains 8 months ago

What's Changed

New Contributors

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.31...0.1.32

ring-attention-pytorch - 0.1.31

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.30...0.1.31

ring-attention-pytorch - 0.1.30

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.29...0.1.30

ring-attention-pytorch - 0.1.29

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.28...0.1.29

ring-attention-pytorch - 0.1.28

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.27...0.1.28

ring-attention-pytorch - 0.1.27

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.26...0.1.27

ring-attention-pytorch - 0.1.26

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.25...0.1.26

ring-attention-pytorch - 0.1.25

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.24...0.1.25

ring-attention-pytorch - 0.1.24

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.23...0.1.24

ring-attention-pytorch - 0.1.23

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.22...0.1.23

ring-attention-pytorch - 0.1.22

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.20...0.1.22

ring-attention-pytorch - 0.1.21

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.20...0.1.21

ring-attention-pytorch - 0.1.20

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.19...0.1.20

ring-attention-pytorch - 0.1.19

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.17...0.1.19

ring-attention-pytorch - 0.1.17

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.16...0.1.17

ring-attention-pytorch - 0.1.18

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.16...0.1.18

ring-attention-pytorch - 0.1.16

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.15...0.1.16

ring-attention-pytorch - 0.1.15

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.14...0.1.15

ring-attention-pytorch - 0.1.14

Published by lucidrains 8 months ago

Full Changelog: https://github.com/lucidrains/ring-attention-pytorch/compare/0.1.12...0.1.14

Package Rankings
Top 37.37% on Pypi.org
Related Projects
local-attention

An implementation of local windowed attention for language modeling

05 Jul 2020 375
soundstorm-pytorch

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

17 May 2023 1,370
meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

29 Nov 2023 642
PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...

09 Dec 2022 7,595
performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

03 Oct 2020 1,084
simple-hierarchical-transformer

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

06 Apr 2023 204
magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

10 Oct 2023 552
phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 min...

29 Sep 2022 747
equiformer-pytorch

Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and...

29 Oct 2022 242
iTransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...

11 Oct 2023 429
flash-attention

Fast and memory-efficient exact attention

19 May 2022 11,791
En-transformer

Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-E...

27 Feb 2021 208
audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Re...

09 Sep 2022 2,399
CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

20 Mar 2023 223
nuwa-pytorch

Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch

28 Nov 2021 540