nystrom-attention

Implementation of Nyström Self-attention, from the paper Nyströmformer

MIT License

Downloads
2.4K
Stars
120
Committers
3
View Code on GitHub
Ecosystems: Python

Bot releases are hidden (Show)

nystrom-attention - 0.0.12 Latest Release

Published by lucidrains 9 months ago

Full Changelog: https://github.com/lucidrains/nystrom-attention/compare/0.0.11...0.0.12

nystrom-attention - 0.0.11

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.10

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.9

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.8

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.7

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.6

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.5

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.4

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.3

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.2

Published by lucidrains over 3 years ago

nystrom-attention - 0.0.1

Published by lucidrains over 3 years ago

Package Rankings
Top 38.68% on Conda-forge.org
Top 8.07% on Pypi.org
Related Projects
self-attention-cv

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

31 Jan 2021 1,171
molecule-attention-transformer

Pytorch reimplementation of Molecule Attention Transformer, which uses a transformer to tackle th...

30 Nov 2020 58
deformable-attention

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deforma...

17 Mar 2022 273
dilated-attention-pytorch

(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000...

09 Jul 2023 47
make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

29 Sep 2022 1,852
performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

03 Oct 2020 1,084
kronecker-attention-pytorch

Implementation of Kronecker Attention in Pytorch

27 Aug 2020 17
local-attention

An implementation of local windowed attention for language modeling

05 Jul 2020 373
g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

18 May 2021 422
point-transformer-pytorch

Implementation of the Point Transformer layer, in Pytorch

18 Dec 2020 587
flash-attention-jax

Implementation of Flash Attention in Jax

12 Jul 2022 189
memory-transformer-xl

A variant of Transformer-XL where the memory is updated not with a queue, but with attention

10 Jul 2020 45
AoA-pytorch

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Vi...

07 Nov 2020 40
memory-efficient-attention-pytorch

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attenti...

03 Mar 2022 355
flash-attention

Fast and memory-efficient exact attention

19 May 2022 11,791