nystrom-attention

Implementation of Nyström Self-attention, from the paper Nyströmformer

MIT License

Downloads
2.4K
Stars
120
Committers
3
View Code on GitHub
Ecosystems: Python

Commit Statistics

Past Year

All Time

Total Commits
1
20
Total Committers
1
3
Avg. Commits Per Committer
1.0
6.67
Bot Commits
0
0

Issue Statistics

Past Year

All Time

Total Pull Requests
0
2
Merged Pull Requests
0
2
Total Issues
0
10
Time to Close Issues
N/A
about 2 months
Package Rankings
Top 38.68% on Conda-forge.org
Top 8.07% on Pypi.org
Related Projects
point-transformer-pytorch

Implementation of the Point Transformer layer, in Pytorch

18 Dec 2020 587
deformable-attention

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deforma...

17 Mar 2022 273
memory-efficient-attention-pytorch

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attenti...

03 Mar 2022 355
self-attention-cv

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

31 Jan 2021 1,171
flash-attention

Fast and memory-efficient exact attention

19 May 2022 11,791
dilated-attention-pytorch

(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000...

09 Jul 2023 47
AoA-pytorch

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Vi...

07 Nov 2020 40
flash-attention-jax

Implementation of Flash Attention in Jax

12 Jul 2022 189
kronecker-attention-pytorch

Implementation of Kronecker Attention in Pytorch

27 Aug 2020 17
molecule-attention-transformer

Pytorch reimplementation of Molecule Attention Transformer, which uses a transformer to tackle th...

30 Nov 2020 58
g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

18 May 2021 422
memory-transformer-xl

A variant of Transformer-XL where the memory is updated not with a queue, but with attention

10 Jul 2020 45
local-attention

An implementation of local windowed attention for language modeling

05 Jul 2020 373
performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

03 Oct 2020 1,084
make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

29 Sep 2022 1,852