compositional-attention-pytorch

Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process with disentangled search and retrieval head aggregation, in Pytorch

MIT License

Downloads
35
Stars
50
Committers
1
View Code on GitHub
Ecosystems: Python

Commit Statistics

Past Year

All Time

Total Commits
0
8
Total Committers
0
1
Avg. Commits Per Committer
0.0
8.0
Bot Commits
0
0

Issue Statistics

Past Year

All Time

Total Pull Requests
0
0
Merged Pull Requests
0
0
Total Issues
0
0
Time to Close Issues
N/A
N/A
Package Rankings
Top 23.64% on Pypi.org
Related Projects
Compositional-Attention

An implementation of Compositional Attention: Disentangling Search and Retrieval by MILA

31 May 2022 14
memory-efficient-attention-pytorch

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attenti...

03 Mar 2022 356
slot-attention

Implementation of Slot Attention from GoogleAI

29 Jun 2020 384
halonet-pytorch

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Paramete...

24 Mar 2021 200
nystrom-attention

Implementation of Nyström Self-attention, from the paper Nyströmformer

11 Feb 2021 121
memory-compressed-attention

Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizin...

25 Jul 2020 71
kronecker-attention-pytorch

Implementation of Kronecker Attention in Pytorch

27 Aug 2020 17
AoA-pytorch

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Vi...

07 Nov 2020 40
mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101
agent-attention-pytorch

Implementation of Agent Attention in Pytorch

18 Dec 2023 85
coordinate-descent-attention

Implementation of an Attention layer where each head can attend to more than just one token, usin...

31 Mar 2023 46
bidirectional-cross-attention

A simple cross attention that updates both the source and target in one step

27 Mar 2022 145
perceiver-pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

05 Mar 2021 1,049
make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

29 Sep 2022 1,852
isab-pytorch

An implementation of (Induced) Set Attention Block, from the Set Transformers paper

26 Oct 2020 53