coordinate-descent-hierarchical-memory

Implementation of a hierarchical memory module using coordinate descent routing

MIT License

Stars
17
View Code on GitHub
Ecosystems: Python

No README available, please check again later.

Related Projects
coordinate-descent-attention

Implementation of an Attention layer where each head can attend to more than just one token, usin...

31 Mar 2023 46
mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101
ESBN-pytorch

Usable implementation of Emerging Symbol Binding Network (ESBN), in Pytorch

01 Jan 2021 23
bidirectional-cross-attention

A simple cross attention that updates both the source and target in one step

27 Mar 2022 145
memory-efficient-attention-pytorch

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attenti...

03 Mar 2022 356
robocat-pytorch

Implementation of Deepmind's RoboCat, Self-Improving Foundation Agent for Robotic Manipulation, i...

20 Jun 2023 32
videogigagan-pytorch

Implementation of VideoGigaGAN, SOTA video upsampling out of Adobe AI labs, in Pytorch

03 May 2024 62
HTM-pytorch

Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

14 Sep 2021 72
memory-compressed-attention

Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizin...

25 Jul 2020 71
memory-transformer-xl

A variant of Transformer-XL where the memory is updated not with a queue, but with attention

10 Jul 2020 45
decomp-opt-pytorch

Implementation of DecompOpt - Controllable and Decomposed Diffusion Models for Structure-based Mo...

21 Jan 2024 28
llama-qrlhf

Implementation of the Llama architecture with RLHF + Q-learning

23 Nov 2023 155
ReST-EM-pytorch

Implementations and explorations into the ReST𝐸𝑀 algorithm in the new deepmind paper "Beyond Huma...

05 Jan 2024 39
h-transformer-1d

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

28 Jul 2021 153
JEPA-pytorch

Implementation of JEPA, Yann LeCun's vision of how AGI would be built, in Pytorch

21 Aug 2022 90