perceiver-ar-pytorch

Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch

MIT License

Downloads
187
Stars
86
Committers
2
View Code on GitHub
Ecosystems: Python

Bot releases are visible (Hide)

perceiver-ar-pytorch - 0.0.10 Latest Release

Published by lucidrains over 1 year ago

perceiver-ar-pytorch - v0.0.9

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.8

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.7

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.6a

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.6

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.5

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.4

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.3

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.2

Published by lucidrains over 2 years ago

perceiver-ar-pytorch - v0.0.1a

Published by lucidrains over 2 years ago

Package Rankings
Top 16.4% on Pypi.org
Related Projects
EasyContext

Memory optimization and training recipes to extrapolate language models' context length to 1 mill...

05 Apr 2024 574
long-short-transformer

Implementation of Long-Short Transformer, combining local and global inductive biases for attenti...

07 Jul 2021 116
perceiver-pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

05 Mar 2021 1,049
PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...

09 Dec 2022 7,595
g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

18 May 2021 422
local-attention

An implementation of local windowed attention for language modeling

05 Jul 2020 373
dilated-attention-pytorch

(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000...

09 Jul 2023 47
hourglass-transformer-pytorch

Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI

08 Nov 2021 74
perceiver-io

Unofficial implementation of Perceiver IO

28 Aug 2021 117
bidirectional-cross-attention

A simple cross attention that updates both the source and target in one step

27 Mar 2022 143
feedback-transformer-pytorch

Implementation of Feedback Transformer in Pytorch

02 Feb 2021 104
g-mlp-gpt

GPT, but made only out of MLPs

20 May 2021 86
block-recurrent-transformer-pytorch

Implementation of Block Recurrent Transformer - Pytorch

07 Feb 2023 212
omninet-pytorch

Implementation of OmniNet, Omnidirectional Representations from Transformers, in Pytorch

02 Mar 2021 53
Perceiver

Implementation of Perceiver, General Perception with Iterative Attention in TensorFlow

12 Apr 2021 86