pause-transformer

Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount of time on any token

MIT License

Downloads
179
Stars
42
Committers
1
View Code on GitHub
Ecosystems: Python
pause-transformer - 0.0.7 Latest Release

Published by lucidrains about 1 year ago

pause-transformer - 0.0.6

Published by lucidrains about 1 year ago

pause-transformer - 0.0.5

Published by lucidrains about 1 year ago

pause-transformer - 0.0.4

Published by lucidrains about 1 year ago

pause-transformer - 0.0.3

Published by lucidrains about 1 year ago

pause-transformer - 0.0.2

Published by lucidrains about 1 year ago

pause-transformer - 0.0.1

Published by lucidrains about 1 year ago

Package Rankings
Top 30.07% on Pypi.org
Related Projects
h-transformer-1d

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

28 Jul 2021 153
iTransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...

11 Oct 2023 429
esbn-transformer

An attempt to merge ESBN with Transformers, to endow Transformers with the ability to emergently ...

07 Jun 2021 14
gateloop-transformer

Implementation of GateLoop Transformer in Pytorch and Jax

06 Nov 2023 86
simple-hierarchical-transformer

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

06 Apr 2023 204
PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...

09 Dec 2022 7,595
complex-valued-transformer

Implementation of the transformer proposed in "Building Blocks for a Complex-Valued Transformer A...

06 Oct 2023 57
CALM-pytorch

Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composi...

09 Jan 2024 167
parti-pytorch

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

22 Jun 2022 522
local-attention

An implementation of local windowed attention for language modeling

05 Jul 2020 375
panoptic-transformer

Another attempt at a long-context / efficient transformer by me

22 Nov 2021 37
quartic-transformer

Exploring an idea where one forgets about efficiency and carries out attention across each edge o...

03 Feb 2024 43
q-transformer

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Fun...

20 Sep 2023 338
taylor-series-linear-attention

Explorations into the recently proposed Taylor Series Linear Attention

23 Dec 2023 88
ponder-transformer

Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

25 Aug 2021 78