CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

MIT License

Downloads

59.7K

Stars

223

Committers

View Code on GitHub

Ecosystems: Python

CoLT5-attention - 0.11.1 Latest Release

Published by lucidrains about 2 months ago

Full Changelog: https://github.com/lucidrains/CoLT5-attention/compare/0.11.0...0.11.1

CoLT5-attention - 0.11.0

Published by lucidrains 4 months ago

Full Changelog: https://github.com/lucidrains/CoLT5-attention/compare/0.10.19...0.11.0

CoLT5-attention - 0.10.20

Published by lucidrains 8 months ago

CoLT5-attention - 0.10.19

Published by lucidrains 11 months ago

CoLT5-attention - 0.10.18

Published by lucidrains 11 months ago

CoLT5-attention - 0.10.17

Published by lucidrains 11 months ago

CoLT5-attention - 0.10.16

Published by lucidrains 11 months ago

CoLT5-attention - 0.10.15

Published by lucidrains about 1 year ago

CoLT5-attention - 0.10.14

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.12

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.11

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.10

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.9

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.8

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.7

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.6

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.5

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.4

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.3

Published by lucidrains over 1 year ago

CoLT5-attention - 0.10.2

Published by lucidrains over 1 year ago

Package Rankings

Top 8.23% on Pypi.org

Related Projects

DALLE-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

05 Jan 2021 5,563

triton-transformer

Implementation of a Transformer, but completely in Triton

08 Sep 2021 243

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101

muse-maskgit-pytorch

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

03 Jan 2023 860

ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

14 Feb 2024 457

parti-pytorch

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

22 Jun 2022 522

iTransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...

11 Oct 2023 429

PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...

09 Dec 2022 7,595

simple-hierarchical-transformer

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

06 Apr 2023 204

performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

03 Oct 2020 1,084

audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Re...

09 Sep 2022 2,399

equiformer-pytorch

Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and...

29 Oct 2022 242

phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 min...

29 Sep 2022 747

flash-attention

Fast and memory-efficient exact attention

19 May 2022 11,791

meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

29 Nov 2023 642