Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
MIT License
Published by lucidrains 6 months ago
Full Changelog: https://github.com/lucidrains/self-reasoning-tokens-pytorch/compare/0.0.3...0.0.4
Implementation of MagViT2 Tokenizer in Pytorch
Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI
Implementation of Block Recurrent Transformer - Pytorch
A library for advanced large language model reasoning
Implementation of Infini-Transformer in Pytorch
Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the se...
Yet another random morning idea to be quickly tried and architecture shared if it works; to allow...
Implementation of Fast Transformer in Pytorch
Implementation of Agent Attention in Pytorch
Implementation of the GBST block from the Charformer paper, in Pytorch
Some personal experiments around routing tokens to different autoregressive attention, akin to mi...
An implementation of local windowed attention for language modeling
Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composi...
Implementation of Feedback Transformer in Pytorch
Implementation of an Attention layer where each head can attend to more than just one token, usin...
