self-reasoning-tokens-pytorch

Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto

MIT License

Downloads
88
Stars
53
Committers
2
View Code on GitHub
Ecosystems: Python

Commit Statistics

Past Year

All Time

Total Commits
11
11
Total Committers
2
2
Avg. Commits Per Committer
5.5
5.5
Bot Commits
0
0

Issue Statistics

Past Year

All Time

Total Pull Requests
1
1
Merged Pull Requests
1
1
Total Issues
0
0
Time to Close Issues
N/A
N/A
Package Rankings
Top 35.91% on Pypi.org
Related Projects
pause-transformer

Yet another random morning idea to be quickly tried and architecture shared if it works; to allow...

18 Oct 2023 42
token-shift-gpt

Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the se...

17 Aug 2021 47
hourglass-transformer-pytorch

Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI

08 Nov 2021 82
coordinate-descent-attention

Implementation of an Attention layer where each head can attend to more than just one token, usin...

31 Mar 2023 46
charformer-pytorch

Implementation of the GBST block from the Charformer paper, in Pytorch

30 Jun 2021 116
CALM-pytorch

Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composi...

09 Jan 2024 167
magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

10 Oct 2023 552
infini-transformer-pytorch

Implementation of Infini-Transformer in Pytorch

01 May 2024 102
llm-reasoners

A library for advanced large language model reasoning

01 Jun 2023 1,194
fast-transformer-pytorch

Implementation of Fast Transformer in Pytorch

23 Aug 2021 171
feedback-transformer-pytorch

Implementation of Feedback Transformer in Pytorch

02 Feb 2021 104
block-recurrent-transformer-pytorch

Implementation of Block Recurrent Transformer - Pytorch

07 Feb 2023 212
mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mi...

21 Apr 2023 101
agent-attention-pytorch

Implementation of Agent Attention in Pytorch

18 Dec 2023 85
local-attention

An implementation of local windowed attention for language modeling

05 Jul 2020 375