flash-genomics-model

My own attempt at a long context genomics model, leveraging recent advances in long context attention modeling (Flash Attention + other hierarchical methods)

MIT License

Stars
51
Committers
1
View Code on GitHub
Ecosystems: Python

Commit Statistics

Past Year

All Time

Total Commits
0
10
Total Committers
0
1
Avg. Commits Per Committer
0.0
10.0
Bot Commits
0
0

Issue Statistics

Past Year

All Time

Total Pull Requests
0
1
Merged Pull Requests
0
0
Total Issues
0
3
Time to Close Issues
N/A
4 days
Related Projects
alphafold3-pytorch

Implementation of Alphafold 3 in Pytorch

08 May 2024 881
iTransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks...

11 Oct 2023 429
spear-tts-pytorch

Implementation of Spear-TTS - multi-speaker text-to-speech attention network, in Pytorch

19 Jun 2023 254
flash-attention

Fast and memory-efficient exact attention

19 May 2022 11,791
PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectu...

09 Dec 2022 7,595
enformer-pytorch

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

23 Oct 2021 420
MEGABYTE-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Py...

15 May 2023 620
naturalspeech2-pytorch

Implementation of Natural Speech 2, Zero-shot Speech and Singing Synthesizer, in Pytorch

19 Apr 2023 1,269
voicebox-pytorch

Implementation of Voicebox, new SOTA Text-to-speech network from MetaAI, in Pytorch

01 Aug 2023 599
alphafold2

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details...

01 Dec 2020 1,563
En-transformer

Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-E...

27 Feb 2021 208
tf-bind-transformer

A repository with exploration into using transformers to predict DNA ↔ transcription factor binding

08 Dec 2021 81
simple-hierarchical-transformer

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

06 Apr 2023 204
EasyContext

Memory optimization and training recipes to extrapolate language models' context length to 1 mill...

05 Apr 2024 574
magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

10 Oct 2023 552