The full training script for Enformer - Tensorflow Sonnet
APACHE-2.0 License
A simple and working implementation of Electra, the fastest way to pretrain language models from ...
PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO
Implementation and replication of ProGen, Language Modeling for Protein Generation, in Jax
Implementation of Alphafold 3 in Pytorch
Longformer: The Long-Document Transformer
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transfo...
A repository with exploration into using transformers to predict DNA ↔ transcription factor binding
GLM (General Language Model)
Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch
Implementation of ProteinBERT in Pytorch
My own attempt at a long context genomics model, leveraging recent advances in long context atten...
Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2