pytorch-PKD-for-BERT-compression

roberta_zh

RoBERTa中文预训练模型: RoBERTa for Chinese

Knowledge_distillation_via_TF2.0

The codes for recent knowledge distillation algorithms and benchmark results via TF2.0 low-level API

minimal-llama

distillbart-mnli

No Teacher BART distillation experiment for NLI tasks

bert_on_stilts

Fork of huggingface/pytorch-pretrained-BERT for BERT on STILTs

BertSum

Code for paper Fine-tune BERT for Extractive Summarization

transformer-smaller-training-vocab

Temporary remove unused tokens during training to save ram and speed.

MAE-pytorch

Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners

bert4keras

keras implement of transformers for humans

dataset-distillation

Open-source code for paper "Dataset Distillation"

bert-squeeze

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡

channel-distillation

PyTorch implementation for Channel Distillation

RepDistiller

[ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge dist...

rlrd-aaai2022

An official implementation of "An Efficient Combinatorial Optimization Model Using Learning-to-Ra...

Tencent2020_Rank1st

The code for 2020 Tencent College Algorithm Contest, and the online result ranks 1st.