model-soups-text-classification

Shows an implementation of model soups (https://arxiv.org/abs/2203.05482) for text classification models.

APACHE-2.0 License

Stars

6

View Code on GitHub View on X

Ecosystems: Jupyter Notebook

Model Soups for Text Classification

Shows an implementation of model soups [1] for text classification models.

Notebooks

notebooks/train-models.ipynb shows how to obtain the fine-tuned text classification
models using Trainer and AutoModelForSequenceClassification classes from Transformers.
notebooks/model-soups.ipynb shows how to obtain model soups using two recipes presented in [1].
notebooks/logit-ensembling.ipynb shows how to perform the classic logit ensembling. It's here for
comparison purposes.

Results

TBA

References

[1] Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (ICML 2022)

Related Projects

multi-label-text-classification

A multi-label text classifier to predict the subject areas of arXiv papers from their abstract bo...

text-classify-with-cnn

Easy to follow text classifying implementation using a Conv. Neural Network (Tensorflow)

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

29 Oct 2018 132,459

pydata-dc-2016-tutorial

Tutorial: Machine Learning with Text in scikit-learn

Document_Classification

Python code for classification of documents into different classes using machine learning

Sentiment-Classification-Arabic-COVID-Tweets