adamw_bfloat16

diffusers-torchao

End-to-end recipes for optimizing diffusion models with torchao and diffusers (inference and FP8 ...

esgd

ESGD-M is a stochastic non-convex second order optimizer, suitable for training deep learning mod...

smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

FlexiGen

Running large language models on a single GPU for throughput-oriented scenarios.

minimal-gpt-neox-20b

Complex-YOLOv4-Pytorch

The PyTorch Implementation based on YOLOv4 of the paper: "Complex-YOLO: Real-time 3D Object Detec...

GaLore

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Adan-pytorch

Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

minimal-opt

lion-tf

A TensorFlow implementation of the Lion optimizer

quantized-training

Explore training for quantized models

lion-pytorch

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly bet...

ali-pytorch

Adversarially Learned Inference in Pytorch

MAE-pytorch

Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners

ml-design-patterns

Software Architecture for ML engineers