quantization

AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. D...

NWT-pytorch

Implementation of NWT, audio-to-video generation, in Pytorch

QSNNs

Quantization-aware training with spiking neural networks

AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantiz...

diffusers-torchao

End-to-end recipes for optimizing diffusion models with torchao and diffusers (inference and FP8 ...

GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ

smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

SpQR

bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.

torchprep

quantized-training

Explore training for quantized models

discrete-key-value-bottleneck-pytorch

Implementation of Discrete Key / Value Bottleneck, in Pytorch

encodec

State-of-the-art deep learning based audio codec supporting both mono 24 kHz audio and stereo 48 ...

neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techn...

usc_dae

Repository for Unsupervised Sentence Compression using Denoising Auto-Encoders