CUDA-Learn-Notes

🎉 Modern CUDA Learn Notes with PyTorch: fp32/tf32, fp16/bf16, fp8/int8, flash_attn, rope, sgemm, sgemv, warp/block reduce, dot, elementwise, softmax, layernorm, rmsnorm.

GPL-3.0 License

Stars
1.3K
View Code on GitHub Visit Website
Ecosystems: Cuda, PyTorch
CUDA-Learn-Notes - v0.8

Published by DefTruth 2 months ago

What's Changed

Full Changelog: https://github.com/DefTruth/CUDA-Learn-Notes/compare/v0.7...v0.8

CUDA-Learn-Notes - CUDA Learn Note v0.6

Published by DefTruth 3 months ago

Full Changelog: https://github.com/DefTruth/CUDA-Learn-Notes/compare/v0.5...v0.6

CUDA-Learn-Notes - CUDA Learn Notes v0.5

Published by DefTruth 4 months ago

Full Changelog: https://github.com/DefTruth/CUDA-Learn-Notes/compare/v0.3...v0.5

CUDA-Learn-Notes - v0.3 flash_attn-1 fwd f32

Published by DefTruth 7 months ago

Full Changelog: https://github.com/DefTruth/CUDA-Learn-Note/compare/v0.2...v0.3

CUDA-Learn-Notes - CUDA Learn Note v0.2

Published by DefTruth 7 months ago

Full Changelog: https://github.com/DefTruth/CUDA-Learn-Note/compare/v0.1...v0.2

CUDA-Learn-Notes - CUDA Learn Note v0.1

Published by DefTruth 9 months ago

CUDA Learn Note v0.1

Related Projects
thorough-pytorch

PyTorch入门教程，在线阅读地址：https://datawhalechina.github.io/thorough-pytorch/

31 Jul 2021 2,433
PyTorchTricks

Some tricks of pytorch...

23 Dec 2019 1,153
DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for...

01 Jun 2018 4,992
pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

13 Aug 2016 77,536
tensor4

tensor4 - pytorch to C++ convertor using lightweight templated tensor library

22 Sep 2018 28
DeepLearningExamples

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reprod...

02 May 2018 13,307
TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating poin...

20 Sep 2022 1,482
ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform

30 Jun 2017 19,444
the-incredible-pytorch

The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relat...

11 Feb 2017 11,389
code-of-learn-deep-learning-with-pytorch

This is code of book "Learn Deep Learning with PyTorch"

20 Jun 2017 2,740
poolformer

PoolFormer: MetaFormer Is Actually What You Need for Vision (CVPR 2022 Oral)

22 Nov 2021 1,287
ppq

PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.

30 Dec 2021 1,523
pytorch2c

A Python module for compiling PyTorch graphs to C

15 Feb 2017 89
KuiperInfer

校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-perfor...

21 Nov 2022 2,418