kernl

0.2.2 (2023-02-17)

Bug Fixes

• fix 0 stride in CG pool management (#291) (4e4cf69)

0.2.1 (2023-02-08)

Bug Fixes

• fix release version (#277) (2a553e0)

Documentation

• add dynamo code to notebook as a comment (#274) (55cb686)

New features

• Whisper support
• New debugger
• Improve warmup speed
• Improve CUDA graph memory allocation

What's Changed

• fix: python version by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/124
• fix: change python version by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/126
• feat: layernorm rms replacement for T5 by @gaetansnl in https://github.com/ELS-RD/kernl/pull/107
• docs: update install steps by @Thytu in https://github.com/ELS-RD/kernl/pull/128
• fix: pin Pytorch version to 1.12.1 by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/130
• feat: update notebook T5 by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/127
• fix: pinpoint pytorch in Docker image by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/132
• feat: add ort benchmark script in xp folder by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/139
• fix: remove fx profiler by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/144
• feat: update to PyTorch 1.14 by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/134
• fix: rename all close by @gaetansnl in https://github.com/ELS-RD/kernl/pull/147
• feat: improve debugger api by @ayoub-louati in https://github.com/ELS-RD/kernl/pull/116
• docs: add static documentation website by @white-gorilla in https://github.com/ELS-RD/kernl/pull/112
• fix: removes the noindex meta. by @white-gorilla in https://github.com/ELS-RD/kernl/pull/157
• feat: add stale workflow by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/155
• fix: attention bug with large values by @gaetansnl in https://github.com/ELS-RD/kernl/pull/158
• feat: only close issues marked as questions by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/162
• feat: monthly triton update by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/160
• feat: simplify tests by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/159
• feat: check stability of output by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/163
• docs: install instructions simplification by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/150
• feat: better precision when the mask is big (covers lots of tokens) by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/168
• feat: remove nvfuser by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/175
• fix: bug where some specific shapes make inference on cuda graphs crashing by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/177
• feat: add autotune to optimize attention speed by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/172
• feat: implements highlighted features in the static site. by @white-gorilla in https://github.com/ELS-RD/kernl/pull/171
• feat: add google analytics (ga4) and cookie consent. by @white-gorilla in https://github.com/ELS-RD/kernl/pull/186
• fix: layernorm stride by @gaetansnl in https://github.com/ELS-RD/kernl/pull/190
• fix: support stride for layernorm when dim is more than 3 by @gaetansnl in https://github.com/ELS-RD/kernl/pull/192
• feat: add performance dynamic chart. by @white-gorilla in https://github.com/ELS-RD/kernl/pull/191
• docs: add some code reference by @jonathlela in https://github.com/ELS-RD/kernl/pull/199
• feat: attention for skinny Q tensor by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/198
• fix: site deployment workflow miss mkdocstrings python handler by @jonathlela in https://github.com/ELS-RD/kernl/pull/209
• feat: openai whisper support by @gaetansnl in https://github.com/ELS-RD/kernl/pull/122
• fix: disabled search field on landing page. by @white-gorilla in https://github.com/ELS-RD/kernl/pull/210
• feat: update triton by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/211
• fix: missing dependency by @gaetansnl in https://github.com/ELS-RD/kernl/pull/218
• feat: update torch and modify cuda graphs by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/220
• fix: fix flake8 config by @jonathlela in https://github.com/ELS-RD/kernl/pull/231
• feat: reduce cg memory footprint by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/225
• fix: pin protobuf dependency to bert tutorial by @jonathlela in https://github.com/ELS-RD/kernl/pull/221
• docs: contribution guide. by @white-gorilla in https://github.com/ELS-RD/kernl/pull/222
• fix: stop regenerating graph module by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/235
• fix: test crashing on cuda graph by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/243
• fix: update README with GPU requirements by @jonathlela in https://github.com/ELS-RD/kernl/pull/251
• feat: add navigation footer. by @white-gorilla in https://github.com/ELS-RD/kernl/pull/250
• fix: attention refactor and base doc by @gaetansnl in https://github.com/ELS-RD/kernl/pull/216
• fix: update pytorch nightly to fix memory leak by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/255
• docs: how to support a new model tutorial by @jonathlela in https://github.com/ELS-RD/kernl/pull/181
• feat: replace whisper script by notebook + test fix by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/257
• fix: update dep + fix by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/265
• feat: reduce memory overhead in CUDA Graph by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/268
• feat: add links to tutorials on README.md by @pommedeterresautee in https://github.com/ELS-RD/kernl/pull/269
• feat: new debugger by @gaetansnl in https://github.com/ELS-RD/kernl/pull/196
• fix: remove old debugger doc by @jonathlela in https://github.com/ELS-RD/kernl/pull/271

New Contributors

• @Thytu made their first contribution in https://github.com/ELS-RD/kernl/pull/128
• @jonathlela made their first contribution in https://github.com/ELS-RD/kernl/pull/199

Full Changelog: https://github.com/ELS-RD/kernl/compare/v0.1.0...v0.2.0

• covers Bert (and derivatives), and partially T5
• 3 kernels: Flash Attention, LayerNorm, LinearLayer
• debug mode
• tutorials