hummingbird

What's Changed

• Update apache-tvm to v0.15.0 by @mshr-h in https://github.com/microsoft/hummingbird/pull/709
• All loss functions which supports SGDClassifer are added by @giriprasad51 in https://github.com/microsoft/hummingbird/pull/758
• Forcing users to set override flag to unzip without hash integrity check by @ksaur in https://github.com/microsoft/hummingbird/pull/761
• Polynomial features will support degree from 0 to n by @giriprasad51 in https://github.com/microsoft/hummingbird/pull/763
• Steps towards xgb==2.0 support by @giriprasad51 in https://github.com/microsoft/hummingbird/pull/764
• Support for scikit-learn ==1.4.0 by @ksaur in https://github.com/microsoft/hummingbird/pull/756
• Warnings maintenance by @ksaur in https://github.com/microsoft/hummingbird/pull/757

Breaking API change for model save/load

• In https://github.com/microsoft/hummingbird/pull/761 users are now required to add an additional flag if they wish to unzip without the checksum hashes. Ex: model.load(location, override_flag=True) . This is to prevent any potential exploit of the underlying pickle module, so that users can verify the output checksum for model integrity, or set the override_flag if the model is from a trusted source.

TODOs

• We are aware that documentation is not being generated correctly (#762) due to a permissions change.
• We are continuing to investigate how to support xgb==2.0.

New Contributors

• @giriprasad51 made their first contribution in https://github.com/microsoft/hummingbird/pull/758

Special thanks

Thank you to our newest contributor @giriprasad51 for his 3 PRs, and thank you to @mshr-h for all of the effort on apache-tvm to v0.15.0 and the ongoing support!

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.10...v0.4.11

What's Changed

This maintenance release updates to Python 3.11 (for all but TVM), adds support for the current versions (as of Jan 2024) of LGBM and ONNX related libraries.

We are still working on XGB==2.0 support and TVM==0.12.

• pinning xgb to prior 2.0 by @ksaur in https://github.com/microsoft/hummingbird/pull/734
• Update devcontainer.json to match the latest property format by @mshr-h in https://github.com/microsoft/hummingbird/pull/742
• adding gather to match OHE ONNX changes by @ksaur in https://github.com/microsoft/hummingbird/pull/746
• Removing lgbm pin by @ksaur in https://github.com/microsoft/hummingbird/pull/748
• Python 3.11 by @ksaur in https://github.com/microsoft/hummingbird/pull/749

Thanks to @mshr-h for the ongoing support!

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.9...v0.4.10

What's Changed

• Upgrade PyTorch to 2.0.0 by @mshr-h in https://github.com/microsoft/hummingbird/pull/691
• Removing sklearn-onnx pin now that update is released by @ksaur in https://github.com/microsoft/hummingbird/pull/693
• Fix SGDClassifier predict result bug by @hikari234 in https://github.com/microsoft/hummingbird/pull/703
• Adding hash to save/load for pickle by @ksaur in https://github.com/microsoft/hummingbird/pull/713
• Compatibility with scikit-learn 1.3.0 by @ksaur in https://github.com/microsoft/hummingbird/pull/721

New Contributors

• @hikari234 made their first contribution in https://github.com/microsoft/hummingbird/pull/703

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.8...v0.4.9

What's Changed

New support

• Add support for sklearn.mixture.BayesianGaussianMixture by @mshr-h in https://github.com/microsoft/hummingbird/pull/677
• Allow for identity POST_TRANSFORM by @jppgks in https://github.com/microsoft/hummingbird/pull/674

Bug fixes

• Fix inhomogeneous shape error on numpy array creation by @mshr-h in https://github.com/microsoft/hummingbird/pull/667

Maintenance

• Improvements to the testing and pipeline by @mshr-h in https://github.com/microsoft/hummingbird/pull/658, https://github.com/microsoft/hummingbird/pull/665, https://github.com/microsoft/hummingbird/pull/668, https://github.com/microsoft/hummingbird/pull/681 and @ksaur in https://github.com/microsoft/hummingbird/pull/685, https://github.com/microsoft/hummingbird/pull/686
• Support for scikit-learn 1.2 by @ksaur in https://github.com/microsoft/hummingbird/pull/661
• Dependabot fixes by @ksaur in https://github.com/microsoft/hummingbird/pull/673, https://github.com/microsoft/hummingbird/pull/635

Acks

• Thank you very much to @jppgks for their first contribution in https://github.com/microsoft/hummingbird/pull/674
• Thank you to @mshr-h for the exciting new BayesianGaussianMixture feature and ongoing support!

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.7...v0.4.8

What's Changed

• This patch release fixes a bug in ONNX conversion and allows support of varying batch sizes by @stillmatic in https://github.com/microsoft/hummingbird/pull/654
• Fixes deprecations in https://github.com/microsoft/hummingbird/pull/655

New Contributors

• Thank you to @stillmatic for catching and fixing this bug (https://github.com/microsoft/hummingbird/issues/653) and for the additional maintenance work!

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.6...v0.4.7

What's Changed

Features

• Adds Lasso, Ridge and ElasticNet by @fd0r in https://github.com/microsoft/hummingbird/pull/625
• Added support for more decision conditions in trees and ONNX conversion by @grafail in https://github.com/microsoft/hummingbird/pull/631
• Add support for Tweedie, Poisson and Gamma regressors by @interesaaat in https://github.com/microsoft/hummingbird/pull/650

Maintenance and fixes

• Remove pinned version for onnxconverter-common by @interesaaat in https://github.com/microsoft/hummingbird/pull/618
• Bump action/cache to v3 by @mshr-h in https://github.com/microsoft/hummingbird/pull/619
• Fix deprecation warnings for sklearn/scipy by @mshr-h in https://github.com/microsoft/hummingbird/pull/610
• deprecating email due to spammers.... by @ksaur in https://github.com/microsoft/hummingbird/pull/621
• updating ubuntu version by @ksaur in https://github.com/microsoft/hummingbird/pull/623
• Fix linear models conversion when fit_intercept set to False by @RomanBredehoft in https://github.com/microsoft/hummingbird/pull/630
• Corrects some typos by @RomanBredehoft in https://github.com/microsoft/hummingbird/pull/628
• allow derived types of DataFrame by @liangfu in https://github.com/microsoft/hummingbird/pull/637
• [TVM] Unify load params interface by @liangfu in https://github.com/microsoft/hummingbird/pull/640
• Update TVM to 0.10 by @mshr-h in https://github.com/microsoft/hummingbird/pull/642
• Update to pytorch 1.13 by @interesaaat in https://github.com/microsoft/hummingbird/pull/646
• Update actions/checkout and actions/setup-python by @mshr-h in https://github.com/microsoft/hummingbird/pull/647
• update codecov/codecov-action to v3 by @mshr-h in https://github.com/microsoft/hummingbird/pull/648

New Contributors

• @fd0r made their first contribution in https://github.com/microsoft/hummingbird/pull/625
• @RomanBredehoft made their first contribution in https://github.com/microsoft/hummingbird/pull/630
• @grafail made their first contribution in https://github.com/microsoft/hummingbird/pull/631
• @liangfu made their first contribution in https://github.com/microsoft/hummingbird/pull/637

Special Thanks

• Thank you to @mshr-h for the continued support!!

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.5...v0.4.6

What's Changed

• Update _decomposition_implementations.py by @interesaaat in https://github.com/microsoft/hummingbird/pull/578
• revise example in readme by @vumichien in https://github.com/microsoft/hummingbird/pull/579
• Fix problem with new onnx and protobuf by @interesaaat in https://github.com/microsoft/hummingbird/pull/582
• Bump TVM to v0.8 by @mshr-h in https://github.com/microsoft/hummingbird/pull/581
• Fix the things broken by SKL==1.1.1 by @ksaur in https://github.com/microsoft/hummingbird/pull/588
• Use onnxmltools>=1.6.0,<=1.11.0 instead of onnxmltools>=1.6.0 by @mshr-h in https://github.com/microsoft/hummingbird/pull/592
• Deprecating Python3.7; updating sklearn-onnx version by @ksaur in https://github.com/microsoft/hummingbird/pull/593
• deprecate torch1.7, push to macOS11 by @ksaur in https://github.com/microsoft/hummingbird/pull/594
• Fix doc gen by @ksaur in https://github.com/microsoft/hummingbird/pull/596
• Fix broken link by @mshr-h in https://github.com/microsoft/hummingbird/pull/599
• Fixing documentation generation bug by @ksaur in https://github.com/microsoft/hummingbird/pull/598
• Update Dockerfile by @mshr-h in https://github.com/microsoft/hummingbird/pull/601
• Use a Microsoft compliant image for docker by @interesaaat in https://github.com/microsoft/hummingbird/pull/602
• testing torch==1.12 by @ksaur in https://github.com/microsoft/hummingbird/pull/603
• Use n_features_in_ instead of n_features_ by @mshr-h in https://github.com/microsoft/hummingbird/pull/604
• Use python -m pip instead of the pip executable by @mshr-h in https://github.com/microsoft/hummingbird/pull/605
• check for Sklearn model NotFitted before conversion by @SangamSwadiK in https://github.com/microsoft/hummingbird/pull/607
• Upgrade prophet to v1.1 by @mshr-h in https://github.com/microsoft/hummingbird/pull/608
• Add Python 3.10 by @mshr-h in https://github.com/microsoft/hummingbird/pull/586
• Bump TVM to v0.9 by @mshr-h in https://github.com/microsoft/hummingbird/pull/609
• Pinning onnxconverter-common to avoid dep by @ksaur in https://github.com/microsoft/hummingbird/pull/614
• Use pre-installed LLVM for building TVM on macOS-11 by @mshr-h in https://github.com/microsoft/hummingbird/pull/615

New Contributors

• @vumichien made their first contribution in https://github.com/microsoft/hummingbird/pull/579
• @mshr-h made their first contribution in https://github.com/microsoft/hummingbird/pull/581
• @SangamSwadiK made their first contribution in https://github.com/microsoft/hummingbird/pull/607

Special thanks

Extra special thanks to @mshr-h for the work maintaining our version dependencies and pipeline!

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.4...v0.4.5

This minor release includes bug fixes for performance and external dependency updates.

What's Changed

• Verified compatibility with Torch 1.11 released March10 by @ksaur in https://github.com/microsoft/hummingbird/pull/572
• Fix xgboost tests for xgb > 1.6.0 by @interesaaat in https://github.com/microsoft/hummingbird/pull/576
• Fix for KernelPCA using GPU by @interesaaat in https://github.com/microsoft/hummingbird/pull/575

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.3...v0.4.4

New Contributors

• Thanks to @carolinemckee for the bug report on KPCA #574

This minor release includes bug fixes and external dependency updates.

What's Changed

• Fixed a problem with pandas with the latest xgb models in https://github.com/microsoft/hummingbird/pull/562
• Minor changes to tests related to verifying that new versions of torch, scikit-learn, and Python3.9 work in #554, #560
• allow tree_implementation="gemm" with onnx backend by @jfrery in https://github.com/microsoft/hummingbird/pull/566

New Contributors

• @jfrery made their first contribution in https://github.com/microsoft/hummingbird/pull/566
• @shubh0508 contributed a bug fix in https://github.com/microsoft/hummingbird/pull/568

Full Changelog: https://github.com/microsoft/hummingbird/compare/v0.4.2...v0.4.3

This minor release includes a new operator, some improvements, and some fixes due to external dependency updates.

New Operator:

• Added support for PLSRegressor (#549)

Improvements:

• Better delete (with location) for saved models (#558)
• Doc updates: Installation instructions for Fedora-based distros (#543) and readme update (#550)

External dependency management:

• Hummingbird now works with scikit-learn==1.0.x (#545) and the current version of scipy (scipy==1.7.x) (#552)
• Note that there is currently an open issue (#556) with Onnxruntime and torch==1.10.x on Linux/Mac that causes the tests to hang (for static length dimensions) and fail (for dynamic dimensions). This will be fixed in a subsequent release.

Credits:

Thanks to @akshjain83 and @bibhabasumohapatra for their contributions!

This patch release includes a new operator and some documentation fixes.

New Operator:

• Added support for lightgbm booster (#540)

Documentation Improvements:

• Added TorchServe support to documentation (#537)
• Updated sklearn_year_with_train.ipynb (#532)

Credits:

Thanks to @marsupialtail, @ananiask8, and @parulnith for their contributions!

This release includes integration with Prophet, bug/usability fixes, and versioning fixes.

We are excited to announce trend prediction support for Prophet! See our notebook for examples.

New features:

• Prophet integration (trend prediction) (#519)

Bug fixes/Usability fixes:

• Fix several numerical precision issues in tree-based models (#511)
• Better assertions and error messages (#514/#521)

Versioning fixes:

• Remove constraint on dependencies version (#523)

Special thanks:

• Extra special thanks to @xadupre for helping us resolve library dependency issues
• Thanks to @pnhathuy07 for helping us clean up our asserts
• Thanks to @scnakandala for the ongoing contributions

This patch release includes several improvements related to load/store of a model:

Improvements:

• Better error messages on load/save problems (#499)
• Auto-clean temp directory on load/save (#502)
• Using pickle instead of dill for model load/save to enable the use of Spark broadcast to share the model (#498)

Other Notes:

It seems the most recent release of libomp broke the pipeline for MacOS python3.6 and 3.7. We fixed this by pinning to an older version of libomp (#500). While not directly related to Hummingbird, this information may be useful to MacOS users with older versions of python wanting to use the lightgbm Hummingbird converters.

Credits:

Thanks to @dbanda for the feedback and testing on our load/store.

This release includes many new operators, version upgrades, and minor bug fixes.

New Features:

• ONNXML imputer (#459)
• SKL Bagging Classifier/Regressor (#490)
• SKL GridSearchCV and RandomizedGridSearchCV (#476)
• SKL KMeans (#472)
• SKL MeanShift (#473)
• SKL Stack Classifier and Regressor (#471)
• SKL RidgeCV and LinearSVR (#470)
• New example notebooks (#462) (#461)

Versioning changes:

• Bumped numpy from 1.19.4 to <=1.20.*
• Bumped torch to 1.8.1

Notable bug fixes:

• Fixed error when pandas is passed as input to predict but no test_input provided (#487)

Credits:

Thanks to our first-time contributor @rathijit for fixes to the README.

This release includes several bug fixes, version upgrades, and minor feature upgrades.

Versioning changes:

• Upgrading to Torch 1.8.0
• Deprecating Python 3.5

Notable bug fixes:

• Fix save to pytorch for onnx models with trees (#425)
• Fix bug with degenerate trees (#426)
• Fix bloated models when saving (#429)

Also included are fixes to test flakiness and error messages.

Feature upgrades:

• Add support for modified huber loss in SGD (#415)
• Simplify the discretizer logic (#448)

Credits:

Thanks to our first-time contributors @qin-xiong and @sleepy-owl

Patch release

Version bump:

• upgrading from torch 1.7.0 to torch 1.7.1

Also included:

• Refactoring of '_container' structure
• TVM loading
• TVM padding

Announcing: TVM Support

• This release adds support for TVM (#236), giving us our fastest speeds yet!

New Features

• Adding save/load features for models (#399)
• BatchContainer for batch by batch prediction use case (#377)
• Native support for string features (#396)
• Add batch_benchmark option to do benchmark on a single batch (#369)

New Operators

• Binarizer for ONNX-ML (#353)
• Feature Vectorizer for ONNX-ML (#385)
• Label Encoder for ONNX-ML and SKL (#374)

Credits

Thank you to @scnakandala and @masahi for their ongoing contributions!

Announcing: Integration with PySpark.ML (pyspark)

In this release, we added PySpark.ML support, which will open new doors for collaboration in the Spark space! (#310)

So far, we support Bucketizer, VectorAssembler, and LogisticRegressionModel. We look forward to adding more operators soon!

Announcing: Pandas Dataframes Support

This release also adds support for Pandas Dataframes both at conversion time and inference time (#300).

New Features

• We added benchmarks (#328, #330, #331) from our OSDI paper
• We also have a variety of features and improvements to the user experience:
  • Added the capability of setting number of threads to the model container (#319)
  • Removed the need for requirements on providing input schemas for ONNX models (#334)
  • Added support for ONNX models with multiple inputs (#339)
  • Added batching to output containers (#323)

New Operators

• scikit-learn KNeighbors Classifier (#296) and Regressor (#303)

Credits

Thank you to @scnakandala for SparkML and the additional scikit-learn operators!
Thank you to @vumichien for contributing the README diagrams.

This release adds a huge batch of new operators to scikit-learn, including pipeline support! It also includes bug fixes.

New Features

• Basic support for scikit-learn Pipeline's including FeatureUnion and ColumnTransformer (#251)

New Operators - scikit-learn

• Binarizer (#258)
• KBinsDiscretizer (#285)
• Matrix Decomposition Operators (#277)
  • FastICA
  • KernelPCA
  • PCA
  • TruncatedSVD
• MissingIndicator (#268)
• MLPClassifer (#260)
• MLPRegressor (#288)
• Other Classifiers: (#260)
  • BernoulliNB
  • GaussianNB
  • MultinomialNB
• SelectKBest chi2 support (#262)
• SelectPercentile (#263)
• SimpleImputer (#267)
• PolynomialFeatures (#269)

Other updates:

• We now support xgboost>=0.90 (#253)
• Added optimized_execution to torchscript backend (#276)
• We now allow older version of scikit-learn for compatibility reasons (#274)

Bug fixes:

• Logistic Regression with the lbfgs option (#261)
• Empty trees and multiclass (#265)
• Fix isolation forest for sklearn <= 0.21 (#290)

Credits

A big thank you to our contributors: @scnakandala, @zhanjiezhu
Extra big thanks to @scnakandala for all of the scikit-learn converters!

This release adds TorchScript as a backend, removes the problematic auto-installation of pytorch, improves syntax for the ONNX converter, adds notebook enhancements, and adds 3rd party library version upgrades. Hummingbird now provides the same Sklearn API across backends for executing inference.

Announcing: Integration with TorchScript

This release adds TorchScript as a backend (#225).

Users can convert models with:

hummingbird.ml.convert(model, "torchscript", X)

Installation changes:

After several reports from users across multiple platforms (in terms of both OS and underlying hardware), we changed the Hummingbird installer to require users to first install pytorch before installing Hummingbird (#246). This allows users to select the right pytorch version for a specific platform and should simplify the installation process and remove issues caused by having the wrong pytorch version installed.

ONNX API

For ONNX, we changed the API to have a more seamless experience, allowing users to interact with ONNX models in a consistent way with other models in Hummingbird (#231).

Instead of the user having to instanciate the ONNX runtime session:

-        session = ort.InferenceSession(onnx_model.SerializeToString())
-        onnx_pred = session.run(output_names, inputs)

The user can now just call predict, predict_proba, transform, etc. as with other Hummingbird conversions.

+        onnx_pred = onnx_model.predict(X)

New Operators

• OneHotEncoder - integers (#197)
• Support for Tweedie distribution in LGBM (#242)

Miscellaneous

• The target opset for ONNX is now 11 (#214)
• The target pytorch version is now 1.6.0, except for with Python 3.5 it remains at 1.5.1 for compatibility reasons (#213)
• Docs are now auto-generated (#223)

Credits

Thanks to @KranthiGV for the updated LGBM ONNX notebook example