Fixes

• 🐛 skfb.estimators.RateFallbackClassifierCV accepts only one fallback rate (#11).
• 🐞 skfb.metrics.PAConfusionMatrixDisplay accepts rejector pipelines (#18).

Features

Stable

• Predict-reject recall score skfb.metrics.predict_reject_recall_score (#14).
• New estimators accept fitted estimators and don't require refitting for inference.
• Support for scikit-learn>=1.0,<=1.2.
• Multi-threshold fallback classification: skfb.estimators.multi_threshold_predict_or_fallback and skfb.estimators.MultiThresholdFallbackClassifier.
• Fallback classification based on anomaly detection: skfb.estimators.AnomalyFallbackClassifier (#13 and more).
• Fallback mode "ignore": don't return or store fallbacks (#16 and more).

Experimental

• skfb.estimators.RateFallbackClassifierCV accepts only one fallback rate (#11).
• Error-fallback loss:

  >>> from skfb.experimental import enable_error_rejection_loss
  >>> from skfb.metrics import error_rejection_loss
  

• Tuned multi-threshold fallback classifier w/ cross-validation:

  >>> from skfb.experimental import enable_multi_threshold_fallback_classifier_cv
  >>> from skfb.estimators import MultiThresholdFallbackClassifierCV
  

• Utility to summarize confidence scores class-wise.

Bug fixes

• Incorrect masking of fallbacks of ambiguity-threshold-based rules (#3)
• Errors when fitting skfb.estimators.ThresholdFallbackClassifierCV(fallback_mode="return") (#4)

Improvements

• Passing scikit-learn metrics as scorers in skfb.estimators.ThresholdFallbackClassifierCV(fallback_mode="return") (#4)
• Inference w/o training the fitted base estimator of skfb.estimators.ThresholdFallbackClassifier (#6)

This is the first release of scikit-fallback and it implements rudimentary tools for supporting and evaluating rejections in classification problems:

• sfkb.estimators.ThresholdFallbackClassifier(CV) and RateFallbackClassifier for (meta-)classification w/ a reject option.
• skfb.metrics for Predict-Fallback metrics, confusion matrices, and curves.
• skfb.core.array for NDArray-compatible FBNDArray for storing predictions and fallback masks.