catalyst

[22.04] - 2022-04-29

Added

• catalyst-tune for Config API added #1411
• tests for python 3.9 and 3.10 #1414

Fixed

• catalyst compatibility with python 3.10 #1409

[22.02.1] - 2022-02-27

"Few fixes and Config API v22 MVP".

Added

• catalyst-run for Config API support added #1406

Fixed

• Logger API naming #1405

[22.02] - 2022-02-13

Tl;dr

• Catalyst architecture simplification.
• #1395, #1396, #1397, #1398, #1399, #1400, #1401, #1402, #1403.

Added

• Additional tests for different hardware accelerators setups. Please check out the tests/pipelines folder for more information.
• BackwardCallback and BackwardCallbackOrder as an abstraction on top of loss.backward. Now you could easily log model gradients or transform them before OptimizerCallback.
• CheckpointCallbackOrder for ICheckpointCallback.

Changed

• Minimal python version moved to 3.7, minimal PyTorch version moved to 1.4.0.
• Engines were rewritten on top of Accelerate. First, we found these two abstractions very close to each other. Second, Accelerate provides additional user-friendly API and more stable API for "Nvidia APEX" and "Facebook Fairscale" - it does not support them.
• SelfSupervisedRunner moved to the examples folder from the Catalyst API. The only Runners API, that will be supported in the future: IRunner, Runner, ISupervisedRunner, SupervisedRunner due to their consistency. If you are interested in any other Runner API - feel free to write your own CustomRunner and use SelfSupervisedRunner as an example.
• Runner.{global/stage}_{batch/loader/epoch}_metrics renamed to Runner.{batch/loader/epoch}_metrics
• CheckpointCallback rewritten from scratch.
• Catalyst registry moved to full-imports-paths only.
• Logger API changed to receive IRunner for all log_* methods.
• Metric API: topk_args renamed to topk.
• Contrib API: init imports from catalyst.contrib - removed, use from catalyst.contrib.{smth} import {smth}. Could be change to full-imports-only in future versions for stability.
• All quickstarts, minimal examples, notebooks, and pipelines moved to the new version.
• Codestyle moved to 89 right margin. Honestly speaking, it's much easier to maintain Catalyst with 89 right margin on MBP'16.

Removed

• ITrial removed.
• Stages support removed. While we embrace stages in deep learning experiments, current hardware accelerators are not prepared well for such setups. Additionally, ~95% of dl pipelines are single-stage. Multi-stage runner support is under review. For multi-stage support, please define a CustomRunner with rewritten API.
• Config/Hydra API support removed. Config API is under review. For now, you could write your own Config API with hydra-slayer if needed.
• catalyst-dl scripts removed. Without Config API we don't need them anymore.
• Nvidia Apex, Fairscale, Albumentations, Nifti, Hydra requiremets removed.
• OnnxCallback, PruningCallback, QuantizationCallback, TracingCallback removed from callbacks API. These callbacks are under review now.

If you have any questions on the Catalyst 22 edition updates, please join Catalyst slack for discussion.

[22.02rc0] - 2022-02-07

Tl;dr

Beta version of Catalyst 22 edition.

• core architecture moved to Animus-like (stages were removed)
• engines moved to Accelerate
• config/hydra APIs deprecated in favor of hydra-slayer-custom config runners
• dl-based scripts removed from the API
• self-supervised runner moved to examples - it's better to have custom still
• contrib and utils - truncated
• requirements - simplified
• codestyle moved to -l 89 (better view on 16'' screen ;) )

[21.12] - 2021-12-28

Tl;dr

Distributed engines update (multi-node support) and many other improvements.

Added

• MNIST dataset for SSL banchmark (#1368)
• MoveiLens 20M dataset #1336
• logger property for logging customization (#1372)
• MacridVAE example (#1363)
• SSL benchmark results (#1374)
• Neptune example (#1377)
• multi-node support for engines (#1364)

Changed

• RL examples update to last version (#1370)
• DDPLoaderWrapper updated to new version (#1385)
• num_classes for classification metrics became optional (#1379)
• colab ci/cd update to new verion

Removed

Fixed

• requests requirements for catalyst[cv] added (#1371)
• loader step counter (#1374)
• detection example data preprocessing (#1369)
• gradient clipping with fp16 runs (#1378)
• config API fix for DDP runs (#1383)
• checkpoint creation for fp16 engines (#1382)

Contributors ❤️

@bagxi @ditwoo @MrNightSky @Nimrais @y-ksenia @sergunya17 @Thiefwerty @zkid18

[21.11] - 2021-11-30

Tl;dr

Framework architecture simplification and speedup + SSL & RecSys extensions.

Added

• MultiVAE RecSys example (#1340)`
• Returned resume support - resolved #1193 (#1349)
• Smoothing dice loss to contrib (#1344)
• profile flag for runner.train (#1348)
• MultiDAE RecSys example (#1356)
• SETTINGS.log_batch_metrics, SETTINGS.log_epoch_metrics, SETTINGS.compute_per_class_metrics for framework-wise Metric & Logger APIs specification (#1357)
• log_batch_metrics and log_epoch_metrics options for all available Loggers (#1357)
• compute_per_class_metrics option for all available multiclass/label metrics (#1357)
• pytorch benchmark script and simplified MNIST (#1360)

Changed

• A few framework simplifications were made (#1346):
  • catalyst-contrib scripts reduced to collect-env and project-embeddings only
  • catalyst-dl scripts recuded to run and tune only
  • transforms. prefix deprecated for Catalyst-based transforms
  • catalyst.tools moved to catalyst.extras
  • task-dependent extensions from catalyst.data moved to catalyst.contrib.data
  • catalyst.data.transforms moved to catalyst.contrib.data.transforms
  • Normalize, ToTensor transforms renamed to NormalizeImage, ImageToTensor
  • metric learning extensions moved to catalyst.contrib.data
  • catalyst.contrib moved to code-as-a-documentation development
  • catalyst[cv] and catalyst[ml] extensions moved to flatten architecture design; examples: catalyst.contrib.data.dataset_cv, catalyst.contrib.data.dataset_ml
  • catalyst.contrib moved to flatten architecture design; exampels: catalyst.contrib.data, catalyst.contrib.datasets, catalyst.contrib.layers, catalyst.contrib.models, catalyst.contrib.optimizers, catalyst.contrib.schedulers
  • internal functionality moved to ***._misc modules
  • catalyst.utils.mixup moved to catalyst.utils.torch
  • catalyst.utils.numpy moved to catalyst.contrib.utils.numpy
• default logging logic moved from "batch & epoch" to "epoch"-only to save computation time during logging; to respecify, please use:
  • SETTINGS.log_batch_metrics=True/False or os.environ["CATALYST_LOG_BATCH_METRICS"]
  • SETTINGS.log_epoch_metrics=True/False or os.environ["CATALYST_LOG_EPOCH_METRICS"]
• default metrics computation moved from "per-class & aggregations" to "aggregations"-only to save computation time during logging; to respecify, please use:
  • SETTINGS.compute_per_class_metrics=True/False or os.environ["CATALYST_COMPUTE_PER_CLASS_METRICS"]
• no transformations required for MNIST contrib dataset (#1360

Removed

• A few framework simplifications were made (#1346):
  • catalyst.contrib.pandas
  • catalyst.contrib.parallel
  • catalyst.contrib.models.cv
  • a few catalyst.utils.misc functions
  • catalyst.extras removed from the public documentation

Fixed

• documentation search error (21.10 only) (#1346)
• docs examples (#1362)
• Self-Supervised benchmark: (#1365), (#1361)

Contributors ❤️

@asteyo @Dokholyan @Nimrais @y-ksenia @sergunya17

[21.10] - 2021-10-30

Tl;dr

Readmes and tutorials with a few ddp fixes.

Added

• RSquareLoss (#1313)
• Self-Supervised example updates: (#1305), (#1322), (#1325), (#1335)
• Albert training example (#1326)
• YOLO-X (new) detection example and refactoring (#1324)
• TopKMetric asbtraction (#1330)

Changed

• simlified readme (#1312)
• improved DDP tutorial (#1327)
• CMCMetric renamed from <prefix>cmc<suffix><k> to <prefix>cmc<k><suffix> (#1330)

Removed

Fixed

• Zero seed error (#1329)
• updated codestyle issues (#1331)
• TopK metrics: (#1330), (#1334), (#1339)
• --expdir param for catalyst-dl run (#1338)
• ControlFlowCallback for distributed setup (#1341)

[21.09] - 2021-09-30

Added

• CometLogger support (#1283)
• CometLogger examples (#1287)
• XLA docs (#1288)
• Contarstive loss functions: NTXentLoss (#1278), SupervisedContrastiveLoss (#1293)
• Self supervised learning: ISelfSupervisedRunner, SelfSupervisedConfigRunner, SelfSupervisedRunner, SelfSupervisedDatasetWrapper (#1278)
• SimCLR example (#1278)
• Superivised Contrastive example (#1293)
• extra warnings for runner-callbacks interaction (#1295)
• CategoricalRegressionLoss and QuantileRegressionLoss to the contrib (#1295)
• R2 score metric (#1274)

Changed

• Improved WandbLogger to support artifacts and fix logging steps (#1309)
• full Runner cleanup, with callbacks and loaders destruction, moved to PipelineParallelFairScaleEngine only (#1295)
• HuberLoss renamed to HuberLossV0 for the PyTorch compatibility (#1295)
• codestyle update (#1298)
• BalanceBatchSampler - deprecated (#1303)

Removed

Fixed

• CI/CD (#1292), (#1299), (#1304), (#1306)
• Optuna configs (#1296), (#1296)

Contributors ❤️

@asteyo @AyushExel @bagxi @DN6 @gr33n-made @Nimrais @Podidiving @y-ksenia

Hi guys, nice project!

This is the test case release to check out our updated infrastructure.

[21.08] - 2021-08-31

Added

• RecSys loss functions: AdaptiveHingeLoss, BPRLoss, HingeLoss, LogisticLoss, RocStarLoss, WARPLoss (#1269, #1282)
• object detection examples (#1271)
• SklearnModelCallback (#1261)
• Barlow Twins example (#1261)
• TPU/XLA support (#1275)
  • with updated example
• native sync_bn support for all available engines (#1275)
  • Torch, AMP, Apex, FairScale

Changed

• Registry moved to hydra-slayer (#1264))
• (#1275)
  • batch metrics sync removed from ddp-runs to speedup training process
  • AccumulationMetric renamed to AccumulativeMetric
    • moved from catalyst.metrics._metric to catalyst.metrics._accumulative
    • accululative_fields renamed to keys

Removed

Fixed

• PeriodicLoaderCallback docsting (#1279)
• matplotlib issue (#1272)
• sample counter for the loader (#1285)

Contributors ❤️

@bagxi @Casyfill @ditwoo @Nimrais @penguinflys @sergunya17 @zkid18

[21.07] - 2021-07-29

Added

• added pre-commit hook to run codestyle checker on commit (#1257)
• on publish github action for docker and docs added (#1260)
• MixupCallback and utils.mixup_batch (#1241)
• Barlow twins loss (#1259)
• BatchBalanceClassSampler (#1262)

Changed

Removed

Fixed

• make expdir in catalyst-dl run optional (#1249)
• Bump neptune-client from 0.9.5 to 0.9.8 in requirements-neptune.txt (#1251)
• automatic merge for master (with Mergify) fixed (#1250)
• Evaluate loader custom model bug was fixed (#1254)
• BatchPrefetchLoaderWrapper issue with batch-based PyTorch samplers (#1262)
• Adapted MlflowLogger for new config hierarchy (#1263)

Contributors ❤️

@AlekseySh @bagxi @Casyfill @Dokholyan @leoromanovich @Nimrais @y-ksenia

[21.06] - 2021-06-29

Added

• (#1230)
  • FairScale support
  • DeepSpeed support
  • utils.ddp_sync_run function for synchronous ddp run
  • CIFAR10 and CIFAR100 datasets from torchvision (no cv-based requirements)
  • Catalyst Engines demo
• dataset_from_params support in config API (#1231)
• transform from params support for config API added (#1236)
• samplers from params support for config API added (#1240)
• recursive registry.get_from_params added (#1241)
• albumentations integration (#1238)
• Profiler callback (#1226)

Changed

• (#1230)
  • loaders creation now wrapper with utils.ddp_sync_run for utils.ddp_sync_run data preparation
  • runner support stage cleanup: loaders and callbacks will be deleted on the stage end
  • Apex-based engines now support both APEXEngine and ApexEngine registry names

Fixed

• multiprocessing in minimal tests hotfix (#1232)
• Tracing callback hotfix (#1234)
• Engine hotfix for predict_loader (#1235)
• (#1230)
  • Hydra hotfix due to 1.1.0 version changes
• HuberLoss name conflict for pytorch 1.9 hotfix (#1239)

Contributors ❤️

@bagxi @y-ksenia @ditwoo @BorNick @Inkln

[21.05] - 2021-05-31

Added

• Reinforcement learning tutorials (#1205)
• customization demo (#1207)
• FAQ docs: multiple input and output keys, engine tutorial (#1202)
• minimal Config API example (#1215)
• Distributed RL example (Catalyst.RL 2.0 concepts) (#1224)
• SklearnCallback as integration of sklearn metrics (#1198)

Changed

• tests moved to tests folder (#1208)
• pipeline tests moved to tests/pipelines (#1215)
• updated NeptuneLogger docstrings (#1223)

Removed

Fixed

• customizing what happens in train() notebook (#1203)
• transforms imports under catalyst.data (#1211)
• change layerwise to layerwise_params (#1210)
• add torch metrics support (#1195)
• add Config API support for BatchTransformCallback (#1209)

BONUS: Catalyst workshop videos!

[21.04.2] - 2021-04-30

Added

• Weights and Biases Logger (WandbLogger) (#1176)
• Neptune Logger (NeptuneLogger) (#1196)
• log_artifact method for logging arbitrary files like audio, video, or model weights to ILogger and IRunner (#1196)

• a small hotfix for catalyst.contrib module

[21.04] - 2021-04-17

Added

• Nifti Reader (NiftiReader) (#1151)
• CMC score and callback for ReID task (ReidCMCMetric and ReidCMCScoreCallback) (#1170)
• Market1501 metric learning datasets (Market1501MLDataset and Market1501QGDataset) (#1170)
• extra kwargs support for Engines (#1156)
• engines exception for unknown model type (#1174)
• a few docs to the supported loggers (#1174)

Changed

• TensorboardLogger switched from global_batch_step counter to global_sample_step one (#1174)
• TensorboardLogger logs loader metric on_loader_end rather than on_epoch_end (#1174)
• prefix renamed to metric_key for MetricAggregationCallback (#1174)
• micro, macro and weighted aggregations renamed to _micro, _macro and _weighted (#1174)
• BatchTransformCallback updated (#1153)

Removed

• auto torch.sigmoid usage for metrics.AUCMetric and metrics.auc (#1174)

Fixed

• hitrate calculation issue (#1155)
• ILoader wrapper usage issue with Runner (#1174)
• counters for ddp case (#1174)

Fixed

• minimal requirements issue (#1147)

[21.03.1] - 2021-03-28

Added

• Additive Margin SoftMax(AMSoftmax)(#1125)
• Generalized Mean Pooling(GeM)(#1084)
• Key-value support for CriterionCallback (#1130)
• Engine configuration through cmd (#1134)
• Extra utils for thresholds (#1134)
• Added gradient clipping function to optimizer callback (1124)
• FactorizedLinear to contrib (1142)
• Extra init params for ConsoleLogger (1142)
• Tracing, Quantization, Onnx, Pruninng Callbacks (1127)
• _key_value for schedulers in case of multiple optimizers fixed (#1146)

Changed

• CriterionCallback now inherits from BatchMetricCallback #1130)
  • united metrics computation logic

Removed

• Config API deprecated parsings logic (1142) (1138)

Fixed

• Data-Model device sync and Engine logic during runner.predict_loader (#1134)
• BatchLimitLoaderWrapper logic for loaders with shuffle flag (#1136)
• config description in the examples (1142)
• Config API deprecated parsings logic (1142) (1138)
• RecSys metrics Top_k calculations ([#1140] (https://github.com/catalyst-team/catalyst/pull/1140))

The v20 is dead, long live the v21!

[21.03] - 2021-03-13 (#1095)

Added

• Engine abstraction to support various hardware backends and accelerators: CPU, GPU, multi GPU, distributed GPU, TPU, Apex, and AMP half-precision training.
• Logger abstraction to support various monitoring tools: console, tensorboard, MLflow, etc.
• Trial abstraction to support various hyperoptimization tools: Optuna, Ray, etc.
• Metric abstraction to support various of machine learning metrics: classification, segmentation, RecSys and NLP.
• Full support for Hydra API.
• Full DDP support for Python API.
• MLflow support for metrics logging.
• United API for model post-processing: tracing, quantization, pruning, onnx-exporting.
• United API for metrics: classification, segmentation, RecSys, and NLP with full DDP and micro/macro/weighted/etc aggregations support.

Changed

• Experiment abstraction merged into Runner one.
• Runner, SupervisedRunner, ConfigRunner, HydraRunner architectures and dependencies redesigned.
• Internal settings and registry mechanisms refactored to be simpler, user-friendly and more extendable.
• Bunch of Config API test removed with Python API and pytest.
• Codestyle now supports up to 99 symbols per line :)
• All callbacks/runners moved for contrib to the library core if was possible.
• Runner abstraction simplified to store only current state of the experiment run: all validation logic was moved to the callbacks (by this way, you could easily select best model on various metrics simultaneously).
• Runner.input and Runner.output merged into united Runner.batch storage for simplicity.
• All metric moved from catalyst.utils.metrics to catalyst.metrics.
• All metrics now works on scores/metric-defined-input rather that logits (!).
• Logging logic moved from Callbacks to appropriate Loggers.
• KorniaCallbacks refactored to BatchTransformCallback.

Removed

• Lots of unnecessary contrib extensions.
• Transforms configuration support through Config API (could be returned in next releases).
• Integrated Python cmd command for model pruning, swa, etc (should be returned in next releases).
• CallbackOrder.Validation and CallbackOrder.Logging
• All 2020 year backward compatibility fixes and legacy support.

Fixed

• Docs rendering simplified.
• LrFinderCallback.

Release docs,
Python API minimal examples,
Config/Hydra API example.