mmeval

Features

• Add jax support to accuracy by @fengsxy in https://github.com/open-mmlab/mmeval/pull/61
• Add jax support to MeanIoU by @xuan07472 in https://github.com/open-mmlab/mmeval/pull/67
• Add bleu metric and its test file by @go-with-me000 in https://github.com/open-mmlab/mmeval/pull/66
• Add SAD by @xuan07472 in https://github.com/open-mmlab/mmeval/pull/76
• Add GradientError by @xuan07472 in https://github.com/open-mmlab/mmeval/pull/78
• Add MattingMSE by @xuan07472 in https://github.com/open-mmlab/mmeval/pull/71
• Add ConnectivityError by @xuan07472 in https://github.com/open-mmlab/mmeval/pull/79
• Add DOTAMeanAP metric by @YanxingLiu in https://github.com/open-mmlab/mmeval/pull/65
• Add ROUGE to mmeval by @go-with-me000 in https://github.com/open-mmlab/mmeval/pull/72
• Add KeypointEndPointError by @LareinaM in https://github.com/open-mmlab/mmeval/pull/92
• Add perplexity metric by @go-with-me000 in https://github.com/open-mmlab/mmeval/pull/87
• Add KeypointAUC metric by @LareinaM in https://github.com/open-mmlab/mmeval/pull/91
• Add KeypointNME by @LareinaM in https://github.com/open-mmlab/mmeval/pull/93
• Support NIQE by @Z-Fran in https://github.com/open-mmlab/mmeval/pull/80
• Add WordAccuracy for OCR Task by @Harold-lkk in https://github.com/open-mmlab/mmeval/pull/94
• Add CharRecallPrecision for OCR Task by @Harold-lkk in https://github.com/open-mmlab/mmeval/pull/96
• Add logger argument in BaseMetric by @C1rN09 in https://github.com/open-mmlab/mmeval/pull/102

Enhancements

• Update the test case style of PCKAccuracy by @ice-tong in https://github.com/open-mmlab/mmeval/pull/68
• Rename metrics by @ice-tong in https://github.com/open-mmlab/mmeval/pull/83
• Make scipy as a runtime dependency by @C1rN09 in https://github.com/open-mmlab/mmeval/pull/109
• Refactor single and multi label metrics by @yingfhu in https://github.com/open-mmlab/mmeval/pull/81
• Refactor detection dataset metainfo to lowercase and update detection metric logics by @BIGWangYuDong in https://github.com/open-mmlab/mmeval/pull/98

Bug fixes

• Fix CI failed caused by OneFlow installation and new version of NumPy by @ice-tong in https://github.com/open-mmlab/mmeval/pull/73
• Fix import error in python3.6 by @ytzhao in https://github.com/open-mmlab/mmeval/pull/77
• Upgrade the version of isort to fix lint error by @zhouzaida in https://github.com/open-mmlab/mmeval/pull/85
• Use official isort by @zhouzaida in https://github.com/open-mmlab/mmeval/pull/86
• Fix psnr,snr,ssim,mae and mse fail to compute on videos by @Z-Fran in https://github.com/open-mmlab/mmeval/pull/89
• CircleCI fail due to plum-dispatch v2.0.0 by @C1rN09 in https://github.com/open-mmlab/mmeval/pull/108
• Fix format by @zhouzaida in https://github.com/open-mmlab/mmeval/pull/111
• Add opencv to docs dependency & setup.py to resolve API docs error by @C1rN09 in https://github.com/open-mmlab/mmeval/pull/113

Others

• Add CODEOWNER by @C1rN09 in https://github.com/open-mmlab/mmeval/pull/101

New Contributors

• @fengsxy made their first contribution in https://github.com/open-mmlab/mmeval/pull/61
• @xuan07472 made their first contribution in https://github.com/open-mmlab/mmeval/pull/67
• @go-with-me000 made their first contribution in https://github.com/open-mmlab/mmeval/pull/66
• @ytzhao made their first contribution in https://github.com/open-mmlab/mmeval/pull/77
• @YanxingLiu made their first contribution in https://github.com/open-mmlab/mmeval/pull/65
• @C1rN09 made their first contribution in https://github.com/open-mmlab/mmeval/pull/101
• @LareinaM made their first contribution in https://github.com/open-mmlab/mmeval/pull/92
• @Harold-lkk made their first contribution in https://github.com/open-mmlab/mmeval/pull/94

Full Changelog: https://github.com/open-mmlab/mmeval/compare/v0.2.0...v0.2.1

Since MMEval v0.2.0, we support evaluation with OneFlow!

Features

• Add oneflow distributed backend (#59)
• Support seven metrics for OneFlow backend (#58)

Documentations

• Add mmeval into PYTHONPATH for API doc generation(#45)
• Update qq group link (#56)
• Add OneFlow support (#62)

Contributors

A total of 4 developers contributed to this release.
@vansin @ofhwei @zhouzaida @ice-tong

New Contributors

• @vansin made their first contribution in https://github.com/open-mmlab/mmeval/pull/56
• @ofhwei made their first contribution in https://github.com/open-mmlab/mmeval/pull/59

Full Changelog: https://github.com/open-mmlab/mmeval/compare/v0.1.0...v0.2.0

We are excited to introduce our new open-source library: MMEval, a unified and open cross-ML framework evaluation library.

Install MMEval via pip:

pip install mmeval

MMEval is a machine learning evaluation library that supports efficient and accurate distributed evaluation on a variety of machine learning frameworks. Major features:

• Comprehensive metrics for various computer vision tasks (NLP will be covered soon!)
• Efficient and accurate distributed evaluation, backed by multiple distributed communication backends
• Support multiple machine learning frameworks via dynamic input dispatching mechanism