A lightweight library to compute Diarization Error Rate (DER).
APACHE-2.0 License
This is a lightweight library to compute Diarization Error Rate (DER).
Features NOT supported:
collar
value.For more sophisticated metrics with these supports, please use pyannote-metrics instead.
To learn more about speaker diarization, here is a curated list of resources: awesome-diarization.
Diarization Error Rate (DER) is the most commonly used metrics for speaker diarization.
Its strict form is:
False Alarm + Miss + Overlap + Confusion
DER = ------------------------------------------
Reference Length
The definition of each term:
Reference Length:
The total length of the reference (ground truth).False Alarm
: Length of segments which are considered as speech inMiss
: Length of segments which are considered as speech inOverlap
: Length of segments which are considered as overlapped speechConfusion
: Length of segments which are assigned to different speakersThe unit of each term is seconds.
Note that DER can theoretically be larger than 1.0.
References:
Install the package by:
pip3 install simpleder
or
python3 -m pip install simpleder
Here is a minimal example:
import simpleder
# reference (ground truth)
ref = [("A", 0.0, 1.0),
("B", 1.0, 1.5),
("A", 1.6, 2.1)]
# hypothesis (diarization result from your algorithm)
hyp = [("1", 0.0, 0.8),
("2", 0.8, 1.4),
("3", 1.5, 1.8),
("1", 1.8, 2.0)]
error = simpleder.DER(ref, hyp)
print("DER={:.3f}".format(error))
This should output:
DER=0.350