Fast computation of Krippendorff's alpha agreement measure.
Based on Thomas Grill implementation. Works on Python 3.5+.
Given a reliability data matrix, run:
import krippendorff
reliability_data = ...
krippendorff.alpha(reliability_data)
See sample.py
and alpha
's docstring for more.
pip install krippendorff
The implementation is fast as it doesn't do a nested loop for the coders. However, V should be small, since a matrix of VxV it's used.