I'm looking for a Python library that can compute the confusion matrix for multi-label classification.
FYI:
- scikit-learn doesn't support multi-label for confusion matrix)
- What is the difference between Multiclass and Multilabel Problem
$\begingroup$
$\endgroup$
6 
- $\begingroup$ What did you end up doing? $\endgroup$Morteza Shahriari Nia– Morteza Shahriari Nia2016-10-13 22:27:47 +00:00Commented Oct 13, 2016 at 22:27
- $\begingroup$ @MortezaShahriariNia I stayed monolabeled. $\endgroup$Franck Dernoncourt– Franck Dernoncourt2016-10-13 22:33:11 +00:00Commented Oct 13, 2016 at 22:33
- $\begingroup$ what would a theoretical multilabel confusion matrix look like? I do not think it applies, does it? $\endgroup$user798719– user7987192017-05-25 04:05:59 +00:00Commented May 25, 2017 at 4:05
- 3$\begingroup$ very funny that all 3 answers to this question are of such remarkably low quality. $\endgroup$Monica Heddneck– Monica Heddneck2017-08-11 06:41:14 +00:00Commented Aug 11, 2017 at 6:41
- $\begingroup$ Not aware of any packages, but you could perhaps consider all possible multi-label combination as a separate class and use some of the already-available packages for multi-class. Then, from that confusion, build your multi-label matrix $\endgroup$Valentin Calomme– Valentin Calomme2018-01-30 22:20:03 +00:00Commented Jan 30, 2018 at 22:20
| Show 1 more comment
7 Answers
$\begingroup$ $\endgroup$
1 Also take a look at scikit-multilearn. It is a very good library that extends sklearn for multi-label learning. However, I'm not sure how the confusion matrix works for multi-label problems...
This guy claims he has solved it.
- $\begingroup$ skmultilearn has no confusion matrix utils. $\endgroup$Ricardo Decal– Ricardo Decal2021-06-10 19:34:12 +00:00Commented Jun 10, 2021 at 19:34
$\begingroup$ $\endgroup$
Although this question is old, I am writing this answer for new audience.
scikit-learn now supports confusion matrix for multi-label classification.
https://scikit-learn.org/stable/modules/generated/sklearn.metrics.multilabel_confusion_matrix.html
$\begingroup$ $\endgroup$
1 There are many different parameters which can evaluate the performance of your method by comparing the real and predicted labels. I suggest PyCM module which can give a vast variety these parameters which are suitable for multi-class classification.
- $\begingroup$ Does not support multi-label. github.com/sepandhaghighi/pycm/issues/211 $\endgroup$Ricardo Decal– Ricardo Decal2021-06-10 19:32:45 +00:00Commented Jun 10, 2021 at 19:32
$\begingroup$ $\endgroup$
2 
- 3$\begingroup$ how about multi-label multi-class? Is it supported? $\endgroup$Franck Dernoncourt– Franck Dernoncourt2017-02-15 17:49:16 +00:00Commented Feb 15, 2017 at 17:49
- $\begingroup$ No, this library does not support multi-label $\endgroup$Ricardo Decal– Ricardo Decal2021-06-10 19:30:20 +00:00Commented Jun 10, 2021 at 19:30
$\begingroup$ $\endgroup$
1 Sklearn has a method for it using which you can compute confusion matrix for multi class.
from sklearn import cross_validation confusion_matrix(original, Predicted) - 2$\begingroup$ how about multi-label? github.com/scikit-learn/scikit-learn/issues/3452 is still open $\endgroup$Franck Dernoncourt– Franck Dernoncourt2016-10-17 14:40:05 +00:00Commented Oct 17, 2016 at 14:40
$\begingroup$ $\endgroup$
1 Scikit-learn does support multi-label confusion matrix. See the links below for documentation and user guide:
http://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html
http://scikit-learn.org/stable/modules/model_evaluation.html#confusion-matrix
- 1$\begingroup$ Thank you, that's why I am looking for an alternative option :-) $\endgroup$Franck Dernoncourt– Franck Dernoncourt2017-02-15 17:48:58 +00:00Commented Feb 15, 2017 at 17:48
$\begingroup$ $\endgroup$
Look at sed_eval library. It is developed for evaluating event detection in audio which is a multi-label problem (as in each audio, multiple events exist). They have many evaluation options, which might fit to your needs. You can get the true-positive rate, ... and from there computing the confusion matrix is not that hard.
