I need to calculate the accuracy but at a false positive rate of 1%. I am not sure if it is the normal accuracy that we can calculate with sklearn or I need a customized formula?
$\begingroup$ $\endgroup$
Add a comment |
1 Answer
$\begingroup$ $\endgroup$
Calculate this by finding the threshold at which the false positive rate is 1%. Your model outputs a probability, not a category. You get a category by seeing on which side of some threshold, probably 0.5, the probability lies. Test some other thresholds. You can use the ROC curve for this.
Once you have that threshold, calculate the accuracy.
However, be cautious about accuracy (or any threshold-based scoring rule, such as F1 score) as a scoring rule.
(Frank Harrell’s blog that gets linked is a good read, too.)
