3
$\begingroup$

Say I fit a logistic classifier on a supervised dataset with binary labels. If I select a threshold of decision of 0.5, which assumption am I implicitly making? Is there any situation where 0.5 makes sense?

This page suggests that we should always tune the decision threshold to optimize some target metric of interest. So intuitively, my guess is that 0.5 only makes sense if the metric of interest is the accuracy AND the class probabilities are equal AND the misclassification costs are equal. In any other case, a threshold of 0.5 should not be used. Is this correct?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
3
$\begingroup$

I would say that a threshold of 0.5 should not be blindly used. It can be optimal under certain circumstances, just as any other threshold in (0,1) can be optimal under other circumstances. There is no reason to tune a threshold but forbid the one value 0.5.

(And of course, I maintain that it is very often the case that we don't have two possible courses of action, but more, even if there are two underlying classes. A clinical test that yields a low probability of me having a certain disease can lead to the action to do nothing. A medium probability could lead to doing more tests. A high probability could lead to treatment starting immediately. So it makes really no sense to train a single threshold.)

Improve this answer
$\endgroup$
2
  • $\begingroup$ Suppose I pick a threshold other than 0.5, but I want to compute BOTH the predicted class, and the probability that the observation belongs to that class. I don't understand how I can compute the latter (probability) in a principled way once the threshold is shifted from 0.5. $\endgroup$ Commented May 6, 2024 at 12:40
  • 1
    $\begingroup$ @MikeS: The predicted probability of an instance to belong to the target class is always just the output of the logistic regression (or of any other probabilistic classifier). The threshold pertains to how we treat" a given instance, given its probabilistic classification. This thread may be helpful: stats.stackexchange.com/q/312119/1352. Are you asking about the probability that an instance truly belongs to the target class *conditional on us treating it as such? $\endgroup$ Commented May 6, 2024 at 12:43

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.