Questions tagged [threshold]
Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*
263 questions
0 votes
0 answers
80 views
Is using the TEST set to calculate the optimal threshold for binary classification and then calculating the accuracy on the same test set wrong
I have a dataset that has been split into 2 parts, train and test set. After training a model with the training set to classify between class 0 and 1, I used the sklearn roc_curve to calculate the ...
1 vote
0 answers
58 views
Is there a general formula for M-up N-down psychophysical staircase convergence points?
Setting aside more modern adaptive methods for the moment, I'm trying to find a formula that takes M (the number of successive misses required to increase stimulus salience) and N (the number of ...
1 vote
1 answer
110 views
Optimal threshold for a predictive covariate on treatment effect
here a physician with a moderate understanding of statistics and am seeking guidance on analyzing a continuous covariate (biomarker) that I suspect is predictive of treatment effect. My analysis ...
1 vote
0 answers
105 views
Best way to solve a multi-class thresholding problem?
I have a multi-class classification problem, where I classify my images into n classes and also a background class. I am currently trying to figure out how to find the optimal thresholds (for each ...
1 vote
0 answers
66 views
Is it a good practice to use AB testing to choose threshold of ML models?
Assuming there is a ML model learning the likelihood of an item being clicked by users. The output is typically a number between [0,1] indicating the probability of the item. Then you need to decide a ...
1 vote
1 answer
101 views
Why can a classifier's predicted labels be improved (with respect to the same metric the classifier optimizes) by adjusting classification threshold?
I am hoping to enhance my (and others' perhaps too) understanding of some basic principles, which seem surprisingly elusive. For a start, I would like to consider imbalanced binary classification, ...
0 votes
0 answers
271 views
Is it better to use the W or p-value to determine normality using Shapiro-Wilk?
I have read somewhere that W values above 0.9 are considered normal. So I wanted to use that as a cutoff for normality. However, upon running various scenarios I have come up with the result of W = 0....
0 votes
0 answers
50 views
Probabilistic Bounds on the Frequency of Empirical Mean Threshold Crossings for Sub-Gaussian or Bernoulli Variables
Consider one $1-$sub-gaussian or Bernoulli variables $X$, we i.i.d. sample $X$ $n$ times $(X_1,...,X_n)$; and $\mu_X<S$, $S$ is a constant threshold. We have such a quantity: $N_X = \sum_{i=1}^n \...
1 vote
1 answer
67 views
Extract credible/confidence interval of a threshold in a Bayesian posterior draws distribution [closed]
I have a Bayesian model created through bayer package in R on which I need to calculate confidence/credible intervals for a ...
4 votes
2 answers
280 views
What data are used to find the final threshold for a medical diagnostic test?
Suppose I have some blood measurement X whose values correlate with some disease Y (so people with the disease use to have larger values of X). Moreover suppose that the disease is rare, say 1% of a ...
1 vote
1 answer
202 views
Choosing the correct evaluation metric between F1-score and Area under the Precision-Recall Curve (AUPRC)
We're currently working on detecting specific objects (e.g. poultry farms, hospitals) from satellite images. We've modeled the problem as a binary image classification task (i.e. classifying images ...
2 votes
1 answer
418 views
What is the standard threshold value that is best for accuracy when employing Euclidean distance as a metric for gauging textual similarity?
I'm using Euclidean distance as a metric to compare two sentences for similarity while clustering them using my custom incremental KMeans algorithm. The current threshold value I'm using is 0.7 which ...
2 votes
1 answer
113 views
What if instead re-training our classification model, we only adjust the probability threshold?
As far as i know theoretically our model tend to be drifting/shifting as time goes on and need to be retrained. i wonder if its acceptable that instead of retraining the classification model, we keep ...
0 votes
0 answers
57 views
non-linear correlation or finding thresholds for changes with relatively few data points
I'm a novice at stats and thought I'd ask this question to those with much more experience than me. I've got temperature data and count data for number of animals hibernating in man-made boxes. The ...
1 vote
0 answers
69 views
Why following Poisson distribution can be explained as a result of chance?
I'm reading a journal article which applies Poisson distribution in determining how many factors can be regarded as beyond the poverty threshold. My question is: why applying Poisson distribution and ...