Timeline for How to compute confidence intervals for the performance estimate of nested cross validation?
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 2, 2021 at 15:08 | history | edited | Michael Berk | CC BY-SA 4.0 | deleted 26 characters in body |
| May 29, 2021 at 2:33 | comment | added | Michael Berk | Hey @fabiob. Yes regarding the variance - because we fit using regular CV, there is dependence between splits. For the second point, it's the mean of a single "holdout_loss" vector at index [i, i]. Remember each of the values in that matrix is a vector based on the user-specified loss function. | |
| May 28, 2021 at 16:18 | comment | added | fabiob | hey @Michael, thanks so much, your write up makes the content of the original paper more accessible. some comments: "Also note that each of our “fold_n_loss” vectors are biased because we refit the model on previously seen data." what is biased is the variance between the vector elements, right? also, what is meant with "mean(holdout_loss[i, i])". is the mean taken over the K-1 folds? | |
| May 28, 2021 at 16:06 | vote | accept | fabiob | ||
| May 26, 2021 at 16:15 | history | edited | Michael Berk | CC BY-SA 4.0 | added 697 characters in body |
| May 26, 2021 at 15:07 | history | answered | Michael Berk | CC BY-SA 4.0 |