When reading about how to approximate the distribution of the sample mean I came across the nonparametric bootstrap method. Apparently one can approximate the distribution of $\bar{X}_n-\mu$ by the distribution of $\bar{X}_n^*-\bar{X}_n$, where $\bar{X}_n^*$ denotes the sample mean of the bootstrap sample.
My question then is: Do I need the centering? What for?
Couldn't I just approximate $\mathbb{P}\left(\bar{X}_n \leq x\right)$ by $\mathbb{P}\left(\bar{X}_n^* \leq x\right)$?
Yes, you can approximate $\mathbb{P}\left(\bar{X}_n \leq x\right)$ by $\mathbb{P}\left(\bar{X}_n^* \leq x\right)$ but it is not optimal. This is a form of the percentile bootstrap. However, the percentile bootstrap does not perform well if you are seeking to make inferences about the population mean unless you have a large sample size. (It does perform well with many other inference problems including when the sample size size is small.) I take this conclusion from Wilcox's Modern Statistics for the Social and Behavioral Sciences, CRC Press, 2012. A theoretical proof is beyond me I'm afraid.
A variant on the centering approach goes the next step and scales your centered bootstrap statistic with the re-sample standard deviation and sample size, calculating the same way as a t statistic. The quantiles from the distribution of these t statistics can be used to construct a confidence interval or perform a hypothesis test. This is the bootstrap-t method and it gives superior results when making inferences about the mean.
Let $s^*$ be the re-sample standard deviation based on a bootstrap re-sample, using n-1 as denominator; and s be the standard deviation of the original sample. Let
$T^*=\frac{\bar{X}_n^*-\bar{X}}{s^*/\sqrt{n}}$
The 97.5th and 2.5th percentiles of of the simulated distribution of $T^*$ can make a confidence interval for $\mu$ by:
$\bar{X}-T^*_{0.975} \frac{s}{\sqrt{n}}, \bar{X}-T^*_{0.025} \frac{s}{\sqrt{n}}$
Consider the simulation results below, showing that with a badly skewed mixed distribution the confidence intervals from this method contain the true value more frequently than either the percentile bootstrap method or a traditional inverstion of a t statistic with no bootstrapping.
compare.boots <- function(samp, reps = 599){ # "samp" is the actual original observed sample # "s" is a re-sample for bootstrap purposes n <- length(samp) boot.t <- numeric(reps) boot.p <- numeric(reps) for(i in 1:reps){ s <- sample(samp, replace=TRUE) boot.t[i] <- (mean(s)-mean(samp)) / (sd(s)/sqrt(n)) boot.p[i] <- mean(s) } conf.t <- mean(samp)-quantile(boot.t, probs=c(0.975,0.025))*sd(samp)/sqrt(n) conf.p <- quantile(boot.p, probs=c(0.025, 0.975)) return(rbind(conf.t, conf.p, "Trad T test"=t.test(samp)$conf.int)) } # Tests below will be for case where sample size is 15 n <- 15 # Create a population that is normally distributed set.seed(123) pop <- rnorm(1000,10,1) my.sample <- sample(pop,n) # All three methods have similar results when normally distributed compare.boots(my.sample) This gives the following (conf.t is the bootstrap t method; conf.p is the percentile bootstrap method).
97.5% 2.5% conf.t 9.648824 10.98006 conf.p 9.808311 10.95964 Trad T test 9.681865 11.01644 With a single example from a skewed distribution:
# create a population that is a mixture of two normal and one gamma distribution set.seed(123) pop <- c(rnorm(1000,10,2),rgamma(3000,3,1)*4, rnorm(200,45,7)) my.sample <- sample(pop,n) mean(pop) compare.boots(my.sample) This gives the following. Note that "conf.t" - the bootstrap t version - gives a wider confidence interval than the other two. Basically, it is better at responding to the unusual distribution of the population.
> mean(pop) [1] 13.02341 > compare.boots(my.sample) 97.5% 2.5% conf.t 10.432285 29.54331 conf.p 9.813542 19.67761 Trad T test 8.312949 20.24093 Finally here is a thousand simulations to see which version gives confidence intervals that are most often correct:
# simulation study set.seed(123) sims <- 1000 results <- matrix(FALSE, sims,3) colnames(results) <- c("Bootstrap T", "Bootstrap percentile", "Trad T test") for(i in 1:sims){ pop <- c(rnorm(1000,10,2),rgamma(3000,3,1)*4, rnorm(200,45,7)) my.sample <- sample(pop,n) mu <- mean(pop) x <- compare.boots(my.sample) for(j in 1:3){ results[i,j] <- x[j,1] < mu & x[j,2] > mu } } apply(results,2,sum) This gives the results below - the numbers are the times out of 1,000 that the confidence interval contains the true value of a simulated population. Notice that the true success rate of every version is considerably less than 95%.
Bootstrap T Bootstrap percentile Trad T test 901 854 890