The Bootstrap Sees Skewness

The Central Limit Theorem hands you a Gaussian and shrugs off everything beyond variance. The bootstrap does more: for the right kind of statistic, its sampling distribution automatically encodes the skewness of your data, which is precisely the first term the CLT threw away. That is the source of the bootstrap's real practical edge — not consistency, but second-order accuracy — and also the reason it fails spectacularly when you point it at things the CLT was never going to help you with either.

The plug-in principle, applied twice

Any parameter you actually care about can be written as a functional of the underlying distribution. The mean is $\mu(F) = \int x \, dF$, the median is $F^{-1}(1/2)$, the variance is $\int (x - \mu(F))^2 \, dF$. Given data $X_1, \ldots, X_n \sim F$, the empirical distribution $\hat F_n$ puts mass $1/n$ on each observed $X_i$, and the plug-in estimator is $\theta(\hat F_n)$. So $\bar X_n = \mu(\hat F_n)$, the sample median is $\hat F_n^{-1}(1/2)$, and so on.

Once you have $\hat\theta_n = \theta(\hat F_n)$, you want its sampling distribution — the law of $\hat\theta_n - \theta(F)$ when the data are drawn from $F$. But you don't have $F$. Efron's move (1979): plug in again. Sample $X_1^*, \ldots, X_n^*$ iid from $\hat F_n$ — this is exactly "resample with replacement" — compute $\hat\theta_n^* = \theta(\hat F_n^*)$, and use the conditional distribution of $\hat\theta_n^* - \hat\theta_n$ as a proxy. In symbols,

Read aloud: the sampling distribution of your estimator, if only you knew $F$, is approximated by the resampling distribution you can actually compute from the data. Here $\mathcal L$ is "law of", $P^*$ (below) means probability under the bootstrap conditional on the sample, and every starred quantity is a resample analogue of its unstarred cousin.

What first-order consistency guarantees

The classical result, Bickel–Freedman (1981) with related work by Singh (1981), says this approximation is asymptotically valid whenever $\mathrm{Var}(X_1) < \infty$:

where $s_n$ is the sample standard deviation. In words: with enough data, the bootstrap CDF is a uniformly good stand-in for the true sampling CDF of the standardized mean. That is exactly what the CLT already promises for the right-hand side — convergence to $\Phi$. At first order the bootstrap and the normal approximation supply the same object. The reason people reach for the bootstrap anyway is that it does more than first order.

Where the extra accuracy lives: Edgeworth

The CLT tells you the CDF of the standardized mean converges to $\Phi$. Edgeworth tells you how fast and in which direction. Let $\gamma = \mathbb E[(X_1 - \mu)^3]/\sigma^3$ be the population skewness, and assume mild smoothness (Cramér's condition, plus a finite third moment). Then

Read aloud: the true CDF equals the normal CDF, minus a skewness-driven correction of order $1/\sqrt n$, plus a smaller remainder. Here $\phi = \Phi'$ is the standard normal density and $(x^2 - 1)$ is a Hermite polynomial. If the data is right-skewed ($\gamma > 0$), the right tail is heavier than $\Phi$ suggests, so $P(T_n \le x) < \Phi(x)$ for large positive $x$, and the sign of the correction confirms that. The CLT alone can be off by $\Theta(1/\sqrt n)$; the Edgeworth term is what closes that gap.

Now watch what happens when you bootstrap. Under $\hat F_n$, the "population" mean is $\bar X_n$, the population variance is $s_n^2$, and the population skewness is the sample skewness $\hat\gamma = \frac{1}{n}\sum_i (X_i - \bar X_n)^3 / s_n^3$. The resample $X_i^*$ is iid from $\hat F_n$, so Edgeworth applies in the bootstrap world too, with the sample cumulants:

The two expansions have the same shape. The only difference between them, at the leading correction, is $\gamma$ versus $\hat\gamma$. And by a standard CLT applied to third moments, $\hat\gamma - \gamma = O_p(1/\sqrt n)$. Subtract:

In words: the bootstrap doesn't just match the normal approximation — it agrees with the true sampling distribution to order $1/n$, one whole power of $n$ better than the CLT alone.

That is why the bootstrap works. It plugs in a $\sqrt n$-consistent estimate of the very term the normal approximation was missing. Nothing more mysterious. The load-bearing step is Edgeworth's expansion existing for both $F$ and $\hat F_n$, with only $\hat\gamma - \gamma$ separating them. Singh's 1981 paper made this rigorous; Hall (1988, 1992) turned it into an industry.

Studentize, or lose the bonus

The argument above used the pivot $T_n = \sqrt n(\bar X_n - \mu)/\sigma$. But $\sigma$ is unknown. If instead you keep the un-studentized $\sqrt n(\bar X_n - \mu)$ and compare its true law to a normal $N(0, s_n^2)$ approximation, you inherit an $O_p(1/\sqrt n)$ error from $s_n^2 - \sigma^2$ that drowns the Edgeworth bonus. The fix is to studentize before you bootstrap. For $T_n = \sqrt n(\bar X_n - \mu)/s_n$, the analogous Edgeworth expansion has a different Hermite polynomial,

but the same argument goes through, and Hall (1992) shows that the bootstrap-t — bootstrap the studentized statistic, use its quantiles for a confidence interval — achieves $O_p(1/n)$ two-sided coverage error, while the plain normal-based $t$-interval is stuck at $O(1/\sqrt n)$. If you take one lesson home from Hall's book, it is studentize the statistic before you resample. The percentile bootstrap on a raw mean is throwing away exactly what makes the bootstrap worth using.

Where the whole scheme collapses: the maximum

The bootstrap requires the functional $\theta$ to be smooth in $F$ — technically, Hadamard-differentiable at $F$ tangentially to a suitable set — and the sample must be informative about the parts of $F$ that matter. Take the maximum of a uniform sample: $X_1, \ldots, X_n \sim U(0, \theta)$, $M_n = \max_i X_i$. The true limiting distribution of the recentered, rescaled max is

a continuous law. Now the bootstrap version. Draw indices $J_1, \ldots, J_n$ iid uniform on $\{1, \ldots, n\}$ and set $M_n^* = \max_i X_{J_i}$. The event $\{M_n^* = M_n\}$ is exactly "at least one $J_i$ picked the index of the sample max", so

The bootstrap distribution of $M_n^*$ has a point mass of size $\approx 0.632$ at $M_n$, no matter how large $n$. It cannot converge to a continuous law. This is Bickel and Freedman's canonical counterexample, and any statistic that pivots on extremes or on a boundary of the parameter space inherits some version of it — Andrews (2000) catalogued the boundary case, where the parameter of interest lies on the edge of its allowed set and standard bootstrap coverage fails.

The takeaway from the failures is not "don't bootstrap". It is that the bootstrap silently assumes the functional is differentiable at $F$ in the right sense, and the sample has enough coverage of the region that matters. Extremes and boundaries violate both. For those, use the $m$-out-of-$n$ bootstrap, subsampling (Politis–Romano), or an entirely different tool.

Recap: what's proved, what's sketched, what's open

Proved rigorously: Bickel–Freedman consistency for the mean under finite variance, and Singh/Hall's Edgeworth-based $O_p(1/n)$ error for the studentized bootstrap under a finite third moment plus Cramér's condition. The max-of-uniforms failure is a full computation, not a heuristic.

Sketched here: the two-Edgeworth subtraction that yields the second-order accuracy result. I gave the algebra for the mean; extending it to a general smooth functional needs the delta method plus a von Mises expansion, which is where Hadamard differentiability enters.

Open or subtle in practice: the exact range of statistics for which the second-order accuracy of the bootstrap-t holds without adjustment — heavy-tailed data, weakly dependent data, high-dimensional means where $p/n \not\to 0$ (Chernozhukov–Chetverikov–Kato have pushed the theory here since roughly 2013, and the story is still being written). Trust the bootstrap on smooth functionals of well-behaved iid data. Verify it, or replace it, everywhere else.