Revision note, September 2026. The original 2019 version of this article recommended Shapiro-Wilk for small samples and Anderson-Darling for large samples. That rule is too simplistic and, in important cases, wrong. I have rewritten the article to distinguish the two tests more carefully and, more importantly, to explain why formal normality testing should not be used as an automatic gatekeeper for t-tests, ANOVA, regression, or other downstream analyses.

Normality tests are easy to run and surprisingly easy to misuse.

A common workflow is:

That looks systematic. It is often bad statistical practice.

The first question should not be

Which normality test should I use for this sample size?

It should be

Only after answering that does it make sense to discuss Shapiro-Wilk, Anderson-Darling, Q-Q plots, residual diagnostics, transformations, robust methods, or alternative models.

What a normality test actually tests

For a sample

a normality test typically considers the null hypothesis

for some unknown parameters $\mu$ and $\sigma$.

That is a very specific hypothesis: exact distributional normality.

Two consequences follow immediately.

First, failing to reject $H_0$ does not prove that the population is normal. A small sample may simply have too little power to detect the departure that exists.

Second, rejecting $H_0$ does not tell us whether the departure matters for the analysis we care about. With enough observations, a formal test can identify deviations from exact normality that are scientifically or inferentially negligible.

So the output of a normality test is not

It is evidence about the compatibility of the data with a particular normal model.

The normality assumption is often attached to the wrong object

The original version of this article stated too broadly that t-tests and ANOVA require the observed data themselves to be normally distributed.

That is not the right general statement.

For a classical linear model,

the finite-sample normal-theory assumption concerns the errors

not the marginal distribution of $Y$ across every observation pooled together.

Similarly:

• in a paired t-test, the relevant object is the distribution of the paired differences;
• in a two-sample t-test, exact small-sample theory is based on assumptions about the distributions within groups, while large-sample inference can be much more robust;
• in ANOVA, diagnostics concern within-group errors or residuals rather than the unadjusted response distribution treated as one homogeneous sample;
• in regression, a histogram of the raw outcome may tell us very little about whether the model's error structure is adequate.

This distinction matters because testing the wrong object can reject a model that is perfectly reasonable for the scientific question.

Why sample-size cutoffs are the wrong way to choose a test

The old rule

should be discarded.

There is no general sample-size threshold at which Anderson-Darling becomes the correct replacement for Shapiro-Wilk.

Power depends on at least three things:

Simulation studies comparing normality tests repeatedly show that rankings change with the alternative distribution. Shapiro-Wilk often has very strong omnibus power, while Anderson-Darling is designed to place greater weight on discrepancies in the tails. Neither statement implies a universal sample-size switching rule.

And crucially, both tests become more sensitive as the sample size increases.

Anderson-Darling does not solve the large-sample problem of detecting tiny but irrelevant deviations. A large enough dataset can make either test reject a distribution that is close enough to normal for the intended inference.

Shapiro-Wilk: an order-statistic correlation test

The Shapiro-Wilk statistic is

where

are the ordered observations and the coefficients $a_i$ are derived from the expected order statistics of a normal sample.

The test is sensitive to systematic departures in the ordered sample from what normal order statistics should look like.

Historically, Shapiro-Wilk earned its reputation because it performs very well across many non-normal alternatives. That does not mean it is only a small-sample test.

The useful interpretation is instead:

Shapiro-Wilk is a strong general-purpose test of normality, but its p-value must still be interpreted in the context of sample size and the scientific consequences of non-normality.

There is also a software-specific caveat. Current SciPy documentation notes that for

the $W$ statistic is accurate but its reported p-value may not be. That is an implementation limitation, not a theoretical rule saying that Shapiro-Wilk suddenly becomes statistically inappropriate at $n=5001$.

Anderson-Darling: an empirical-distribution test with tail emphasis

The Anderson-Darling statistic belongs to the family of empirical distribution function tests.

In simplified form it measures a weighted discrepancy between the empirical distribution

and the fitted theoretical distribution

For ordered observations, the familiar computational form includes terms such as

and

which gives observations near the tails substantial influence.

That is the key conceptual difference:

This can be valuable when tail behaviour is substantively important. Examples include reliability, risk, extremes, or analyses in which tail probabilities drive decisions.

But “tail-sensitive” is not the same as “better for large samples.”

Small samples: failure to reject may mean very little

Suppose $n=12$ and the Shapiro-Wilk test returns

It is tempting to write:

The data are normally distributed.

That conclusion is too strong.

A better statement is:

The sample does not provide strong evidence against the fitted normal model.

With only twelve observations, many non-normal populations can easily produce samples that look unremarkable. Formal tests have limited power to discriminate among nearby distributions.

This is one reason graphical diagnostics are particularly important in small samples, even though the graphs themselves are also noisy.

Large samples: statistical significance can become scientifically uninteresting

Now suppose $n=100{,}000$.

The true population distribution is almost normal but has a tiny amount of skewness.

A formal test may produce

That result answers the narrow mathematical question:

It does not answer the question:

Those are different questions.

For many estimators of means and regression coefficients, large-sample inference can remain useful far outside exact normality. Lumley, Diehr, Emerson, and Chen made this point forcefully in their review of large public-health datasets: the practical value of t-tests and linear regression does not come from real outcomes being exactly normal.

Normality should not be a pre-test for choosing the scientific question

There is another subtle problem with the usual workflow.

Researchers sometimes do this:

1. test normality;
2. if $p>0.05$, perform a t-test;
3. otherwise, perform a rank-based test.

But a t-test and a rank-based test do not necessarily estimate or test the same scientific quantity.

A two-sample t-test is fundamentally about a difference in means.

A Wilcoxon-Mann-Whitney procedure is based on ranks and has a different probabilistic interpretation unless additional distributional assumptions are imposed.

So the choice between them should not be reduced to whether a normality test crossed $0.05$.

The estimand comes first.

Q-Q plots answer a different and often more useful question

A normal Q-Q plot compares observed order statistics with theoretical normal quantiles.

The point is not merely to decide whether the dots lie perfectly on a line. They almost never do.

The useful questions are structural:

• Is there systematic skewness?
• Are both tails heavier than expected?
• Is one tail driving the discrepancy?
• Are there a few isolated outliers?
• Does the centre fit well while the extremes do not?
• Is there evidence of a mixture or several populations?

These patterns can tell us how the normal model fails, which is often more useful than knowing that a formal hypothesis test rejected it.

A better diagnostic workflow

Instead of choosing Shapiro-Wilk or Anderson-Darling from a table of sample-size thresholds, I would use the following sequence.

1. Identify the statistical object

Ask what needs to be approximately normal, if anything:

2. Identify why normality matters

Is normality needed for:

• exact finite-sample inference;
• prediction intervals;
• a likelihood model;
• tail probabilities;
• residual modelling;
• merely a convenient approximation?

The consequence determines how strict the diagnostic needs to be.

3. Plot the data or residuals

Use a Q-Q plot together with context-specific residual plots. Look for the shape of the departure rather than merely a binary verdict.

4. Use a formal test if it answers a useful question

Shapiro-Wilk is a strong omnibus option. Anderson-Darling is attractive when tail discrepancies deserve extra weight.

But interpret

as evidence against exact normality, not as a certificate of whether a downstream method is valid.

5. Evaluate consequences directly

If the analysis matters, check the robustness of the actual inferential target.

That may mean:

• heteroskedasticity-robust standard errors;
• bootstrap intervals;
• permutation procedures appropriate to the null hypothesis;
• robust regression;
• alternative likelihoods;
• transformations;
• simulation under plausible non-normal distributions;
• sensitivity analyses for outliers or tail assumptions.

This is usually more informative than debating whether a normality-test p-value is $0.04$ or $0.06$.

Python examples

Shapiro-Wilk

1
2
3
4
5
6
7
8
import numpy as np
from scipy.stats import shapiro

rng = np.random.default_rng(2026)
x = rng.normal(size=80)

result = shapiro(x)
print(result.statistic, result.pvalue)

Anderson-Darling

Current SciPy versions can evaluate Anderson-Darling against a normal distribution directly:

1
2
3
4
from scipy.stats import anderson

result = anderson(x, dist="norm")
print(result.statistic)

Depending on the SciPy version and method used, the result may expose critical values or a p-value. The important point is to consult the documentation for the installed version rather than assuming all releases expose the same interface.

Q-Q plot

1
2
3
4
5
import matplotlib.pyplot as plt
from scipy.stats import probplot

probplot(x, dist="norm", plot=plt)
plt.show()

For model diagnostics, replace x with the relevant residuals or differences rather than automatically testing the raw outcome.

Shapiro-Wilk versus Anderson-Darling

A better comparison is therefore:

The main lesson

The useful distinction is not

versus

It is

Shapiro-Wilk and Anderson-Darling are both useful tools. Neither should be treated as a mechanical permission slip for the rest of the analysis.

Normality is a modelling assumption whose importance depends on the estimand, model, sample size, and inferential procedure.

That is a much more useful question than asking which normality test wins a sample-size contest.

References

• Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3/4), 591-611. https://doi.org/10.2307/2333709
• Anderson, T. W., & Darling, D. A. (1954). A test of goodness of fit. Journal of the American Statistical Association, 49(268), 765-769. https://doi.org/10.1080/01621459.1954.10501232
• Stephens, M. A. (1974). EDF statistics for goodness of fit and some comparisons. Journal of the American Statistical Association, 69(347), 730-737. https://doi.org/10.1080/01621459.1974.10480196
• Yap, B. W., & Sim, C. H. (2011). Comparisons of various types of normality tests. Journal of Statistical Computation and Simulation, 81(12), 2141-2155. https://doi.org/10.1080/00949655.2010.520163
• Lumley, T., Diehr, P., Emerson, S., & Chen, L. (2002). The importance of the normality assumption in large public health data sets. Annual Review of Public Health, 23, 151-169. https://doi.org/10.1146/annurev.publhealth.23.100901.140546
• SciPy documentation. scipy.stats.shapiro and scipy.stats.anderson, consulted September 2026.