Understanding the z-score can significantly enhance your data analysis skills. Here’s a quick guide to what z-scores are and why they matter:

🔍 What is a Z-Score?

A z-score, or standard score, indicates how many standard deviations an element is from the mean. A z-score of 0 means the value is exactly average, while a z-score of +1.5 indicates a value 1.5 standard deviations above the average.

\[z = \frac{x - \mu}{\sigma} \quad \text{(population)}, \qquad z = \frac{x - \bar{x}}{s} \quad \text{(sample)} .\]

The distinction matters more than it looks. Using $\mu$ and $\sigma$ assumes you know the population parameters; using $\bar{x}$ and $s$ means both are estimated from the same data being standardised, which introduces dependence between the z-scores. They are no longer independent, and by construction they always sum to zero and have a sample standard deviation of exactly 1.

Standardising does not change the shape of a distribution. It shifts and rescales it, so a skewed variable remains exactly as skewed after conversion to z-scores. This is the single most common misconception about the technique: z-scores do not make data normal.

📊 Why Use Z-Scores?

  • Comparability: Z-scores allow comparison between different data sets with various means and standard deviations.
  • Outlier Detection: High or low z-scores can reveal outliers in data.
  • Standardization: Z-scores help standardize data, preparing it for techniques that assume normal distribution.

Comparability is the strongest of the three. A score of 85 on one exam and 630 on another are incomparable until both are expressed in standard deviations from their respective means. This is also why z-scores underpin distance-based methods such as k-means and k-nearest neighbours, where an unscaled variable with large units would otherwise dominate the distance metric.

⚠️ The Sample Size Trap in Outlier Detection

The familiar rule of thumb, that $ z > 3$ indicates an outlier, carries a mathematical constraint that is rarely mentioned. For a sample of size $n$, the largest possible z-score is bounded:
\[|z|_{\max} \le \frac{n - 1}{\sqrt{n}} .\]
The consequences are concrete. With $n = 10$, as in the example below, the maximum attainable z-score is about 2.85, so the $ z > 3$ rule can never flag anything, no matter how extreme the value. You need $n \ge 11$ before the threshold is reachable at all, and considerably more before it is a sensible criterion.

There is a second problem in the opposite direction. A genuine outlier inflates both $\bar{x}$ and $s$, and since it appears in its own denominator, it drags the threshold toward itself. This is masking: one extreme value can hide another, and two outliers together can hide each other so effectively that neither is flagged.

clean    <- c(10, 11, 12, 11, 10, 12, 11, 10, 11, 12)
polluted <- c(clean, 95)      # one clear outlier

z <- function(x) (x - mean(x)) / sd(x)
max(abs(z(polluted)))         # ~3.0, only just detectable

Add a second outlier and the standard deviation grows enough that neither reaches the threshold.

🛡️ A Robust Alternative

When the goal is outlier detection rather than standardisation, the modified z-score replaces the mean with the median and the standard deviation with the median absolute deviation:

\[M_i = \frac{0.6745\,(x_i - \tilde{x})}{\text{MAD}}, \qquad \text{MAD} = \operatorname{median}\left(|x_i - \tilde{x}|\right).\]
The constant 0.6745 rescales MAD so that it estimates the standard deviation for normally distributed data, keeping the familiar interpretation. Because both the median and MAD have a breakdown point of 50%, they are unaffected until half the data is contaminated, which eliminates the masking problem entirely. A threshold of $ M > 3.5$ is the usual convention. 
modified_z <- function(x) {
  med <- median(x)
  mad_val <- median(abs(x - med))
  0.6745 * (x - med) / mad_val
}

modified_z(polluted)          # the outlier now scores far beyond 3.5

One caveat: if more than half the values are identical, MAD is zero and the score is undefined. Falling back to the mean absolute deviation handles that case.

🚧 Limitations of Z-Scores

  • Assumption of Normality: Z-scores are most effective when the data follows a normal distribution. Their reliability decreases with data that is heavily skewed or has extreme outliers.
  • Context Dependent: The interpretation of a z-score can vary by context; a z-score considered high in one field might be average in another.
  • Oversimplification: Relying solely on z-scores might oversimplify the analysis, potentially overlooking important nuances in the data.

The normality point is specifically about interpretation, not computation. You can standardise any numeric variable, but the familiar mapping from z-scores to probabilities, where roughly 68% of values fall within one standard deviation and 95% within two, holds only for normal data. For a heavy-tailed distribution those percentages can be badly wrong, and Chebyshev’s inequality gives the only distribution-free guarantee: at least $1 - 1/k^2$ of any distribution lies within $k$ standard deviations, which for $k=2$ promises merely 75%.

There is also a leakage trap when standardising for machine learning. Computing the mean and standard deviation over the full dataset before splitting lets test-set information influence training. Fit the scaler on the training data only, then apply those same parameters to validation and test sets.

💡 Conclusion

Z-scores transform your data, making complex analyses more accessible and your conclusions more reliable. Whether you’re examining student test results or assessing stock market fluctuations, z-scores can offer a clear picture of how each data point relates to the whole.

Use them freely for comparison and scaling. For outlier detection, prefer the modified z-score, and check that your sample is large enough for your chosen threshold to be attainable at all.

Tutorial: Computing Z-Scores in R

Here is a step-by-step tutorial on how to compute z-scores in the R programming language.

Step 1: Install and Load Necessary Packages

First, ensure you have the necessary packages installed. For basic z-score computation, the base R functions are sufficient, and everything in this tutorial runs without additional packages. The dplyr package becomes useful once you are standardising columns inside a larger data pipeline.

# Run once, interactively, not as part of a script
# install.packages("dplyr")

library(dplyr)

Step 2: Create Your Data

Let’s create a sample data set for demonstration purposes.

# Sample data: test scores
test_scores <- c(78, 85, 92, 88, 76, 95, 89, 84, 91, 87)

Step 3: Compute the Mean and Standard Deviation

Calculate the mean and standard deviation of the data set.

mean_score <- mean(test_scores)
sd_score <- sd(test_scores)

Note that sd() in R uses the $n-1$ denominator, giving the sample standard deviation. If you need the population version, multiply by $\sqrt{(n-1)/n}$.

Step 4: Calculate the Z-Scores

Use the mean and standard deviation to compute the z-scores.

z_scores <- (test_scores - mean_score) / sd_score

Base R also provides scale(), which does the same thing and returns the parameters it used as attributes, which is what you need when applying the same transformation to new data later.

z_alt <- as.numeric(scale(test_scores))
attr(scale(test_scores), "scaled:center")   # the mean used
attr(scale(test_scores), "scaled:scale")    # the sd used

Step 5: Combine the Data for Better Visualization

Combine the original scores with their corresponding z-scores into a data frame for better visualization.

# Create a data frame
scores_data <- data.frame(
  Test_Score = test_scores,
  Z_Score = z_scores
)

# Print the data frame
print(scores_data)

Complete R Script

Here is the complete R script combining all the steps, with the robust variant included for comparison:

library(dplyr)

# Sample data: test scores
test_scores <- c(78, 85, 92, 88, 76, 95, 89, 84, 91, 87)

# Standard z-scores
mean_score <- mean(test_scores)
sd_score   <- sd(test_scores)
z_scores   <- (test_scores - mean_score) / sd_score

# Modified (robust) z-scores
med      <- median(test_scores)
mad_val  <- median(abs(test_scores - med))
mod_z    <- 0.6745 * (test_scores - med) / mad_val

scores_data <- data.frame(
  Test_Score = test_scores,
  Z_Score    = round(z_scores, 3),
  Modified_Z = round(mod_z, 3)
)

print(scores_data)
cat("\nMax attainable |z| for n =", length(test_scores), ":",
    round((length(test_scores) - 1) / sqrt(length(test_scores)), 3), "\n")

That final line is a useful habit. It prints the ceiling on any z-score this sample can produce, which immediately shows whether your outlier threshold is even reachable.

This tutorial provides a clear path to computing z-scores in R, allowing you to standardize and compare your data effectively.

References

  • Shiffler, R. E. (1988). Maximum Z scores and outliers. The American Statistician, 42(1), 79-80.
  • Iglewicz, B., & Hoaglin, D. C. (1993). How to Detect and Handle Outliers. ASQC Quality Press.
  • Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean. Journal of Experimental Social Psychology, 49(4), 764-766.
  • Rousseeuw, P. J., & Croux, C. (1993). Alternatives to the median absolute deviation. Journal of the American Statistical Association, 88(424), 1273-1283.

You may also enjoy