Articles, newest first
Bandits or A/B Tests: What Adaptive Allocation Buys and What It Costs
Over 20,000 users, an even split between a 10 percent and a 13 percent variant gives up 301 conversions to learn which is better. Thompson sampling gives up ...
Read article
A Model That Fits the Data Can Still Be Wrong
Agreement with observed data is evidence that a model can reproduce those observations. It does not establish that the model is unique, that its parameters a...
Read article
The Problem With “Anti-Nutrients”
Lectins, phytates and oxalates are real compounds with real biological effects. The mistake is turning a context-dependent biochemical property into a genera...
Read article
Extreme Value Theory: Estimating the Tail You Have Not Seen
The worst hour in three years of load data was 265. The ten-year level is 326 and the hundred-year level 502. A normal fit says 159. The sample maximum, a ma...
Read article
Queueing: Why 90 Percent Utilisation Means Waiting
A maintenance crew is busy 85 percent of the time and the planner wants 95, because idle technicians are waste. The queue has other ideas. The last ten point...
Read article
The Elbow Method Does Not Estimate the True Number of Clusters
The elbow method is useful as a heuristic for balancing fit against complexity, but it is often interpreted too strongly. Within-cluster sum of squares decre...
Read article
A Forecast Distribution Should Be Calibrated and Sharp
A probabilistic forecast is useful only if its stated uncertainty is credible and concentrated enough to support decisions. Calibration without sharpness can...
Read article
Temporal Validation in Machine Learning: Testing Models Against the Future
Temporal validation evaluates machine learning models the way they will be used: trained on the past and tested on the future.
Read article
CUPED and Regression Adjustment: Cutting A/B Test Variance With Data You Already Have
The experiment needs 1,570 users per arm to detect a two percent lift. Each user's spend over the previous month is already in the warehouse and correlates 0...
Read articleRandomness Does Not Owe Us a Reversal
Four heads in a row do not make tails due on an independent fair coin. But history can matter in other mechanisms. Three examples show exactly when the next ...
Read article