Experimental Design Blueprint with Power Analysis

Thoughtful experiment logs help product teams align on hypotheses before shipping features. This blueprint captures the essentials—design tables, power analysis code, and interpretation guidelines—all rendered cleanly by the DataLog theme.

Hypotheses

Sample-size planning

1
2
3
4
5
6
7
8
9
10
11
12
13
from statsmodels.stats.power import NormalIndPower

effect = 0.04  # minimum detectable effect (absolute)
alpha = 0.05
power = 0.8
baseline = 0.32

analysis = NormalIndPower()
n_per_group = analysis.solve_power(effect_size=effect / (baseline * (1 - baseline)) ** 0.5,
                                  power=power,
                                  alpha=alpha,
                                  ratio=1.0)
print(round(n_per_group))

Reminder: Adjust for multiple comparisons if you expect to peek at intermediate checkpoints.

Test plan

Decision framework

1. Pre-register hypotheses and guardrails in _datasets/experiment-hypotheses.csv.
2. Automate metric extraction via notebooks stored in _notebooks/.
3. Attach Tableau or Looker dashboards with the viz-block include for executive readouts.

By combining Markdown tables, statistical code, and callouts, the post becomes a reusable experimentation template for every squad.