SYS/DEF
α risk
The budgeted false positive: rejecting H0 when there is no effect. 5% means one false alarm in twenty true-null tests.
Definition
α risk (type I error) is the probability, set in advance, of declaring an effect when H0 is true. The 0.05 threshold is a lab convention, not a physical constant. On each independent true-null test, you “win” a false positive one time in twenty. Ten tests is another story — hence Bonferroni and friends.
α is not the p-value. α is the budget; p is one sample’s bill.
When it matters
At design time, in writing, with power and MDE. A stricter α (1%) if a false win is expensive to build. A looser α if you are exploring — but then you do not call it a ship decision.
It is also consumed by peeking and by multiple metrics. The 5% α on the slide is the real α only if the procedure is the one you planned.
Pitfalls
Lowering α after a p of 0.06 to “pass”. Believing α = 5% protects a farm of 30 tests. Confusing α and β. Announcing “we’re at 95% confidence” while mixing CI, power, and 1 − p.
Related terms
Related tool
Visitors, conversions and α: z, p, decision — the α budget is read here, not on a slide: Open A/B significance.