SYS/DEF
CUPED
Controlled-experiment using pre-experiment data: strip variance with a pre-period covariate. Less noise, not a different lift.
Definition
CUPED (Deng et al., Microsoft) reduces a metric’s variance by regressing it on a covariate observed before the experiment — typically the same metric in a pre-period. If the correlation ρ between pre-period and outcome is strong, residual variance is about Var × (1 − ρ²). Less variance → more power, or a shorter test, for the same real effect.
CUPED does not create lift. It sharpens the estimator. A ρ of 0.4 removes 16% of variance; a ρ of 0.8 removes 64%. With no correlated covariate, CUPED does nothing.
When it matters
On noisy continuous metrics (RPV, AOV, time spent) when you have a stable user history. Less useful on a rare cold conversion, where the pre-period is often a string of zeros.
It is a planning tool: estimate ρ, then the duration factor you gain, before promising a one-week test to the board.
Pitfalls
Using a post-treatment covariate (it absorbs the effect). Believing CUPED licenses peeking. Applying it as a black box without checking that the pre-period itself is unbiased (seasonality, new cohort). Confusing variance reduction with bias reduction — CUPED does not fix an SRM.
Related terms
Related tool
How much variance ρ removes, and the test time you gain — educational, not an engine on your logs: Open CUPED / variance reduction.