SYS/DEF
Confidence interval
The range of effects still plausible given the data. A p-value without a CI is a decision without a magnitude.
Definition
A 95% confidence interval is a procedure: if you repeated the experiment many times, about 95% of intervals built this way would cover the true value. It is not “95% chance the true lift sits in the interval” in the Bayesian sense — even if, in practice, we use it as a range of uncertainty.
For relative lift, the CI is computed on the difference of proportions (Wald, or a variant) then translated into %. An interval that contains 0 says “no effect” remains compatible with the data, even if p is 0.049.
When it matters
Always, next to the point lift. It is the honest readout for a committee: “we estimate +8%, and the range runs from +1% to +15%” is not the same decision as “+8%, range −4% to +21%”.
It matters for practical significance: an effect statistically non-zero whose whole CI sits under the implementation cost is not a ship.
Pitfalls
Displaying ±2 points because “it looks cleaner”. Reading a 95% CI as an individual guarantee. Recomputing the CI at every peek (the bounds dance, so does α). Forgetting that Wald is fragile on small n or rates near 0 — hence the method note on the lift tool.
Related terms
Related tool
Relative lift and 95% CI, shown together, no marketing rounding: Open lift and confidence interval.