SYS/DEF
Sequential test
A boundary that moves with n, so intermediate looks do not explode α. SPRT is not a classic A/B looked at more often.
Definition
A sequential test defines, for each sample size, a stopping bound: you may look often, because the stopping rule was calibrated to keep α (and often β) under control. Evan Miller’s SPRT for two proportions compares a T − C score to a 2√N-style boundary — pedagogically clear, it is not the Waudby-Smith confidence sequence some platforms use.
The difference with naïve peeking: here the procedure includes the looks. There, you steal looks from a procedure that forbade them.
When it matters
When the business cannot wait for a fixed n (short season, incident to watch) and accepts a slightly more conservative design or a different expected size. When you want to monitor without lying about α.
It is not mandatory. A fixed-n test with an end date remains Shamalo’s default, easier to explain.
Pitfalls
Wiring an SPRT after two weeks of classic peeking. Believing “sequential” means “we can ship as soon as it hits Monday’s bound”. Ignoring that some platforms run modern sequential (Waudby-Smith): do not paste their p-values onto a fixed z, nor the reverse. Forgetting SRM: an elegant boundary on a broken split is still invalid.
Related terms
Related tool
α inflation from k looks, and Evan Miller’s SPRT boundary — not a Waudby-Smith implementation: Open peeking / sequential testing.