This report shows the same A/B test through three lenses. The default view (CUPED) is the most trustworthy — the others are sanity checks.
Every KPI row shows an observed delta (▲ +5.2%) and a 95% confidence interval (95% CI [-2.1, +12.3]) below it. The CI is the range of plausible true effects given the data:
| CI looks like | Interpretation | Bar colour |
|---|---|---|
[+2, +8] all positive |
Test is reliably better | green |
[-9, -1] all negative |
Test is reliably worse | red |
[-5, +12] crosses 0 |
Can't tell — treat as noise | grey |
A grey bar means the test might be moving the metric, but we don't yet have enough data to say so with confidence. Wide CIs come from heavy-tailed revenue + short test windows + small effect sizes — not from a bug. To halve a CI typically takes ~4× more data.
Conversion (payers / DAU) × ARPPU (revenue / payer) = ARPDAU. Showing them as separate rows makes it obvious whether a revenue lift comes from more payers, bigger baskets, or both.
See the A/B Testing section of the framework README for the full technique reference.