What is a confidence interval?
A confidence interval is a range of plausible values for a result, such as a lift, calculated from your data. A 95% interval comes from a method that, if you repeated the test many times, would produce ranges containing the true value about 95% of the time.
Last reviewed .
How to read a confidence interval
Suppose an A/B test shows a 10% lift with a 95% confidence interval from 2% to 18%. The data is consistent with a small gain or a large one, and 10% is just the middle estimate.
A wide interval means your estimate is uncertain. A narrow one means it's more precise, which usually requires more data.
What to watch for
If the interval includes zero for a difference, the data doesn't clearly show that an effect exists. That's the same information a p-value gives you, but an interval also shows how big the effect could be.
The confidence level, such as 90% or 95%, is a choice. A higher level gives a wider interval.
Common questions
What does a 95% confidence interval mean?
It means the method used to build the interval captures the true value in about 95% of repeated samples. It doesn't mean there's a 95% chance the true value is in your one interval.
How do you make a confidence interval narrower?
Collect more data, since larger samples give more precise estimates. Reducing variability in your measurements helps too. Lowering the confidence level also narrows the range, but at the cost of certainty.
What is the difference between a confidence interval and a p-value?
A p-value measures how surprising your data would be if there were no real effect. A confidence interval gives a range of plausible effect sizes, so it also tells you how large the effect might be.
Why do marketers care about confidence intervals?
They show how much to trust a result before you shift budget or change a page. A promising lift with a wide interval may not be reliable.
Want help putting this to work?
Tell us where your marketing stands today, and we'll show you what to do first.