## What is a False Negative Rate?

False negative rate refers to the proportion of significance tests that failed to reject the null hypothesis when the null hypothesis is indeed false. The beta of of a test procedure provides a conservative (upper) bound on the rate of type II error in tests performed using this procedure relative to an effect size of certain magnitude, or higher. Assessing the actual false negative rate is impossible unless the proportion of nulls which are actually false is known.

Running tests with higher statistical power to detect smaller effect sizes will improve the rate of false negatives at the cost of larger sample size / longer test duration, assuming there are at least some false null hypotheses. If there are no false nulls, the false negative rate will be 0 regardless of any characteristics of the test procedure.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.