Table 2 Some surprising statements about P values, results of Bayesian methods, and empirical evidence supporting the predictions of Bayesian methods
Surprising P-value misinterpretations (false statements) | Correction | Reference |
 The P value is the probability that the null hypothesis is true |  The P value assumes the null hypothesis is true |  [10] |
 P ≤ 0.05 means the null hypothesis is false, or should be rejected |  P ≤ 0.05 simply flags the data as being unusual if all the assumptions used to compute it were correct |  [10] |
 P > 0.05 means the null hypothesis is true, or should be accepted |   P > 0.05 only suggests that the data are not unusual if all the assumptions used to compute it were correct; the same data would also not be unusual under many other hypotheses |  [10] |
 If you reject the null hypothesis because P ≤ 0.05, the chance your “significant finding” is a false positive is 5% |  The P value only refers to how often you would be in error over very many uses of the test across different studies, and not in a single use of the test |  [10] |
Surprising results of Bayesian methods | Â | |
 In late-phase clinical trials with equipoise (the prior probability of the null hypothesis is 50%), a study with a P = 0.05 makes the posterior probability of the null hypothesis no less than 13% |  [11] | |
 In more exploratory research (the prior probability of the null hypothesis is, say, 75%), a study with a P = 0.05 or P = 0.01 makes the posterior probability of the null hypothesis no less than 31% and 10%, respectively |  [11] | |
 An adequately powered (80%) exploratory epidemiologic (prior 1:10, bias 0.3, α = 0.05) study with a statistically significant finding has a positive predictive value (PPV) 20% and, if underpowered (20%), a PPV of 10% |  [12] | |
 In large traditional cohort studies (prior 1:20, bias 0.1, α = 0.05, power 90%), the false positive to false negative ratio of findings is 32:1 |  [13] | |
 In a well done (power 95%, α = 0.05) cohort study testing SNPs with less than compelling evidence (prior 1:100), with a statistically significant finding (P = 0.05 or 0.01) the PPV is 16.1% and <60%; even with fairly compelling prior evidence (prior 1:10), the PPV is 67.9% and <90% |  [14] | |
Surprising empirical evidence supporting the predictions of Bayesian methods | Â | |
 In traditional genome epidemiology [a “few candidate risk factors are selected based on diverse considerations” (low prior); small sample size (low power, given the small size of expected effect); “discovery hunting using conventional levels” of statistical significance, confounding, selective reporting (bias)], the crude replication rate of statistically significant genetic associations is ~1.2% |  [13] | |
 Hallmarks of discovery exploratory research (low priors, low BF, high bias): “vibration of effects” (evidence of inflated early effect sizes in epidemiologic associations), “Proteus phenomenon” (a rapid early sequence of extreme, opposite results in retrospective hypothesis-generating molecular research), and “winners curse” (the first positive study provides inflated estimates compared to reality) | ||