
Standard practice

Treatment A

Treatment B

Optimal choice


Simulation 1

140

150

160

B

Simulation 2

100

110

120

B

Simulation 3

110

100

100

Standard

Simulation 4

100

150

130

A

Simulation 5

130

120

110

Standard

Average expected net benefit

116

126

124

Standard/A/B = 40%/20%/40%

 Results are expressed as monetary net benefits, each simulation is equally likely to be 'true'. Treatment A is associated with the highest expected net benefit (AUD $126), but because the distribution of monetary net benefits is skewed, it is preferred in only 20% of samples. Treatment A is therefore optimal, but the error probability associated with this choice is 80%. This probability is substantially higher than the 5% used for tests of statistical significance. The choice to remain with standard practice still carries a 40% probability of not returning the highest monetary net benefits and could be expected to incur economic costs of AUD $10 (AUD $126 minus AUD $116). The alternative with the highest monetary net benefit is the optimal decision, but that decision can be highly uncertain.