Statistics review 11: Assessing risk

Abstract

Relative risk and odds ratio have been introduced in earlier reviews (see Statistics reviews 3, 6 and 8). This review describes the calculation and interpretation of their confidence intervals. The different circumstances in which the use of either the relative risk or odds ratio is appropriate and their relative merits are discussed. A method of measuring the impact of exposure to a risk factor is introduced. Measures of the success of a treatment using data from clinical trials are also considered.

Introduction

As an example, we shall refer to the findings of a prospective cohort study conducted by Quasney and coworkers [1] of 402 adults admitted to the Memphis Methodist Healthcare System with community-acquired pneumonia. That study investigated the association between surfactant protein B and acute respiratory distress syndrome (ARDS). Patients were classified according to their thymine/cytosine (C/T) gene coding, and patients with the C allele present (genotype CC or CT) were compared with those with genotype TT. The results are shown in Table 1.

Table 1 Number of patients according to genotype and disease outcome

The risk that an individual with the C allele present will develop ARDS is the probability of such an individual developing ARDS. In the study we can estimate this risk by calculating the proportion of individuals with the C allele present who develop ARDS (i.e. 11/219 = 0.050).

Relative risk

Relative risk (RR), or the risk ratio, is the ratio of the risk for the disease in the group exposed to the factor, to that in the unexposed group. For the data given in Table 1, if the presence of the C allele is regarded as the risk factor, then the RR for ARDS is estimated by the following:

This implies that people with the C allele present are approximately nine times as likely to develop ARDS as those without this allele. In general, using the notation presented in Table 2, the RR can be expressed as follows:

Table 2 Observed frequencies

The estimate of RR does not follow a Normal distribution. However, an approximate 95% confidence interval for the true population RR can be calculated by first considering the natural logarithm (ln) of the estimated RR. The standard error (SE) of ln RR is approximated by:

The 95% confidence interval [2] for the population ln RR is

(ln RR - 1.96 SE [ln RR]) to (ln RR + 1.96 SE [ln RR])

For the data given in Table 1, ln RR = ln(9.19) = 2.22, and the SE of ln RR is

Therefore, the 95% confidence interval for the population ln RR is given by

2.22 - 1.96 × 1.040 to 2.22 + 1.96 × 1.040 (i.e. 0.182 to 4.258)

We need to antilog (e^x) these lower and upper limits in order to obtain the 95% confidence interval for the RR. The 95% confidence interval for the population RR is therefore given by the following:

e^0.182 to e^4.258 (i.e. 1.12 to 70.67)

Therefore, the population RR is likely to be between 1.12 and 70.67. This interval gives a very wide range of possible values for the risk ratio. It is wide because of the small sample size and the rarity of ARDS. However, the interval suggests that the risk ratio is greater than 1, indicating that there is a significantly greater risk for developing ARDS in patients with the C allele present.

A RR equal to 1 would represent no difference in risk for the exposed group over the unexposed group. Therefore, a confidence interval not containing 1 within its range suggests that there is a significant difference between the exposed and the unexposed groups.

Odds ratio

The use of odds was introduced in Statistics review 8 [3]. The odds of an individual exposed to a risk factor developing a disease is the ratio of the number exposed who develop the disease to the number exposed who do not develop the disease. For the data given in Table 1, the estimated odds of developing ARDS if the C allele is present are 11/208 = 0.053.

The odds ratio (OR) is the ratio of the odds of the disease in the group exposed to the factor, to the odds of the disease in the unexposed group. For the data given in Table 1, the OR is estimated by the following:

This value is similar to that obtained for the RR for these data. Generally, when the risk of the disease in the unexposed is low, the OR approximates to the risk ratio. This applies in the ARDS study, where the estimate of the risk for ARDS for those with the C allele absent was 1/183 = 0.005. Therefore, again, the OR implies that patients with the C allele present are approximately nine times as likely to develop ARDS as those with genotype TT. In general, using the notation given in Table 2, the OR can be expressed as follows:

An approximate 95% confidence interval for the true population OR can be calculated in a similar manner to that for the RR, but the SE of ln OR is approximated by

For the data given in Table 1, ln OR = 2.26 and the SE of ln OR is given by the following:

Therefore, the 95% confidence interval for the population ln OR is given by

2.26 - 1.96 × 1.049 to 2.26 + 1.96 × 1.049 (i.e. 0.204 to 4.316)

Again, we need to antilog (e^x) these lower and upper limits in order to obtain the 95% confidence interval for the OR. The 95% confidence interval for the population RR is given by the following:

e^0.204 to e^4.316 (i.e. 1.23 to 74.89)

Therefore the population OR is likely to be between 1.23 and 74.89 – a similar confidence interval to that obtained for the risk ratio. Again, the fact that the interval does not contain 1 indicates that there is a significant difference between the genotype groups.

The OR has several advantages. Risk cannot be estimated directly from a case–control study, in which patients are selected because they have a particular disease and are compared with a control group who do not, and therefore RRs are not calculated for this type of study. However, the OR can be used to give an indication of the RR, particularly when the incidence of the disease is low. This often applies in case–control studies because such studies are particularly useful for rare diseases.

The OR is a symmetric ratio in that the OR for the disease given the risk factor is the same as the OR for the risk factor given the disease. ORs also form part of the output when carrying out logistic regression, an important statistical modelling technique in which the effects of one or more factors on a binary outcome variable (e.g. survival/death) can be examined simultaneously. Logistic regression will be covered in a future review.

In the case of both the risk ratio and the OR, the reciprocal of the ratio has a direct interpretation. In the example given in Table 1, the risk ratio of 9.19 measures the increased risk of those with the C allele having ARDS. The reciprocal of this (1/9.19 = 0.11) is also a risk ratio but measures the reduced risk of those without the C allele having ARDS. The reciprocal of the odds ratio – 1/9.63 = 0.10 – is interpreted similarly.

Both the RR and the OR can also be used in the context of clinical trials to assess the success of the treatment relative to the control.

Attributable risk

Attributable risk (AR) is a measurement of risk that takes into account both the RR and the prevalence of the risk factor in a population. It can be considered to be the proportion of cases in a population that could be prevented if the risk factor were to be eliminated. Whereas RR is a risk ratio, AR is a risk difference. It can be derived as follows using the notation in Table 2.

If exposure to the risk factor were eliminated, then the risk for developing the disease would be that of the unexposed. The expected number of cases is then given by this risk multiplied by the sample size (n):

The AR is the difference between the actual number of cases in a sample and the number of cases that would be expected if exposure to the risk factor were eliminated, expressed as a proportion of the former. From Table 2 it can be seen that the actual number of cases is a + c, and so the difference between the two is the number of cases that can be directly attributed to the presence of the risk factor. The AR is then calculated as follows:

Where the overall risk is defined as the proportion of cases in the total sample [4].

Consider the example of the risk of ARDS for different genotypes given in Table 1. The overall risk for developing ARDS is estimated by the prevalence of ARDS in the study sample (i.e. 12/402 [0.030]). Similarly, the risk among the unexposed (i.e. those without the C allele) is 1/183 (0.005). This gives an AR of (0.030 - 0.005)/0.030 = 0.816, indicating that 81.6% of ARDS cases can be directly attributable to the presence of the C allele. This high value would be expected because there is only one case of ARDS among those without the C allele.

There are two equivalent formulae for AR using the prevalence of the risk factor and the RR. They are as follows:

Where RR is the relative risk, p_E is the prevalence of the risk factor in the population and p_C is the prevalence of the risk factor among the cases. The two prevalence measurements can then be estimated from Table 2 as follows:

For the data in Table 1, the RR = 9.19, p_E = 219/402 = 0.545, and p_C = 11/12 = 0.917. Thus, both formulae give an AR of 81.6%.

Providing the disease is rare, the second formula allows the AR to be calculated from a case–control study in which the prevalence of the risk factor can be obtained from the cases and the RR can be estimated from the OR.

The approximate 95% confidence limits for attributable risk are given by the following [4]:

For the data given in Table 1:

This gives the 95% confidence interval for the population AR as

This indicates that the population AR is likely to be between 31.2% and 97.8%.

Risk measurements in clinical trials

Risk measurements can also be calculated from the results of clinical trials where the outcome is dichotomous. For example, in the study into early goal-directed therapy in the treatment of severe sepsis and septic shock by Rivers and coworkers [5], one of the outcomes measured was inhospital mortality. Of the 263 patients who were randomly allocated to either early goal-directed therapy or standard therapy, 236 completed the therapy period with the outcomes shown in Table 3.

Table 3 Outcomes of the study conducted by Rivers and coworkers

The RR is calculated as above, but in this situation exposure to the factor is considered to be exposure to the treatment, and the presence of the disease is replaced with success in the outcome (survived), giving the following:

This indicates that the chance for those who undergo early goal-directed therapy having a successful outcome is 1.34 times as high as for those who undergo the standard therapy.

The OR is obtained in a similar manner, giving the following:

This indicates that the odds of survival for the recipients of early goal-directed therapy are twice those of the recipients of the standard therapy. Because this is not a rare outcome, the RR and the OR are not particularly close, and in this case the OR should not be interpreted as a risk ratio. Both methods of assessing increased risk are viable in this type of study, but RR is generally easier to interpret.

The AR indicates that 14.4% of the successful outcomes can be directly attributed to the early goal-directed therapy and is calculated as follows:

Risk difference

Another useful measurement of success in a clinical trial is the difference between the proportion of adverse events in the control group and the intervention group. This difference is referred to as the absolute risk reduction (ARR). Therefore, for the data given in Table 3, the proportion of adverse outcomes in the control group is 59/119 (0.496) and that in the intervention group is 38/117 (0.325), giving an ARR of 0.496 -0.325 = 0.171. This indicates that the success rate of the therapy is 17.1% higher than that of the standard therapy.

Because the ARR is the difference between two proportions, its confidence interval can be calculated as shown in Statistics review 8 [3].

For the data given in Table 3 the SE is calculated as 0.0634, giving a 95% confidence interval of 0.047 to 0.295. This indicates that the population ARR is likely to be between 4.7% and 29.5%.

Number needed to treat

The number needed to treat (NNT) is also a measurement of the effectiveness of a treatment when the outcome is dichotomous. It estimates the number of patients who would need to be treated in order to obtain one more success than that obtained with a control treatment. This could equally well be described as the number that would need to be treated in order to prevent one additional adverse outcome as compared with the control treatment. This definition indicates its relationship with the ARR, of which it is the reciprocal.

For the data given in Table 3 the NNT value is 1/0.171 = 5.8, indicating that the intervention achieved one more success for every six patients receiving the early goal-directed therapy as compared with the standard therapy.

In an intervention the NNT would be expected to be small; the smaller the NNT, the more successful the intervention. At the other end of the scale, if the treatment had no effect then the NNT would be infinitely large because there would be zero risk reduction in its use.

In prophylaxis the difference between the control and intervention proportions could be very small, which would result in the NNT being quite high, but the prophylaxis could still be considered successful. For example, the NNT for use of aspirin to prevent death 5 weeks after myocardial infarction is quoted as 40, but it is still regarded a successful preventive measure.

Number needed to harm

A negative NNT value indicates that the intervention has a higher proportion of adverse outcomes than the control treatment; in fact it is causing harm. It is then referred to as the number needed to harm (NNH). It is a useful measurement when assessing the relative benefits of a treatment with known side effects. The NNT of the treatment can be compared with the NNH of the side effects.

As the NNT is the reciprocal of the ARR, the confidence interval can be obtained by taking the reciprocal of the confidence limits of the ARR. For the data given in Table 3 the 95% confidence interval for the ARR is 0.047 to 0.295, giving a 95% confidence interval for NNT as 3.4 to 21.3. This indicates that the population NNT is likely to lie between 3.4 and 21.3.

Although the interpretation is straightforward in this example, problems arise when the confidence interval includes zero, which is not a possible value for the NNT. Because the difference in the proportions may be quite small, this should result in a large NNT, which is clearly not the case. In this situation the confidence interval is not the set of values between the limits but the values outside of the limits [6]. For example, if the confidence limits were calculated as -15 to +3, then the confidence interval would be the values from -8 to -15 and 3 to +8.

Limitations

The use of the term 'attributable risk' is not consistent. The definition used in this review is the one given in the cited references, but care must be taken in interpreting published results because alternative definitions might have been used.

Care should be taken in the interpretation of an OR. It may not be appropriate to regard it as approximating to a RR. Consideration needs to be given to the type of study carried out and the incidence of the disease.

Conclusion

RR and OR can be used to assess the association between a risk factor and a disease, or between a treatment and its success. Attributable risk measures the impact of exposure to a risk factor. ARR and NNT provide methods of measuring the success of a treatment.