COMMENTARY

# Understanding the Effect Size of ADHD Medications: Implications for Clinical Care

Stephen V. Faraone, PhD

Disclosures

September 18, 2003

### Clinical Benefit

Another approach to interpreting the SMD is to translate it into a concept that is clinically meaningful, such as the probability that a treated patient will show a level of improvement that exceeds that of a randomly selected placebo patient. Computing this probability of benefit is fairly complex and will not be discussed here beyond noting that it equivalent to the area under the receiver operating characteristic curve. For details, see Faraone et al.[4]

Figure 1 shows the relationship between the SMD and the probability of benefit and also includes the average effect sizes for nonstimulant, immediate-release stimulant, and long-acting stimulant medications computed from available clinical trials. The plot includes the range of SMDs found in ADHD clinical trials.[5] It shows that when the SMD equals zero, the probability that a drug outperforms placebo is 0.5, which is no better than the flip of a coin. When the SMD equals one, the probability that a drug outperforms placebo is .76. This is actually fairly good for a medicine used in the mental health field. For ADHD medicines, the probability for a drug to outperform a placebo is 0.67 for nonstimulants, 0.74 for immediate-release stimulants, and 0.75 for long-acting stimulants.

The standardized mean difference predicts the probability that a drug will lead to a better outcome than placebo for a random patient.

Another estimate of effect size derived from the SMD is the odds that a drug is better than placebo. This is computed as follows:

 Odds Drug Better Than Placebo = Probability Drug Better Than Placebo Probability Placebo Better Than Drug

Therefore, an odds of 2 to 1 means that patients are twice as likely to do better with a drug than without. Figure 2 shows the relationship between the SMD and the odds that a drug outperforms placebo. When the SMD equals zero, the odds that a drug outperforms placebo are 1 to 1, which means that the patient is no better off taking the drug than not. When the SMD equals one, the odds are 3.2 to 1. For ADHD medicines, the odds that a drug outperforms placebo are 2 to 1 for nonstimulants and 3 to 1 for immediate-release and long-acting stimulants.

The standardized mean difference predicts the odds that a drug will lead to a better outcome than placebo for a random patient.

There is one serious limitation to the SMD and the related quantities discussed above. These measures provide an excellent method of computing a standard measure of effect across different studies. But they do not directly provide information about the degree of improvement. The relative percent improvement provides a simple approach to this issue and is easily computed as the ratio of percent improvement in the drug group divided by percent improvement in the placebo group.

 Relative Percent Improvement = Percent Improved on Drug Percent Improved on Placebo

The relative percent improvement has an easy interpretation: it is the number of times more likely a patient is to improve on a drug compared with placebo. So a relative percent improvement of 10 would mean that treated patients show 10 times the level of improvement as untreated patients. We can translate the SMD effect size into a relative percent improvement statistic, but to do so, we need to choose a definition of improvement. One sensible choice is to define improvement as having an outcome that is better than 90% of untreated patients. For the ADHD drug classes presented in Figure 1 and Figure 2, the relative percent improvement statistics are 2.5 for nonstimulants, 3.6 for immediate-release stimulants, and 3.7 for long-acting stimulants.

Like the relative percent improvement, the odds ratio also relies on percent improvement scores. As its name indicates, the odd ratio is computed as the ratio of 2 odds: the odds of improvement on drug and the odds of improvement on placebo. The formula is:

 Odds Ratio = Odds of Improvement on Drug Odds of Improvement on Placebo

This statistics tell us the increase in the odds for improvement that can be attributed to the treatment. For example, for long-acting stimulants, the odds ratio is 5.3, which indicates that treated patients are 5 times more likely to improve than untreated patients. For immediate-release stimulants and nonstimulants, the odds ratios are 5 and 3.1, respectively.