Comment on A
Spreadsheet for Analysis of Straightforward Controlled Trials Alan M Batterham Sportscience 7,
sportsci.org/jour/03/amb.htm, 2003 (598 words) |
|
The accompanying article and spreadsheets represent an excellent learning, teaching, and research tool. The pertinent features from other spreadsheets on the newstats.org site are combined to good effect in the context of the analysis of randomized controlled trials and crossovers. The inserted comments feature in the spreadsheets is very helpful for students and researchers familiarizing themselves with the techniques, serving as a quick desktop reference without having to constantly go back to the associated article or other literature. All features of the spreadsheet are highly relevant to the proper analysis of randomized controlled trials. I particularly like the features relating to the analysis of transformed variables, plots of change scores, individual responses, comparison of pre-test values, and likelihoods for specified minimum clinically important differences. These issues are often neglected in the analysis of trials and the appropriate presentation of results. In particular… • Screening of data to determine whether a transformation is appropriate (and what type) is essential. The spreadsheet facilitates the acquisition and application of this essential knowledge and skill. This point relates also to the simple plots to check assumptions of homoscedasticity. These sections of the accompanying article introduce the reader gently to concepts of non-uniformity of error, before referring directly to homo- and heteroscedasticity–terms that often strike fear into the hearts of students and some more experienced researchers alike. • The provision of an estimate of the SD of individual responses to treatment is a major step forward in providing an accessible, user-friendly tool. Trialists have long recognized that the ‘average’ response to treatment does not apply to equally to all those receiving the treatment. Some people respond well to treatment, some don’t change, and some may get worse. The quantification of this heterogeneity of treatment response illuminates the analysis and interpretation of the trial. • The comparison of pre-test values is a very useful and oft-neglected addition. In a large-sample randomized controlled trial, theoretically the groups should be equivalent for the outcome (and other unmeasured) variables at baseline. However, by chance in small sample studies or in quasi-random designs, the groups may not be equivalent. I agree that the confidence interval for the treatment-control difference at baseline is largely irrelevant. The simple Cohen effect size for the difference, linked to the minimum clinically important difference, is probably the best way to judge whether there is a substantial difference at baseline that would be deemed problematic. • Perhaps the most important section is the chances that the effect is clinically, practically or mechanistically important. This is an issue that receives insufficient attention in many trials, both a priori in determining appropriate sample sizes and following the data analysis in interpreting the observed effects. Rather, all too often the focus is on statistical significance at some arbitrary P value. The percent likelihoods and associated qualitative labels provide a welcome antidote to this dubious process. I have checked the spreadsheet and can find no obvious errors. I used a real data set from a publication in Medicine and Science in Sports and Exercise a few years ago on the effect of exercise training on lipid-lipoprotein profiles in children. It all checks out and, with the additional features, shows how valuable the output is in interpreting and presenting the results.
|