Commentary
on A Decision Tree for Controlled Trials Greg Atkinson Sportscience 9, 40-41
(sportsci.org/jour/05/ga.htm) |
|
To aid the communication of the various types
of experimental design, Alan and Will used a notation system presented in
their Figure 1. Such schematics have been attempted before, but I think the
notation used by Alan and Will has the great advantage that the arrows show
exactly which time-point is compared to other time-points for generation of
the change or difference scores. This favourable aspect of the notation
system communicates the precise link between experimental design and analysis
of data. Alan and Will included the time-series or
quasi-experimental design in their paper at my suggestion. Researchers might
wonder why such a design would be adopted at all, given its obvious lack of a
control group. One example might be situations in which time itself is the
intervention. Such is the case in studies on circadian variation in
performance. Any readers who have tried to research elite athletes might also
find it difficult or maybe even unethical to include a control group. In the
future, Alan and Will might like to extend their statistical expertise to
this situation in particular, since the analysis of time-series data might involve
complicated covariate analysis (to control for intervening variables also
changing over time) of correlated data-sets. In the paper, two important issues were also
mentioned. First, even in a fully controlled trial, it was pointed out that
there may be reactive effects due to the participants knowing they have been
allocated to either the treatment or control group. Another so-called threat
to validity in a controlled trial is the potential for change in
participants' behaviour if they receive feedback about their pre-treatment
scores before the treatment or post-test. Even performing the pre-treatment
test can in principle affect the control and experimental treatments
differently. Although any
physiological responses to exercise might not be due to such reactive
effects, this threat to experimental validity might influence an outcome
measure of human performance. One design that is supposed to estimate
reactive effects due to the pre-treatment measurements is called the Solomon
4-group. Using the new notation, the design is as follows:
The Solomon 4-group is a complicated design
and demands a large sample size (due to the inclusion of four groups).
Nevertheless, I have seen it employed in some large scale studies on physical
activity interventions, for example. Secondly, the important issue of lack of retention
of research participants (often called subject mortality or attrition) was
mentioned by Alan and Will. If a treatment has been so badly received by
participants that they decide to vote with their feet, a researcher can
hardly label the treatment a success, even if the data analysed on the
remaining "selected sample" suggests that this is so! The CONSORT statement cited by Alan and
Will deals with this important issue by advising researchers to distinguish
between two types of analysis: intention-to-treat,
where you include all participants in the analysis, regardless of how
well they complied with the treatment, and as-treated, where you include only those who did everything
properly (Altman et al., 2001). A good
reference on the Web is the intention-to-treat page at Gerard Dallal's statistics
site. Intention-to-treat
analyses are an issue where the outcome is mortality or morbidity that can be
quantified without a post-test, but exercise and sport-science analyses are
mainly as-treated, because participants have to get through the treatments
and perform the post-test before they can be included. Regardless, it is important to document what happens to all the participants,
and to justify the approach you have taken in the analysis. Back to article/homepage
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang L (2001). The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Annals of Internal Medicine 134, 663-694 Published Dec 2005 |
