A Battle Worth Fighting: a
Comment on The Vindication of
Magnitude-Based Inference Martin Buchheit Sportscience 22,
sportsci.org/2018/CommentsOnMBI/mb.htm, 2018 Summary: MBI has changed the research and practice
of thousands of sport scientists, but it is now under undeserved attack by defenders
of p values. I trust the analytical foundations of MBI, and I consider it
irreplaceable for research with samples and for monitoring individual
athletes. I encourage all sport and exercise scientists to fight for MBI.
Start by retweeting #supportMBI. This comment is
co-published in Sport Performance & Science Reports. I am not a qualified
statistician, but after 20
years of sport research and applied
field work with elite athletes (Buchheit, 2017a), I consider myself an experienced user of
statistics. Applying statistics to athletes' data was not easy when I started
(Buchheit, 2016): effects in the small sample sizes encountered by
coaches and sport scientists were almost always statistically
non-significant, and nothing practical existed for assessing changes in
individual athletes probabilistically. Discovering magnitude-based inference
(MBI) and using it on every type of dataset–from individual athletes
especially–was more than a game changer. As I have repeatedly said, MBI changed my life, and I am forever indebted to its progenitors, Will
Hopkins and Alan Batterham (Batterham and Hopkins, 2006). It is within this context that I want to
contribute to the current debate over MBI. It seems to me that
the people already skeptical of MBI are using the debate to reject the method
before the debate is over. Following the recent critique (Sainani, 2018), the editor-in-chief of Medicine in Sports and
Exercise (MSSE) has even instructed his associate editors to reject
manuscripts using MBI, and his decision is to be enshrined in editorial
policy. This astonishing decision was made without waiting for the rebuttal
letter from Will and Alan and in spite of what seems to me to be convincing evidence
that the claims in the critique are all either misguided or wrong (Hopkins and Batterham, 2018). The debate has now shifted toward a battle of
position or power and away from an objective discussion about advantages and
disadvantages of the different methods, as good science requires. Science is
supposed to be self-correcting, but I am rather afraid to see the overall
scientific community going backwards with these close-minded attitudes (Buchheit, 2017b; Buchheit, 2017c). Like the protagonists in the never-ending battles
of religion, researchers are now adopting extreme positions (McGuigan, 2016)–statistical methods included–and the very recent
trend here seems to be that MBI should be scrapped!? This is too sad and bad
for me not to react. As an early adopter
of MBI (Buchheit, 2016), I can testify to the challenge of getting it
accepted by our peers in most journals. It seemed to me that there was a lack
of understanding and an unwillingness of editors and reviewers to see outside
their box (Buchheit, 2017b). The dozens of reviewers’ comments that I compiled attest to those difficulties. Colleagues who wanted
to reduce the risk of rejection would sometimes put MBI and null-hypothesis
significance testing (NHST) in the same paper, a strategy that often resulted
in contradictory conclusions within the same paragraph. I am proud to say
that sometime around 2010 I decided never again to test the null-hypothesis and report
p values in my research. I have even taken my name off papers, when
co-authors insisted on including p values. The point that I want
to make here is that NHST is not a viable alternative to MBI for sport
science, because it systematically misses two important aspects in our
research and service work: · Consideration of the magnitude for the effect of
interest. "Are the changes in performance of my athletes worthwhile (for
example, after a novel training block)?" Using NHST with small samples,
changes can be non-significant but substantial in magnitude–in other words, a
potentially useful intervention can be interpreted as worthless. A rare
occurrence in sport science is the equally perverse conclusion: with large
samples, changes can be significant but trivial in magnitude–in other words,
a useless intervention can be interpreted as worthwhile. With MBI, the focus
is first on understanding if the magnitude is relevant in relation to the so
called smallest worthwhile change (Buchheit, 2018a; Buchheit, 2018b), and then on the probabilities for the magnitude to
be worthwhile, worthless, and harmful. Irrespective of the claim about error
rates with MBI, using NHST alone will never help to understand magnitudes of
an effect (Buchheit, 2016; Cohen, 1990). ·
Consideration
of individual responses. "Has this player improved on that test?" I
have written extensively on the subject, but this is clearly the aspect where
MBI has been the most influential for me. Every day, sport scientists need to
make inferences on potential changes in various measures (e.g., locomotor
performance, body composition, strength tests) to help the performance
managers and coach make decisions (Buchheit, 2017a; Lacome et al., 2018). Before MBI, practitioners were left alone to
decide on which changes were important or not, leading to very subjective
decisions. However, in elite sports, where every decision can have large
consequences, bringing objectivity is key to reduce error and in turn,
improve precision. The overall concept of using both the typical of error of
measurement and a well-defined smallest worthwhile change to assess
likelihood of a substantial change within an individual athlete (Hopkins, 2004; Hopkins, 2017) is now close to becoming the norm, thankfully (Buchheit, 2017a). In fact, MBI has made individual monitoring one of
the more interesting activities in sport science (Buchheit, 2014; Lacome et al., 2018). This analytical aspect could be examined with
NHST, but there would be the same problems as with samples: non-significant
substantial changes with simple change scores, and significant trivial
changes with lots of repeated measurement on the individual athlete. In conclusion, MBI
has changed the practices of thousands of sport scientists in the academic
world and in the field. There is nothing to replace MBI when monitoring
athletes, and I trust the analytical foundations of MBI for research with
samples. My trust is based on the following: Alan and Will are amongst the most highly cited
researchers in exercise and sport, their knowledge of the inference
literature is clearly beyond reproach, and their logic is impeccable. So I will keep using MBI for my daily work with
athletes (Buchheit, 2017a; Lacome et al., 2018) and for publishing research. If I cannot submit the
research to MSSE, no problem, provided we still have good journals with
reviewers and editors who understand the value of MBI. We must fight and win
this battle. Start by retweeting #supportMBI. Batterham AM, Hopkins WG (2006). Making meaningful
inferences about magnitudes. International Journal of Sports Physiology and
Performance 1, 50-57 Buchheit M (2014). Monitoring training status with
heart-rate measures: do all roads lead to Rome? Frontiers in Physiology 27,
73 Buchheit M (2016). The numbers will love you back in
return-I promise. International Journal of Sports Physiology and Performance
11, 551-554 Buchheit M (2017a). Want to see my report, coach?
Aspetar Sports Medicine Journal, 34-43 http://www.aspetar.com/journal/upload/PDF/2017216161135.pdf Buchheit M (2017b). Outside the box. International
Journal of Sports Physiology and Performance 12, 1001-1002 Buchheit M (2017c). Houston, we still have a
problem. International Journal of Sports Physiology and Performance 12,
1111-1114 Buchheit M (2018a). Magnitudes matter more than
Beetroot Juice. Sport Performance & Science Reports Janv, v1.
https://sportperfsci.com/wp-content/uploads/2018/01/SPSR15_Buchheit-M._180115_final-1.pdf Buchheit M (2018b). Trivial effects are clearly
important. Sport Performance & Science Reports Janv, v1.
https://sportperfsci.com/wp-content/uploads/2018/01/SPSR14_Buchheit-M._171224_final.pdf Cohen J (1990). Things I have learned (so far).
American Psychologist 45, 1304-1312 Hopkins WG (2004). How to interpret changes in an
athletic performance test. Sportscience 8, 1-7 Hopkins WG (2017). A spreadsheet for monitoring an
individual's changes and trend. Sportscience 21, 5-9 Hopkins WG, Batterham AM (2018). The vindication of
magnitude-based inference. Sportscience 22, 19-27 Lacome M, Simpson BM, Buchheit M (2018). Monitoring
training status with player-tracking technology. Still on the road to Rome.
Aspetar Sports Medicine Journal, 54-66
https://mart1buch.files.wordpress.com/2018/06/lacome-still-on-the-road-to-rome.pdf McGuigan MM (2016). Extreme positions in sport
science and the importance of context: it depends? International Journal of Sports
Physiology and Performance 11, 841 Sainani KL (2018). The problem with
"magnitude-based inference". Medicine and Science in Sports and Exercise (in press) doi:
10.1249/MSS.0000000000001645 Back to index of comments. Back to The Vindication of Magnitude-Based Inference. Published July 2018. |
