Read on Twitter
A new systematic review showing the superiority of motor control exercises for low back pain has just come out, and I thought I would highlight some aspects of this review that made me a bit sceptical of their (overplayed) conclusion. https://bit.ly/2GXOTHz
The first bit relates to the eligibility criteria. The "many trials do shitty interventions therefore they shouldn't be included in reviews" argument always comes up. While I agree there are lots of trials with problematic interventions, I disagree with the proposed solution
Mostly because a lot of times the criteria to separate the good from the bad is unclear/highly subjective. This review is a good example of that. There are lots of subjective decisions here that are difficult for readers to understand and this is not good. 
In reviews that aim to answer a clinical question based on a restricted sample of clinical trials (like this one), it is (even more) fundamental that authors are 100% transparent about their decisions. I would like to know which trials they excluded and their reasons for doing so
This is important here as they included ~5% of all potentially eligible trials (28/524). For me, more importantly then knowing the characteristics of the included trials, I want to know more about the excluded trials and their characteristics. A PRISMA flow diagram isn't enough
This is actually a requirement in the AMSTAR-2 tool, a systematic review appraisal tool - see item 7 https://amstar.ca/Amstar_Checklist.php
In fact, this is one of the critical domains in the tool - and sadly it's often ignored in non-Cochrane reviews.
In fact, this is one of the critical domains in the tool - and sadly it's often ignored in non-Cochrane reviews.
So take home #1 here: We know nothing about 90% of trials that were excluded and this is red flag #1
The second bit I want to address is their results. There are ***SIXTY*** forest plots in supplementary file (if I counted right). Do you know how many pooled estimates were *statistically significant* in favour of the intervention? 22! 22/60, or 37%.
As they did not list a primary outcome in their review, I think it's safe to say that it's a bit of a stretch to conclude that there is "significant evidence supporting the effectiveness of SMA for the treatment of LBP". There is not! And that's red flag #2
Among the small % of outcomes with a "+ve finding", many of them may not be clinically important. You know that by looking at the effect estimate as a whole, which include the point estimate AND the 95%CIs. There are 8/22 *+ve* estimates whose 95%CI is very close to 0.
Remember this: The point estimate is *NOT* the effect. And if you think I'm speaking Portuguese here, you gotta read this https://bit.ly/31ggZof
To be fair, they did disclose in the discussion that there were 'a number of outcomes' that weren't significant. However this seems a little bit downplayed, doesn't it? Surely "a number of outcomes" does not make you think that 63% of outcomes weren't sig.
Another issue that deserves attention is this claim in the limitations section. That could be true, but their lack of transparency creates a conundrum for the reader: Did this review do a good job at selecting a homogenous sample of trials or did they just cherry pick trials?
Finally, these are their conclusions. My take is: even with what they considered to be the most appropriate subset of trials (which we can't know), their results are disappointing at best and don't seem to justify the enthusiasm in the conclusions.
Final take home: Be sceptical about reviews that make bold claims about treatment effects, especially when previous reviews have found exactly the opposite. It's October 2020 and I thought we were past the motor control vs other treatments discussion. Sadly it seems we aren't.
