Read on Twitter
A new systematic review showing the superiority of motor control exercises for low back pain has just come out, and I thought I would highlight some aspects of this review that made me a bit sceptical of their (overplayed) conclusion. https://bit.ly/2GXOTHz ">https://bit.ly/2GXOTHz&q...
The first bit relates to the eligibility criteria. The "many trials do shitty interventions therefore they shouldn& #39;t be included in reviews" argument always comes up. While I agree there are lots of trials with problematic interventions, I disagree with the proposed solution
Mostly because a lot of times the criteria to separate the good from the bad is unclear/highly subjective. This review is a good example of that. There are lots of subjective decisions here that are difficult for readers to understand and this is not good. 
https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">
" title="Mostly because a lot of times the criteria to separate the good from the bad is unclear/highly subjective. This review is a good example of that. There are lots of subjective decisions here that are difficult for readers to understand and this is not good. https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">" class="img-responsive" style="max-width:100%;"/>
In reviews that aim to answer a clinical question based on a restricted sample of clinical trials (like this one), it is (even more) fundamental that authors are 100% transparent about their decisions. I would like to know which trials they excluded and their reasons for doing so
This is important here as they included ~5% of all potentially eligible trials (28/524). For me, more importantly then knowing the characteristics of the included trials, I want to know more about the excluded trials and their characteristics. A PRISMA flow diagram isn& #39;t enough
This is actually a requirement in the AMSTAR-2 tool, a systematic review appraisal tool - see item 7 https://amstar.ca/Amstar_Checklist.php
In">https://amstar.ca/Amstar_Ch... fact, this is one of the critical domains in the tool - and sadly it& #39;s often ignored in non-Cochrane reviews.
In">https://amstar.ca/Amstar_Ch... fact, this is one of the critical domains in the tool - and sadly it& #39;s often ignored in non-Cochrane reviews.
So take home #1 here: We know nothing about 90% of trials that were excluded and this is red flag #1
The second bit I want to address is their results. There are ***SIXTY*** forest plots in supplementary file (if I counted right). Do you know how many pooled estimates were *statistically significant* in favour of the intervention? 22! 22/60, or 37%.
As they did not list a primary outcome in their review, I think it& #39;s safe to say that it& #39;s a bit of a stretch to conclude that there is "significant evidence supporting the effectiveness of SMA for the treatment of LBP". There is not! And that& #39;s red flag #2
Among the small % of outcomes with a "+ve finding", many of them may not be clinically important. You know that by looking at the effect estimate as a whole, which include the point estimate AND the 95%CIs. There are 8/22 *+ve* estimates whose 95%CI is very close to 0.
Remember this: The point estimate is *NOT* the effect. And if you think I& #39;m speaking Portuguese here, you gotta read this https://bit.ly/31ggZof ">https://bit.ly/31ggZof&q...
To be fair, they did disclose in the discussion that there were & #39;a number of outcomes& #39; that weren& #39;t significant. However this seems a little bit downplayed, doesn& #39;t it? Surely "a number of outcomes" does not make you think that 63% of outcomes weren& #39;t sig.
Another issue that deserves attention is this claim in the limitations section. That could be true, but their lack of transparency creates a conundrum for the reader: Did this review do a good job at selecting a homogenous sample of trials or did they just cherry pick trials?
