twittersphere trialists - can I pick your brain about pilot RCT interpretation. Trial community appropriately critical of those over interpreting small pilot trials. However, one function of pilot is to see if there is a 'signal' worth further investigating. how to do this? 1/n

seems silly to take to definitive evaluation a pilot trial that mets it feasibility rules (i.e. big trial could be done) but where no hint of intervention potential. lack of clarity in field about how to do this. Anyone know good examples? also, views on following ideas 2/n

with psychotherapy pilot of two active (high intensity) treatments in mind: 1. compute 95% CI of within arm effects. continuation: novel intervention should show large pre post effect size (with lower bound CI not crossing into medium). trial should be powered to detect this 3/n

2. compute 95% CI of between arm effect size differences. plot mean and upper and lower CI, indicating clearly minimum clinically important difference for improvement/deterioration on that measure...... 4/n

2. (cont). continuation: somewhere in 95% CI should fall possibility of novel intervention being greater than control; ideally, mean effect size favouring novel intervention > MCID and lower 95% CI not crossing into favouring control intervention 5/n

write this up cautiously (i.e. as evidence about whether to proceed to definitive trial or not; not as clear evidence of novel intervention efficacy). 6/n

additional questions: 1. impute or not to impute when doing above analyses?. 2: what to do with health economic data if you have it? (i.e. is it at all valid to run boot strapping to visually present likely cost-effectiveness)

would especially value comments from people who have helpfully called out psychotherapy community on over interpreting new therapies e.g. @IoanaA_Cristea and people active in open science/replication movement in psychology. keen to get balance right here!

just to disclose personal interest: trying to work out how to best write up my own pilot RCT of novel therapy v CBT on sample of 80 depressed folks!