Image via WikipediaThe 30-Second Skinny There's little better evidence for an intervention than the double blind randomized placebo controlled trial, but these trials still need to be properly scrutinized. In one double blind trial on acupuncture as a treatment for smoking cessation, the drop-outs were distributed across the three groups in a way that suggested that the blinding simply hadn't worked, making the resulting evidence all but worthless.
The holy grail of proof for medical interventions is the large scale double bind randomized placebo controlled trial. However, a positive result from such a trial can't be treated as a watershed of proof for any particular intervention. More importantly, a positive result should invite a greater scrutiny, especially if the result suggests a massive shift in our understand of the way that bodies work.
I recently encountered such a trial in the form of Low level laser for the stimulation of acupoints for smoking cessation: a double blind, placebo controlled randomised trial and semi structured interviews by Catherine M. Kerr, Paul B. Lowe and Neil I. Spielholz. It is a belief of acupuncture and acupressure that chi points on the ear trigger endorphin releases that can benefit someone in their attempts to give up smoking. Put simply it is said to bypass cigarette cravings by giving them another kind of hit.
Kerr and co decided to test this hypothesis using laser acupuncture. They gathered 387 volunteers together, each regular smokers who wanted to quit. They were randomized into three groups. Group A received three genuine treatments and one sham treatment. Group B received four genuine treatments. Group C received four sham treatments. This randomization was carried out in such a way that neither researchers or subjects knew which group they had been assigned to. The sham treatment involved using a laser unit that was identical in appearance and weight to the genuine handsets, and application of the unit was also identical in experience. After the four treatment sessions there was a 3 month and a 6 month follow up to establish whether or not subjects were smoking. The results seem impressive and unequivocal.
Not only does the 4 treatment group outperform the control group by some distance, the 3 treatment / 1 sham group outperforms the control group to a lesser extent. So we have a decent sized sample, a control group, proper blinding and randomization, and a demonstration that the number of treatments have an incremental effect. Is it time to allow acupuncture into the fold of proven medicine; it is time to work towards establishing the nature and make-up of chi?
No. And here's why.
I've kept a little piece of information back. The trial had 47 drop-outs of the 387 volunteers. Here's how the paper puts it:
Three hundred and eighty seven persons were initially accepted on to the study. At the start of the treatment time 6% (n=23) of persons did not attend for the treatment which had been scheduled after the initial screening and acceptance. A further 7% (n=24) withdrew after receiving either one or two treatments. The remaining 87% (n=340) went on to complete their respective treatment schedule. The number of participants in each group was now Group A (3 laser and 1 sham treatments) 121, Group B (4 laser treatments) 130, Group C (4 sham treatments) 89.
This means that if the 387 had been randomized equally, each group would have had 129 participants in each. Therefore we see that the drop outs came mostly from the control group, with a few dropping out of group A. Here's how the paper discusses this odd drop-out distribution.
However the largest number of persons failing to complete the treatment (n = 38) occurred from within the control Group C, reducing its participant number to 89. This was a considerable reduction in comparison to the remaining participant numbers in Group A (n = 117) and Group B (n = 125) and there was a concern that this could be having the pseudo-effect of enhancing the effectiveness of the laser acupoint stimulation treatment. Steiner and Geddes (2001) suggest that one way of dealing with missing data is to assume the worst case scenario and accept that the significant outcome that was achieved was the result of so many persons leaving the control group. Applied to this study, their suggestion would be to record all the dropouts in Control Group C as having been successful in ceasing smoking.
So in the minds of the researchers the worst case scenario was that everyone who dropped out of the study did so because they had given up smoking. Their concern, and perhaps this is understandable given that they had invested greatly in an eighteen-month study, was how the drop-outs may have affected the data. What their concern ought to have been was the distribution itself. We'd expect to see, under normal circumstances, a roughly equal number of drop-outs in each group, about 16. What we have instead is a distribution of drop-outs that matches the level of sham treatments each group gets. If you had to imagine the distribution of drop-outs in a non-blinded trial, chances are you'd give them a similar distribution. People are going to be more inclined to drop out of a study if they know they are receiving sham treatment; this is even more likely if they understand that the only point of having the sham treatment is if they don't know.
The worst case scenario, then, is not that the drop-outs all went on to quit smoking, skewing the data. The worst, and here most likely, case scenario is that the blinding had failed across all three groups, rendering the study as a whole useless. The researchers could, perhaps, be forgiven for not realising this, but any peer-reviewed journal worth its salt should have picked up on this flaw. The evidence says more about the editorial quality of the Journal of Chinese Medicine than it does about the ability of laser stimulation of the ear to aid in people kicking the habit.
No comments:
Post a Comment
Please keep comments on topic, and be respectful of one another.