Would've Could've

The 30 Second Skinny Mrs Xiao-Ping Zhai runs a fertility clinic based on Traditional Chinese Medicine. Some time ago The Beacon complained to the ASA that she was using the name Doctor in her advertising despite not being medically qualified, and that she was making outlandish claims for the success of her treatment. Mrs Zhai failed to be able to provide evidence for these claims and so agreed not to make them in advertising in future. In explaining away science's failure to find a positive result for TCM and fertility to the BBC, she betrays some clouded logic that even serves to undermine her own claims.

Here's a simple point that is worth remembering. If robust clinical trials have systematically failed to show a positive result for a particular treatment, then there is no solid evidence for that particular treatment. That's so obvious it's practically a tautology, which is why it angers me when practitioners make claims like the following:

One high-profile practitioner, Dr Xiao-Ping Zhai[sic], of The Zhai Fertility Treatment Clinic, said there were indeed problems with the way in which these trials were designed and that different analysis would show the benefits.


"Certainly for those with unexplained fertility problems in particular, we know acupuncture and traditional Chinese medicine can be beneficial. What matters is both the expertise and experience of the practitioner, but most of all the treatment of the patient as an individual. It is the tailored treatment which is key.

"We need clinical trials that take this into account."

This was Mrs Zhai's answer, given in a BBC report, to the recent findings published by the British Fertility Society. They analysed 14 acupuncture trials, covering 2,670 individuals undergoing IVF with attendant acupuncture and found that there was no benefit to be had in going under the needle. They had hoped to look at Traditional Chinese Herbal Medicine, but none of the existing trials met with their search criteria; i.e. none of the TCHM trials were of sufficient quality and rigour to be considered for analysis.

Mrs Zhai provides two canards here. Firstly is the idea that research is fundementally there to establish "what we already know". If they fail to confirm our beliefs then the trial is at fault, not the belief. Rather than take the hit and state that, yes, in the circumstances set out in the various trials, acupuncture was showed to have no effect, she states that we know "certainly" that TCM can be of benefit, shifts the blame of the failure onto individual practitioners, and raises canard number two, that treatment of the patient is individual, and so trials looking at the same treatment administered to a group of people can't shed any light on them.

That, potentially, could be the case, but it raises a problem. Zhai discusses people "with unexplained fertility problems". In other words, people who can't get pregnant for no readily apparent reason. Let's first look at how we decide whether or not a woman is infertile. We decide based on them not getting pregnant, but after how long? Figures vary. In the book Slow, by Carl Honore it was suggested that the average was 18 months. The figures are often put on a sliding scale, which makes calculating an overall average, with its comforting "am I normal" quality, fairly difficult and ultimately misleading. The NHS website gives the following: "Most couples (about 84 out of every 100) will get pregnant within a year if they have regular sex and don’t use contraception. About 92 out of 100 couples trying to get pregnant will do so within two years."

On the NHS couples must wait 3 years before being place on an IVF treatment, but the majority decide to start earlier and go private. A Daily Mail article (I know, I know) covered research by Dr Geeta Nargund, that seemed to show there was little point in starting treatment after only a year, though this was based on a computer simulation of 100,000 women, rather than historical data. Nargund's model suggested there was a small increase in the possibilities of conception through IVF after one year of trying naturally, but this small increase came at a cost, the dangers attendant to multiple births, for instance. The point here is that it seems likely that couples seek out fertility treatment privately before they really need to.

I suspect that Zhai's view of reality is this. She privately treats people who have not been trying to conceive for very long. These are the inexplicably infertile patients Zhai refers to; couples who are fertile, but impatient. She then treats them with TCM, but those that conceive were going to conceive any way. It is this misconception (ho ho) that clinical trials would seek to rectify, and without clinical trials backing up Zhai's claims, she cannot say that properly designed trials "would" show a benefit, only that they "might". That is the nature of clinical evidence, after all.

Lastly, if the treatment is tailored to the individual, then on what grounds? These are the mysteriously infertile, so how is it that this mystery is unfolded? Or when Zhai refers to individualisation, are we really talking about trying everything they can think of until conception occurs? Are we talking about the "throwing shit at a wall" approach so favoured by many CAM users?

Zhai's clinic was the subject of a recent complaint to the ASA. I know, because I was the one that made it. For some time the clinic, a satellite to the London Fertility Clinic, had been making fairly strong claims about their success rates. The ASA, after a little persuasion, took up the complaint, and it got as far as the draft adjudication which is presented to the ASA council for final signing off. It is fairly rare, but not impossible, for council to disagree with the adjudication. Here, the recommendation was that the complaint be upheld. This was down to the fact that the evidence provided by the Clinic did not offer any detail as to how it was compiled. They also upheld the part of the complaint addressing the fact that the clinic gave Zhai the title of Doctor, and (all together now) although doctor is remarkably not a protected term, it must not be used in a context that suggests that someone is a qualified medical doctor if they are not. In other words, don't call yourself a doctor in an ad for a fertility clinic if you're not a doctor.

 

Once the draft was published and distributed, it might have been in the clinic's powers to provide the missing information, place their evidence into context, and overturn the draft decision. However, they decided to opt for "resolution". This meant, in effect, that they asked the ASA if, should the clinic remove the advert, the complaint would be resolved, not go to council, and most important of all, not get published in detail on the ASA website. It could be that they just didn't have time to sort out the relevant information; it could be that there was a gaping flaw in the way their data had been compiled; it could be that the evidence, not flattering to begin with, would have been even less flattering once placed in a proper context; I don't know, because it never happened. All I am left with are a number of "mights" and my own preconceived notions.

NB I still have a copy of the Zhai Clinic Draft Adjudication. I'm not sure if I'm allowed to do anything with it, however. I may publish the clinic's defense later tonight.