Health Through Knowledge

The Serano Group


Saturday, December 13, 2008

Seeing Nothing

Lyme disease research is manipulated. Take a look.

Two controlled trials of antibiotic treatment in patients with persistent symptoms and a history of Lyme disease, New England Journal of Medicine, 2001. 345(2):85-92.

Klempner, M. S., Hu, L. T., Evans, J., Schmid, C. H., Johnson, G. M., Trevino, R. P., Norton, D., Levy, L., Wall, D., McCall, J., Kosinski, M., Weinstein, A.  

The NIH-funded Klempner study is often cited as important to our understanding of treatment for Lyme disease. At The Serano Group, we find the study interesting mostly because it shows how easy it is to manipulate research and how readily scientists, journalists, and the public accept this type of bias uncritically. (You can read our detailed analysis of the study here, but you may want to continue reading for background.)

While the study has been discussed often since its 2001 publication, its primary failure as a piece of research is rarely mentioned: the study measured hardly anything post-treatment. The Klempner study tested a single treatment for patients with persisting Lyme disease and was unable to discern anything. The only question that remains is how hard did they try?

Typically, research not demonstrating anything is hard to get published. But due to political and commercial pressures to promote a distorted Lyme disease agenda, this study was rushed to press and continues to be cited as important evidence that persisting Lyme borreliosis does not exist and should not be treated.

Of course, this violates a basic tenet of research. Science just does not work this way. You can do an experiment that indicates something exists because the thing studied seemed to cause an effect. But, you can’t claim something does not exist because you did an experiment and did not demonstrate anything. To have an indication, let alone evidence that something does not exist requires many experiments trying every possibility of showing the thing exists using the most sensitive available measurements.

There are two reasonable possibilities for the Klempner study outcome: either the treatment tested does not do anything or Klempner did not have methodology sufficient to measure the difference. A third possibility is that he did find differences and failed to disclose them.

The Serano Group uses the Klempner study as a litmus test. Any individual or organization promoting it or citing it as evidence is suspect for bias. The study shows so little and tries so hard to see nothing that only extremists, academic sycophants, and the uninformed would find it meaningful.

To summarize: 64 subjects with symptoms indicating persisting Lyme disease were treated with two grams intravenous antibiotic ceftriaxone (Rocephin) daily for 30 days, followed by 200 mg daily oral doxycycline for 60 days. Another 65 subjects received a non-drug treatment looking like the real antibiotics, a placebo. Three months after completing treatment, subjects completed a check-the-box self-assessment of their general health, the standard SF-36 questionnaire. The SF-36 generates eight scores, 0-100, which Klempner does not report or analyze. Instead, he reduces the eight scores to two summary scores of physical and mental status. These scores were not reported either, but simply rated “Improved”, “Worse”, or “Unchanged”. Once each subject’s score was reduced to this single rating for mental and physical health, statistical analysis was performed. The analysis found the treated group did not appear significantly different than the placebo group. (The study also stated subjects filled out a Fibromyalgia Impact Questionnaire, but did not report results.)

That, despite all the fine print, is all the Klempner study shows for evidence. The same groups that say only objective data should be used for any Lyme disease decisions suddenly based the conclusions of their multi-million-dollar study on a subjective patient self-assessment of their medical condition.

There were many other areas of the study where the cards were stacked. Researchers specifically selected subjects who had already failed at least one treatment similar to the one being tested. The tested treatment was light considering how sick the subjects were and then only 75% compliance was required. A negative change in mental score trumped a positive physical score change and vice versa. The limited number of 64 treated subjects was highly likely to miss anything less than a dramatic treatment effect. Many medical studies have thousands, if not tens of thousands subjects.

The study did present detailed data about objective measurements done before treatment, but if similar post-treatment measurements were made, they were not disclosed. If the subjects were evaluated by a nurse or doctor, this was not mentioned. If they were asked if their joints still hurt, if they still had headaches, or whether they could sleep at night, this was not disclosed. Instead, subjects filled out the general health questionnaire which specifically is NOT to be used to assess a medical condition. The SF-36 designers state never to use summary scores without disclosing the eight 0-100 scores. Even the summary scores were not analyzed: t;he subjects were scored the equivalent of a 1, 2, or 3. This is the sum total of post-treatment data Klempner disclosed.

Repeatedly, Lyme disease ideologues say the study showed that just as many symptoms improve on placebo as on treatment, yet no symptoms were measured nor were subjects even asked about them, except in a very general way. (You can look at the SF-36 here.)

This should give us all great pause. To state the obvious: it does not appear Klempner really wanted to see a difference between the treated and untreated. The paper looks like science, but there really is very little there. Why would the IDSA, AMA, NIH, and CDC use this study as a primary indication to not treat patients with persisting borreliosis? Why would thousands of health insurance benefit denials be based on 64 patients?

The Infectious Disease Society of America lets a small number of ideologues, which include Mark S. Klempner, write its Lyme disease guidelines and then uncritically rubber stamps them, citing their 8000 members who are in support. Philip J. Baker, an extremist for the commercial view of Lyme disease, was Lyme disease program officer at the NIH during the Klempner study. Insurance companies continue to cite the Klempner study as justification not paying for any extension of treatment for Lyme disease, even though only only one treatment was poorly evaluated.

In September 2002, Klempner was named an Associate Editor of The New England Journal of Medicine. He then was able to not only perform bad research but be allowed to censor good research.

In October 2003, the National Institutes of Health awarded Boston University Medical Center, where Mark S. Klempner was assistant provost for research, $128-million to build Biosafety Level 4 laboratories for the study of dangerous microbes. Klempner was the principal investigator receiving the grant.

This is not the way our government should do business. And, none of this has anything to do with providing rational health care for anyone.

(Full disclosure: one of the volunteers for The Serano Group, Inc. analyzed the Klempner study in their private consulting business. You can read that analysis here.)

The Serano Group’s updated analysis of the Klempner study is here.

Posted by Joel Spinhirne
(15) Comments | Permalink