Holy sacred cow! Why reactions to the exercise and depression trial go to the heart of scientific controversy.

Drawing by Hilda Bastian.

What do you believe about the effects of exercise and depression ? and why do you believe it? Are you personally unenthusiastic about exercising, or are you closer to religious fervor about it?

These are critical questions. Because it doesn?t matter how much you believe in the importance of science. If you have a very strong prior existing belief, chances are it?s going to exert a strong bias on how you select and react to evidence on the subject.

In the ideal rational world with loads of expertise and time on your hands, that wouldn?t matter when you came across research. If you were interested in the issue, you would carefully assess the biases and strengths of new research, with an equally careful assessment of the existing body of research. You wouldn?t make up your mind about the current state of knowledge till after this systematic assessment was done.

But that?s not what it?s like, is it? In the real world, what we already believe often determines whether we even read something at all. And if it reinforces our belief ? ?Ha! See? I knew it! More proof!? ? we might whizz off an email or a tweet without more than a brief skim of the abstract (or even less).

But if research challenges beliefs we hold dear, we might tear the challenging article to pieces. We tend to look for methodological weaknesses in a way that we don?t do if we agree with conclusions.

This selective skepticism is how we deal with a world of too much information and the confusion it could keep us in. But it can lead us badly astray. It?s one of the ways that sacred cows get to be sacred cows: not looking too closely at welcome news and energetically discounting inconveniently disquieting results. When results inconveniently emerge that challenge orthodoxy, scientific controversy is inevitable.

Exercise and depression is a particularly tough one when it comes to bias. There?s such profound confounding here ? a classic case of ?what comes first, the chicken or the egg?? Does your depression ease because you start exercising ? or can you get out and exercise more because your depression is less crippling? And you can?t blind people to whether or not they?re exercising of course, so that affects self-rating. It?s not surprising then, that high quality systematic reviews on the subject bemoan how little research of minimal bias exists.

What about the ?several meta-analyses? cited by Scicurious in this week?s blog post? Well, one wasn?t a meta-analysis at all: and it wasn?t a systematic review either. Nor did it take into account the biases of the primary research it was considering. The other was a systematic review and meta-analysis ? but it wasn?t of trials directly assessing the impact of exercise in people diagnosed with depression.

Consider the three recent enough, good quality systematic reviews I could find on the direct question (Krogh, Mead, NICE). When raising the bar to include only the most unbiased research, we?re left with a very small body of evidence that suggests maybe a little short-term benefit. No real increase in recovery rates or speed of recovery from clinical depression has been shown. A good quality big trial of long-term sustained exercising could push this either way.

What about Scicurious? critique of the trial that started the reactions? Well, she?s right to say this trial wasn?t seeking to address the efficacy of exercise. But much of the methodology she?s criticizing is good practice in a randomized controlled trial ? like the reliance on standard intention-to-treat analysis (?Not only that, the study did not discount those who did not exercise?). This trial, as Scicurious rightly points out, was meant to test the effectiveness of the encouragement to exercise, not exercise itself. As both the paper and its much-maligned press release made abundantly clear (although the title might have confused the issue).

Is this a perfect trial? No, there?s no such thing. But it?s a very good one. And it?s addressing a critical question, because we need to know what the best advice and support to offer people could be. Truth is, we don?t know how efficacious exercise really is. That matters, partly because anything that can have a positive effect could also have a negative one. If we exercise and feel better we can take credit for our recovery. But if we don?t exercise, will we blame ourselves for still feeling so bad?

fred thompson fred thompson red hook romney tax return the tree of life movie academy award nominees 2012 2012 oscar nominations