Interpreting conflicting evidence

A podcast on sorting through a whole bunch of conflicting data

For the third installment of the Beyond Journal Club podcast series, we tackled a topic that, at first glance, may seem uninteresting to anyone who doesn’t treat a lot of patients with pneumonia: do steroids help in the treatment of community acquired pneumonia?

But that’s only true if you’re hyper-focused on the granular question that the trial we discussed tackled on steroids in pneumonia treatment.

The meta-question that we tried to handle is a bit more interesting, and certainly more widely applicable: what do you make of a set of conflicting trials in the same area of medicine?

Sometimes the evidence tells a consistent story, and sometimes it doesn’t

There are some places in medicine where the more trials you do, the more clear the story becomes: statins, SGLT2 inhibitors, GLP-1 agonists, antihypertensive therapy.

These situations are ones where accumulating data makes you more confident in the applicability of a medication or treatment strategy.

That stuff makes it easy as a doctor - you’re probably right when you just follow the pack and do what the trials say.

But then there are situations where the evidence is a bit murkier and some evidence suggests a benefit and some evidence suggests no benefit or harm.

The story of steroids in pneumonia falls into the second group.

The theory behind why steroids might help in pneumonia is simple - an infection in the lungs that creates inflammation may make it hard to ventilate and a drug like a steroid that reduces inflammation may make things better.

But when you look at the history of the clinical trials in this realm, you see a pretty inconsistent story:

Some treatments strategies are about progression, and others are about triangulating

When the evidence in an area is really consistent, it means that there’s widespread benefit across a lot of different populations.

But steroids in pneumonia is way murkier - some patients seem to benefit and some patients seem to be harmed.

When you encounter a situation like this, where the data are mixed, it probably means that there’s a subgroup that benefits and that some trials are just better at capturing that group than others.

The story of the CAPECOD trial is about doing a bit better of a job in figuring out which patients are the ones that benefit from an anti-inflammatory treatment when they have pneumonia.

It’s a story of triangulation.

The CAPECOD trial builds on the RECOVERY trial, which was a really big deal in the treatment of severe COVID pneumonia, in the effort to find the right subgroup of patients with a lung infection who are sick because of the body’s inflammation rather than the outside invading organism.

It’s the type of story that teaches us about interpreting conflicting evidence, even if you aren’t all that interested in how to treat pneumonia.

Listen to our discussion of steroid treatment in pneumonia and the CAPECOD trial here: