A Cautionary Tale of Clouds and Conclusions

Mistakes happen in research. In some fields, however, mistakes are uncomfortably high-profile.

Take climate science. In recent years, climate investigators have watched their field of study become a political minefield. Under these conditions, results become conspiracies and mistakes become disasters that undermine scientific credibility. Amato Evan, an atmospheric scientist who has recently joined the faculty of Scripps Institution of Oceanography, UC San Diego, says he’s learned to avoid climate change blogs in self-defense.

“One time I read a blog where I was referred to as an "enviro-clown" with a PhD in "computer manipulation of weather and climate,” he said. “Actually, that one was so far out that I just had to laugh!”

A new study by Evan and Scripps colleague Joel Norris has revealed that a recent paper heralded by climate change skeptics actually drew conclusions entirely based on a measurement error. In early 2012, scientists at the University of Auckland found that NASA satellite data suggested that global cloud heights have been decreasing over the past decade. Authors Roger Davies and Matthew Molloy suggested that lowering clouds might be a large negative feedback response to global warming, capable of counteracting rising temperatures.

How would this work? The thing to know is that clouds are a poorly understood player in climate models used to predict the Earth’s response to greenhouse gas emissions. Very little is known about how they might respond to global warming, but the impacts of that response could be large.

“Clouds higher up in the atmosphere emit less infrared radiation to space than do clouds lower in the atmosphere,” Evan said. “If clouds were becoming progressively lower in the atmosphere, there would be an increase in infrared radiation that is emitted out to space.”

In theory, this could help counteract the warming effect caused by greenhouse gases trapping extra energy in the atmosphere, and climate change skeptics quickly pounced on the idea. “It’s been a bad week for climate hotheads,” said one Forbes Magazine article. “The sky is falling,” claimed the Christian Science Monitor.

Not everyone was convinced by the findings however. Evan felt there was something not quite right about this result.

“There just wasn't a clear physical basis as to why clouds should be ‘falling’ over time.” he said. “There wasn't a hypothesis as to why cloud heights should be trending downward, which made me suspicious as to the quality of the data used in the study.”

So Evan and Norris decided to look a little more closely. The cloud height data had come from NASA’s satellite Terra, which has been orbiting the Earth since 1999. They found that the variation in the measurements was much higher in the first few years of collected data, and seemed to settle down later on. The trend in variation matched a known synchronization problem with the satellite’s nine cameras, which was more prevalent in the early years of Terra’s mission. It turns out that the camera problem caused low elevation data points to be missed, creating an artificial trend in the data. The low clouds were being overlooked during the early years of the dataset.

When they recalculated the cloud height data, correcting for the camera bias, Evan and Norris found that cloud heights were actually rising over the past decade. They are quick to say, however, that the cloud height record is too short to use as evidence for any sort of long-term change. In addition, clouds have far more properties relevant to atmospheric warming and cooling than simple elevation. Composition, thickness, and location are just a few. What really matters is how those properties change together.

Evan notes that anyone could get their hands on this exact dataset and thousands more, with a few clicks of a mouse. Modern data collection methods are so prolific that scientific data have become extremely abundant and widely shared. But without a good explanation for one’s findings and an understanding of the data collection process, this story suggests anyone might find “the sky is falling” too.

Kait Frasier is a fourth-year Scripps Oceanography graduate student in the laboratory of oceanographer John Hildebrand.