Providing a rare glimpse of the trade secrets of planetary scientists, the journal Icarus published a study this month that compared lunar crater counts by eight professionals with crowdsourced counts by volunteers.

The professional crater counts varied by as much as a factor of two; some professionals counted twice as many craters as others. But the population of craters found by the volunteers was statistically similar to that found by the experts, said the study's lead author, Stuart Robbins, PhD, of the University of Colorado at Bouder.

These somewhat paradoxical results suggest both that it is harder to count craters than to count, say, train cars, but also that, if there are enough people counting, the robust similarity of their responses washes out slight individual differences.

The professionals included scientists from seven different universities, including Washington University in St. Louis. The volunteers were Moon Mappers who had signed up to count craters through Cosmoquest, a citizen science website.

To find out more, we talked to the WUSTL crater counters: postdoctoral research associate Kelsi Singer, PhD, who has mapped craters on the icy satellites in the outer solar system, and graduate student Michael Zanetti, who has counted craters on the moon and Mars. Both are in the Department of Earth and Planetary Sciences. Neither was surprised by the two-fold discrepancy between expert counts.

"There were very few singletons, craters only one person mapped," Singer said. "Most of the discrepancies showed up in counts of degraded, or 'subdued,' craters, or ones that were very small."

Zanetti explained that one of the images they were asked to analyze included both lunar maria and highlands. "It's not so bad counting craters on the flat, smooth maria created by volcanic eruptions. But it's a different story counting them in the heavily cratered lunar highlands," Zanetti said.

"You're never going to get a true distribution of crater sizes there," he said. "You've reached something called saturation, where each new crater obliterates old ones."

For the most part, Singer added, crater counts are used to reconstruct the history of a planet. "A heavily cratered feature is probably older than one with few craters, she said.

"But we're usually trying to understand whether a feature is a million, 100 million or a billion years old. Even if you are off by a factor of two in your counts, it's not going to change your understanding of the history," she said.

Nonetheless, Zanetti said, when he is working on an important problem, he counts all the craters himself to make sure the count is at least self-consistent. He then publishes his technique and his definition of a crater with his results. "There's a bit of subjectivity to it, but not as much as you might think," he said.

But, he added, it can take a couple of months to count the thousands of craters for really big projects.

"You don't undertake something like that lightly," he said. "The question you're trying to answer has to be a really good one."

Zanetti admits that after a long stint of crater counting, he sees colored circles everywhere, including around the bumps on a stuccoed wall.

So both Zanetti and Singer are happy to know that crater counts could be crowdsourced. "We have WAAAYYY too many craters to possibly map them ourselves," Singer said.

At bottom, the study showed that people are fantastic pattern recognizers. "Pattern-recognition software is still not up to the game," Zanetti said. "It takes as long, if not longer, to double-check the software's counts as it does to do the count yourself."

"But people are good at finding circles," Singer said. "Even if the crater is half gone, our brains fill in the rest of the circle."