Researchers taught thousands of Ugandan school children, some as young as ten years, how to think critically and sniff out bogus health claims. When assessed with various tests, there were twice as many children among those who received the ‘recipe’ for sniffing out medical falsehoods that got a passing grade than the control group comprised of children who were given no instructions. A similar trend was reported for parents, as well, suggesting ‘bullshit detection’ can become an acquired skill — more important than ever in a so-called ‘post-truth’ era.

Credit: Pixabay.

A good bullshit detector starts early

The experiment was led by Andy Oxman, who is the research director at the Norwegian Institute of Public Health. Oxman became inspired to school young people in the ways of critical thinking after attending of his 10-year-old son’s classes almost twenty years ago. Back then, Oxman told his son’s class that some teenagers found that red M&Ms makes you feel good and helped them write and draw more quickly. There were also some side-effects, though: a little pain in their stomach, and they got dizzy if they stood up quickly.

Oxman challenged the school kids to an experiment meant to validate or disprove these findings. The class was divided into a couple of working groups, each with a full bag of M&Ms at their disposal. Although no particular instructions were offered, the children were clever enough to notice that they a) had to try out each different colored M&M to see what the produced effect was and that b) no such test would be completely fair if the children could see the color of the M&M. Essentially, they had discovered the utility of ‘blinding’ in science before anyone had the chance to teach them what it meant.

By the end of the experiment, most of the children reported little to any difference in the effects of differently colored M&M and even questioned the teenagers’ method itself. Oxman had been disappointing by previous interactions with adults meant at instilling critical thinking — and this was before Facebook even existed, let alone the ‘fake news’ craze. This episode, however, suggested that if children were given the means to spot bullshit, they could become immunized against said bullshit. But could he prove it?

Many years later, Oxman got the chance to test this hypothesis in a huge trial involving no fewer than 10,000 school children from 120 primary schools in Uganda. Oxman and colleagues adapted concepts from a popular book called Testing Treatments (free download) which explains in plain English all the concepts people need to grasp in order to separate garbage from genuine health advice. Eventually, the team reached six key points a person would need to grasp in order to think critically about medical treatments. As reported by VOX, these are:

1. Just because a treatment is popular or old does not mean it’s beneficial or safe.
2. New, brand-name, or more expensive treatments may not be better than older ones.
3. Treatments usually come with both harms and benefits.
4. Beware of conflicts of interest — they can lead to misleading claims about treatments.
5. Personal experiences, expert opinions, and anecdotes aren’t a reliable basis for assessing the effects of most treatments.
6. Instead, health claims should be based on high-quality, randomized controlled trials.

According to a Ugandan urban myth, cow dung helps burns heal faster. It does not. Credit: Wikimedia Commons.

Armed with a solid templated and fun exercises, the team comprised of researchers from Uganda, Kenya, Rwanda, Norway, and England, ran their program across more than a hundred schools in Uganda. The researchers also put together a guidebook for teachers and cartoon-filled textbooks for children. The stories presented in the textbooks were also adapted to fit the local narrative, including myth bashing. For instance, many locals recommend putting cow dung on burns as the best treatment for burns [false]. Some of the included myths are known for their particular anti-scientific and menacing nature. For instance, a local myth that had circulated wildly suggested immunization was linked to infertility somehow. As a result, parents stopped allowing their children to become vaccinated with grave consequences. Another myth caused people to replace antiretroviral therapies for HIV with herbal supplements.

To see how all of this work impacted children’s critical thinking, 10,000 fifth-graders, mostly ages 10 to 12, participated in a trial from June to September 2016. Half of the school children had been schooled in detecting bogus medical claims while the other half wasn’t and acted as the control group.

The average score on the test for the kids schooled by Oxman and colleagues was 62.4 percent compared to only 43.1 percent for the control group. More importantly, maybe, twice as many kids from the intervention schools achieved a passing score than those from the control group. The schooling may have even formed the future’s Ugandan critical elite. About one-fifth of the kids schooled by the researchers had mastered the key concepts (more than 20 of 24 answers correct) compared to only 1% of the control group.

When the researchers tried the same thing on parents, they saw similar results. Instead of a course, the parents listened to a dedicated podcast about critical thinking in a medical context. Twice as many parents who listened to the podcast series passed a test on their understanding of key health concepts compared with parents in the control group, as reported in two studies published in the Lancet (1 and 2).

Though it has its limitations, the sheer scale of the study suggests early inoculation of critical thinking works! In today’s age of unshameful, blatant lying in the public space, it’s nice to hear that simple education can actually work.