Here's how AI can REDUCE health misinformation

How the government can build a bullshit detector

It was shortly after my 30th birthday that my life started to go wrong.

We were packing to move to a different flat, and as I bent over to lift a box, my partner realised a horrifying truth that should have long been obvious to us. I was losing my hair.

Suddenly the blocked plug-hole of the shower started to make sense, as did the shedding of hairs when I ran my hand across my head. I almost convulsed as the ravages of age caught up with me in that moment. It was genuinely quite traumatic, as I’d never once considered before that I might go bald.¹

Seven years later and the decay is almost complete. Though I still have some hair on top of my head, it is very much the definition of living in denial. What was once a bald spot, became a bald patch, then a bald hemisphere. And though face on, I can look into a mirror and tell myself everything is mostly okay, I also know that there’s a very important reason why when I get a haircut, I ask the barber not to hold up a mirror to show me the back of my head.

Clearly then, I’m definitely not emotionally ready for the moment where I’ll finally face reality and shave my head – a day surely not too far away, especially given that functionally, I have already given up trying to fight the inevitable.

But this wasn’t always the case. Over the last half decade, I’ve tried to find a way out. At my lowest point a few years ago, I tried paying for an expensive chemical treatment to dowse on my scalp – even though I’m pretty sure that it wouldn’t stand up to the rigours of peer review.

And even though my hair did not start to grow back, I did learn something from the experience.

Despite being a cold-hearted rationalist at heart, I can at least a little bit relate to people who seek out ‘alternative’ remedies to health problems. If you’re desperate, and are facing a more serious problem than looking like if Gregg Wallace had let himself go, then of course you’d be tempted to try nonsense like homeopathy, reflexology and acupuncture.

But this is also why alternative medicine makes me angry. It makes me furious that there are charlatans profiting, when at best they’re offering false hope, and at worst, there’s a risk that patients may forego treatments that actually work, in favour of ‘treatments’ that are completely ineffective.

In other words, my views on alternative and ‘complementary’ medicine are in alignment with those of Tim Minchin, as expressed in his beat-poem ‘Storm’: “Do you know what they call 'alternative medicine' that's been proved to work? Medicine."

Anyway, this is why today I’m going to explain why we need better regulation of alternative medicine – and how AI could be the solution to the problem.

If you enjoy nerdy takes on politics, policy, tech and media, then be sure to subscribe (for free!) to get more of this sort of thing in your inbox.

The need for regulation

So why regulate alternative medicine, if it is something that – by definition – doesn’t work? Essentially the problem is that if alternative medicine practitioners were completely unregulated, it would be a disaster, as it would empower the charlatans who take advantage of the vulnerable.

Regulation, though, is perhaps easier said than done. The problem is that it is difficult to ban or restrict certain treatments because… they literally do not do anything. Instead, the harm comes not from someone poking you with needles or dispensing sugar pills, but from the illusion of treatment.²

So to get around this difficulty, the government instead regulates the language used by alternative medicine practitioners – with bodies like the Medicines and Healthcare Products Regulatory Agency (MHRA) and the Advertising Standards Authority (ASA) getting involved when therapists make exaggerated or nonsense claims about what their snake oil can do.³

However, regulating language is also a very difficult thing to do.

At the moment, misleading medical claims made by alternative therapists are only really identified when small activist groups like the Good Thinking Society point it out proactively – or when something has already gone wrong, and complaints are made.

And inevitably, there is no way that regulators and a few nerdy activists can keep up with an entire industry, because there are thousands of alternative and complementary medicine practitioners – in fact, over 6000 are registered with the Complementary and Natural Healthcare Council (CNHC), a voluntary register of providers.

So given this, how can regulators make sure that every clinic is complying with the law, and not making false or misleading claims?

Enter my technologist friend Simon Perry, who has done something rather clever. A few weeks ago, he published a paper, working with some top scientist colleagues⁴ and a charity called HealthSense UK. Together, they used AI to reveal compelling evidence that most alternative therapists are technically breaking the rules.

In fact, Simon and his colleagues discovered that a staggering 97% of the alternative medicine clinics’ websites that were analysed contained misleading, unscientific health claims.

In other words, they had successfully built a bullshit detector.

So let’s dig into how they did it, what the study found – and why this has bigger implications for government beyond just medical regulation.

The bullshit detector

The first step was to take the dataset of clinics from the CNHC and prune the list to remove clinics that only do things like sports massage and hypnotherapy,⁵ which at least have one foot in the empirical.

This left just over 1300 clinics that offer pseudoscientific treatments like the Alexander Technique, Aromatherapy, Bowen Therapy, Colon Hydrotherapy, Craniosacral Therapy, Healing, Kinesiology, Microsystems Acupuncture, Naturopathy, Reflexology, Reiki, and Shiatsu.⁶

Don’t worry – you don’t need to know what all of these are. But all are considered pseudoscience, and unsupported by scientific evidence.

The next step was to download their websites, and put an AI tool to work, crawling through the first 30 pages of each website to identify claims aren’t supported by scientific evidence.⁷

To do this – and I think this really illustrates the mad, new AI-powered world we’re already living in – Simon used the ChatGPT API, which works in exactly the same way as ChatGPT the app, but enables software to send requests, instead of having to type each request in manually.

In other words, to get the GPT4 model to evaluate the scientific claims on the websites, he didn’t have to write any specialist code. Instead, just like the rest of us using ChatGPT, he had his software ask nicely in plain English.

So his software sent the text of each page, and asked ChatGPT to “carefully review the text for any explicit or implied claims about the effectiveness of treatments in curing, treating or managing illnesses or diseases” – and instructed it to compile a list of exact quotations from the source text, together with an explanation of why the claim is considered false or misleading.⁸

After nine days of data crunching, the team sense-checked the results by comparing sample with the work of human fact-checkers. The good news is that they were about as accurate as each other.⁹

And then, they had the results. As mentioned above, the analysis determined that a staggering 97% of the 725 websites included in the study contained misleading claims – and that 65% of over 8500 individual webpages contained something false, with each website hosting an average of 32 false or misleading claims.

I think these numbers are genuinely shocking – this should be considered a scandal. We now have evidence that medical misinformation is not just rife, but near universal amongst practitioners of alternative medicine.

So I really hope the Department of Health or the MHRA steps in and tightens the screws on the clinics – as that’s potentially a lot of vulnerable people being hoodwinked.

But this isn’t my only conclusion from the study.

Some awkward handwringing

Clearly, the experiment was astonishingly effective. What the researchers found a scalable way to keep tabs on thousands of clinics – and identify the endemic usage of misleading claims to promote their services.

But here’s what I think is most interesting: This doesn’t just have to be a one-off.

Imagine if the MHRA, the ASA, or some other government regulator tasked with regulating specific medical claims used this same technique – perhaps even the same code – to build a full-time bullshit detector for their field. There’s no reason why the MHRA couldn’t write software that systematically evaluates claims made by clinics as their websites are updated.

Suddenly, the government would be in a much stronger position to force clinics to comply, and to limit the spread of health misinformation.

In fact, I think that it’s inevitable that tools like this will be created. Because it’s so obviously useful. And it’s easy too – that’s how a small team were able to create something that did this exact thing so quickly.

And more broadly, I think it’s a striking example of how AI can be operationalised across government, and how AI can make governments wildly more effective.

However, this also brings me to some inevitable handwringing.

Though Simon and his colleagues have illustrated an incredible new capability, I think it is worth worrying, at least a little, about the downstream implications of such technology.

Specifically, there’s no reason why exactly the same text-parsing capability couldn’t be used by bad actors to identify dissent or forms of political speech they don’t like. Instead of asking the AI model to identify claims that aren’t supported by scientific evidence, some hypothetical government censor could ask it to identify text written that goes against the stated party line. That would obviously be bad.¹⁰

However, despite this theoretical concern, and the potentially scary implications, I’m not actually very worried about slippery slopes downstream of using AI to regulate medical information.

This is because whether we like it or not, the AI genie is already out of the bottle. At this point, there are no technical restrictions that could conceivably be placed on the use of AI for practices like this.¹¹ Sure, Simon used OpenAI’s powerful GPT4 model – which could conceivably be switched off by OpenAI – but in time, this sort of textual analysis will be available locally on our devices with no one able to press the “off” switch.¹²

So in other words, any limitations on using AI in government is not going to be technical – it’s going to be legal, institutional and cultural.

And while I’d be uneasy if this sort of text analysis was used to, for example, identify ‘problematic’ social media posts as part of a hate crime law, it would be foolish to let this worry stop us from taking advantage of the same capabilities for more narrow use-cases – like finding text on alternative medicine clinic websites that break specific medical regulations.¹³

It’d be like refusing to use nuclear power, just because the same scientific principles can be harnessed to build nuclear bombs.

So while it will be important to build new norms around when pro-active “bullshit detector” regulation is acceptable or not, fundamentally I think we should take advantage of it.

Because what Simon and his colleagues have demonstrated is an incredible new way to use AI not to spread misinformation – but to prevent it. And they have handed health regulators the tools they need to act.

Phew! You reached the end! Take pity on a balding man by subscribing (for free) to get more politics, policy, tech and media takes direct to your inbox:

1. Despite having both my dad and grandad who lost their hair, as I was growing up I was persuaded by the delusion that baldness is hereditary through the female line. And perhaps knowing it would mean discovering an ugly truth, I never questioned this assumption. Nor did I think to ask my mum if her dad (who died before I was born) lost his hair.

2. This footnote is for the people who want to smugly mention the placebo effect, even though we all know that still doesn’t mean the underlying ‘treatment’ works, and even if we want to take advantage of the placebo effect, we should be able to find a means of doing so without lying to patients.

3. This, incidentally, is why the word “wellness” has become so commonplace: It sounds like a good thing, but is deliberately vague and means nothing. So it’s very useful for charlatans who don’t want to risk using legally actionable words like “cure” or “treatment” or, er, “medicine”.

4. Les Rose, Susan Bewley, Mandy Payne and David Colquhoun.

5. Hypnotherapy sounds like nonsense to me, but it is definitely slightly more scientifically credible than the purest pseudoscientific treatments.

6. I thought a shiatsu was a type of dog.

7. They limited it to 30 pages from each website for ease/cost/etc.

8. The real nerds can find a more detailed breakdown of the prompt and methodology in the actual paper, which is linked above.

9. Interestingly, they found that different humans tended to disagree with each other on whether various claims were in line with the science more than the AI did – suggesting that an AI regulator could apply the rules more consistently.

10. More broadly, I am definitely nervous about the overzealous policing of speech. And my view is that especially since 2016, cries of “misinformation” have been used and abused by individuals and institutions alike to hand-wave away realities that are politically awkward. I’m definitely someone who would rather live in a world of slightly too much “free speech” than too little.

11. Frankly, if the Chinese Ministry of Information isn’t already using something like this to monitor dissent, then someone in Beijing needs sacking.

12. I suspect Meta’s Llama model could already do a pretty decent job of doing the same thing as GPT4 did here – even though it is a much smaller model, specially designed to run locally, on a pretty standard desktop GPU.

13. A significant difference between the two examples I give is that the level of racism in a social media post is relatively subjective, whereas “does claiming magic water can cure serious diseases contravene the scientific consensus in marketing material aimed at consumers” is somewhat less so. And if, on the off-chance, that the scientific consensus is wrong, then the homeopaths can prove their solution works using the scientific method, and collect their Nobel Prize. But I admit there is a slightly blurry line here.