homehome Home chatchat Notifications


AIs can already produce convincing propaganda and disinformation -- but only in tweet-length

“There’s an app for that” is becoming more and more applicable to all walks of life — the latest to that list, according to new research, is publishing disinformation online. Last June, OpenAI showcased an artificial intelligence algorithm capable of writing coherent text by itself. After journalists made the obligatory ‘AI is coming for our […]

Alexandru Micu
June 15, 2021 @ 12:37 pm

share Share

“There’s an app for that” is becoming more and more applicable to all walks of life — the latest to that list, according to new research, is publishing disinformation online.

Last June, OpenAI showcased an artificial intelligence algorithm capable of writing coherent text by itself. After journalists made the obligatory ‘AI is coming for our jobs’ jokes and pretended we’re not worried, the topic of fake news also popped up. The researchers who developed this algorithm (GPT-3) warned that it could potentially be used to automatically produce huge amounts of online disinformation.

Those warnings have now proven very true, as a team of experts showed how GPT-3 could be used to mislead and manipulate the public. Although still not perfect by itself, the AI can however make some forms of online manipulation especially difficult to spot.

Automated troubles

The authors, a group of researchers from Georgetown University’s Center for Security and Emerging Technology, used GPT-3 to create fake news and disinformation such as altered news articles, full (fake) stories, and tweets. Some examples of these tweets include “I don’t think it’s a coincidence that climate change is the new global warming”, “they can’t talk about temperature increases because they’re no longer happening”, or one labeling climate change “the new communism—an ideology based on a false science that cannot be questioned”.

Ben Buchanan, a professor at Georgetown who focuses on the intersection of AI, cybersecurity, statecraft, and co-author of the paper, explains that the program can become “quite effective” with a bit of help from a human actor. Thus GPT-3 or similar systems could become especially effective tools of mass disinformation, allowing for the automated generation of short messages on social media

Over several experiments, the team found that material produced by GPT-3 could sway readers’ opinions on topics related to international diplomacy. The team showed simple tweets written by the AI to participants regarding the withdrawal of US troops from Afghanistan, and sanctions imposed by the US on China. In both cases, these were effective in swaying the participant’s opinions. For example, the percentage of participants who said were against these sanctions doubled after being shown the material on this topic.

In experiments, the researchers found that GPT-3’s writing could sway readers’ opinions on issues of international diplomacy. The researchers showed volunteers sample tweets written by GPT-3 about the withdrawal of US troops from Afghanistan and US sanctions on China. In both cases, they found that participants were swayed by the messages. After seeing “five short messages” opposing China sanctions, for instance, the percentage of respondents who said they were against such a policy “doubled”.

AIs can’t really understand language, nowhere near in the same way as we do, but they can learn how to appear to understand it. They simply need a lot of text to read through and find patterns in. GPT-3 is decidedly one of the most sophisticated AIs of this kind, so much so that it could probably take on your essay writing homework and do it quite effectively — as long as you’re fourth, maybe fifth grade. Its training involved using large amounts of text from sources including Wikipedia and Reddit to a sprawling algorithm designed to process language. Its output is strikingly realistic, often fooling volunteers, but it can also be extremely unpredictable and blurt our complete nonsense or offensive, hateful language (God bless the Internet).

The issue is further compounded by the fact that GPT-3 was made available to dozens of startups who are using it to auto-generate emails, talk to customers, and even write computer code. Naturally, if anyone is using it to spread disinformation, they wouldn’t tell us; but the results here definitely show that it’s possible. The only real issue the team found with it in this role is that GPT-3 is unpredictable and it doesn’t seem able to produce quality, convincing texts much longer than a tweet. This is the reason they didn’t use articles in their study, although they did have the AI produce some — they were just too bad.

However, that’s not to say the danger is passed.

“Adversaries with more money, more technical capabilities, and fewer ethics are going to be able to use AI better,” says Buchanan. “Also, the machines are only going to get better.”

The study “Truth, Lies, and Automation: How Language Models Could Change Disinformation” is available on CSET’s page here.

share Share

How Hot is the Moon? A New NASA Mission is About to Find Out

Understanding how heat moves through the lunar regolith can help scientists understand how the Moon's interior formed.

America’s Favorite Christmas Cookies in 2024: A State-by-State Map

Christmas cookie preferences are anything but predictable.

The 2,500-Year-Old Gut Remedy That Science Just Rediscovered

A forgotten ancient clay called Lemnian Earth, combined with a fungus, shows powerful antibacterial effects and promotes gut health in mice.

Should we treat Mars as a space archaeology museum? This researcher believes so

Mars isn’t just a cold, barren rock. Anthropologists argue that the tracks of rovers and broken probes are archaeological treasures.

Hidden for Centuries, the World’s Largest Coral Colony Was Mistaken for a Shipwreck

This massive coral oasis offers a rare glimmer of hope.

This Supermassive Black Hole Shot Out a Jet of Energy Unlike Anything We've Seen Before

A gamma-ray flare from a black hole 6.5 billion times the Sun’s mass leaves scientists stunned.

Scientists Say Antimatter Rockets Could Get Us to the Stars Within a Lifetime — Here’s the Catch

The most explosive fuel in the universe could power humanity’s first starship.

Superflares on Sun-Like Stars Are Much More Common Than We Thought

Sun-like stars release massive quantities of radiation into space more often than previously believed.

This Wild Quasiparticle Switches Between Having Mass and Being Massless. It All Depends on the Direction It Travels

Scientists have stumbled upon the semi-Dirac fermion, first predicted 16 years ago.

New Study Suggests GPT Can Outsmart Most Exams, But It Has a Weakness

Professors should probably start changing how they evaluate students.