Print
Why Manipulating The Internet May Be Enough To Manipulate Tomorrow’s Chatbot Answers
You’d Never Drink From A Well Without Wondering What Was Upstream… So Why Do We Treat AI Differently?
I’m getting real interested in this business of brainwashing artificial intelligence.
Not the old-school kind with flashing lights, sleep deprivation and some poor fellow strapped to a chair. Nah. This version is quieter, cheaper, and a whole lot slicker—and if you’re living off-grid, or just trying to think a little more independently, you’d better understand how it works.
Because that chatbot sitting on your computer may eventually help you research everything from your next solar build and chicken-feed formula to a strange medical study, a breaking news story, or what’s happening halfway around the world.
Which raises one mighty important question:
Who planted the information it’s giving you?
Let’s walk upstream and take a look.
Somebody’s Planting A New Kind Of Crop
People have always tried to shape what other people believe. That’s advertising in a nutshell: somebody wants you to believe X, so they buy a newspaper ad, television commercial, billboard or Facebook campaign and put X in front of your eyeballs.
Simple enough.
But something interesting has changed. Now there’s an incentive to influence not merely the people reading the Internet—but the machines reading it too.
That means filling websites, blogs, social-media platforms and other corners of the Internet with information designed to become unusually visible to AI systems. And there’s already a legitimate commercial industry growing around part of this idea.
Researchers coined the term Generative Engine Optimization, or GEO, for techniques intended to make webpages more visible in answers produced by generative search engines. The original GEO research reported that some techniques increased a source’s visibility by as much as 40 percent under the researchers’ experimental visibility metric.
Now, GEO itself isn’t sinister. It’s basically the younger cousin of SEO, and businesses have perfectly legitimate reasons for wanting their information discovered.
But you can probably see where this road leads.
SEO taught millions of website owners how to get Google’s attention. Now we’re learning how to get the machines answering our questions to notice particular information.
And once governments, corporations, political operatives, scammers, activists and public-relations shops figure out which buttons work best, you’d better believe some of them are going to push those buttons.
Hard.
There Are Two Roads Into The Well
Broadly speaking, there are two different problems worth separating. The first happens while an AI model is being trained, while the second happens when an AI system goes looking for fresh information at the time you ask your question.
They aren’t identical, and mixing them together muddies the water. But both roads can eventually lead straight to your screen.
Pipeline No. 1: Plant It Before The Model Is Born
Large language models are trained on enormous collections of text—books, websites, articles, code, reference material and massive datasets assembled from various sources. Different companies use different mixtures and filtering methods, but the important point for our purposes is pretty simple:
The information environment matters.
A remarkable 2026 study published in Nature examined exactly this problem. Researchers conducted six studies looking at how government-controlled media environments can influence information that eventually enters AI training data, and the answers models produce.
Their Chinese case study found state-coordinated media material inside LLM training datasets. Then the researchers did something even more interesting: they took an open-weight model and gave it additional pretraining on Chinese state-coordinated media.
What happened? The resulting model produced more favorable responses concerning Chinese political institutions and leaders.
The researchers then looked at commercial models and found another fascinating pattern. Asking equivalent questions in Chinese produced more favorable responses about Chinese institutions and leaders than asking those questions in English.
In other words, the digital soil mattered.
You don’t necessarily have to sneak into an AI company’s headquarters and fiddle with the machine. Shape enough of the information environment upstream, and some of that environment can eventually flow downstream into AI systems.
And Sometimes It Doesn’t Take Much Poison
Here’s where things get even stranger. Researchers from Anthropic, the UK AI Security Institute and the Alan Turing Institute deliberately experimented with poisoning training data to see how much malicious material it might take to affect model behavior.
They discovered something nobody should casually shrug off.
In their experimental setup, as few as 250 malicious documents were sufficient to implant a particular backdoor behavior into models ranging from 600 million to 13 billion parameters.
Think about that.
Not 250 million documents. Two hundred and fifty.
Now, there’s an important fence post we shouldn’t drive past here. The researchers were testing a narrow experimental backdoor designed to make models produce gibberish after encountering a particular trigger; they were not demonstrating that 250 webpages can magically turn a major commercial chatbot into somebody’s political spokesman.
That’s a much bigger claim, and the experiment doesn’t necessarily establish it.
But the experiment demonstrated something important anyway: malicious training-data influence may require far less material than you might expect under certain conditions.
And that’s the part that ought to make your ears perk up.
Pipeline No. 2: The Live Wire
Then there’s the second road into your answer, and this one can move a whole lot faster. Many modern AI systems don’t rely entirely on information learned during training; depending on the product and the question, they may search or retrieve fresh information and use those documents to construct an answer.
That’s broadly the territory of retrieval-augmented generation, usually called RAG.
Think of the difference this way. Training-time influence is like putting something strange into the feed while you’re raising the calf; retrieval-time influence is like tossing something into the feed bunk five minutes before supper.
The second method doesn’t necessarily require changing the underlying model at all.
You change what the model finds.
That means the integrity of the source pile suddenly becomes mighty important. If a system retrieves poor, misleading or deliberately planted information, the model may still produce a beautifully written answer based on lousy material.
It’s the oldest computer lesson in the book wearing a brand-new pair of overalls:
Garbage in. Garbage out.
Russia Figured Out The Game
Now we get to one of the more fascinating real-world examples.
Everybody likes blaming the Russians for everything from elections to the neighbor’s missing milk cow, but in this particular case there’s documented evidence worth examining.
Researchers and NewsGuard investigated a network known as Pravda—not simply the old Soviet newspaper, but a sprawling pro-Kremlin network of websites. According to NewsGuard’s 2025 investigation, the network published approximately 3.6 million articles during 2024 alone.
And here’s the kicker. According to that investigation, the apparent strategy wasn’t simply persuading human readers; it included flooding search results and web crawlers with material intended to influence the information AI systems retrieve.
NewsGuard tested 10 major generative-AI tools against false narratives circulated through the network. Its audit reported that the systems repeated those narratives in 33 percent of the tested responses.
That’s an important distinction.
It doesn’t mean somebody in Moscow secretly broke into an AI company’s server room and started turning knobs. Something much simpler can happen.
Fill enough of the pond, and eventually somebody’s bucket comes up with your water in it.
China Shows The Other Side Of The Problem
China gives us another version of the same lesson. That Nature study is especially useful because researchers didn’t merely say, “Hey, Chinese propaganda exists.”
Everybody already knows all governments publish material favorable to themselves.
Instead, the researchers examined part of the actual information pipeline. They found Chinese state-coordinated media in training datasets and experimentally demonstrated that additional pretraining on that material could shift an open-weight model toward more positive answers about Chinese institutions and leaders.
That’s significant because once a government talking point gets copied, translated, summarized, reposted and republished enough times, the original label can disappear. Yesterday it was a government statement; tomorrow it’s a news summary, next week it’s a blog post, and eventually it may wind up inside some giant pile of machine-readable text.
Then the machine may hand some version of it back to you in the calm, confident voice of an apparently neutral assistant.
That’s the laundering effect worth watching.
Then Madison Avenue Smelled Money
Naturally, governments weren’t going to have this playground to themselves. Commercial marketers quickly recognized what was happening.
Remember GEO—Generative Engine Optimization?
The original research tested techniques including adding quotations, statistics and citations, reporting substantial improvements under its visibility metrics. The researchers concluded that GEO techniques could increase visibility by as much as 40 percent in their experimental setup, although that figure shouldn’t be casually treated as “40 percent more traffic” or “40 percent more citations” in every real-world AI system.
That’s not necessarily manipulation. Good information should be well sourced, and there’s nothing sinister about making a useful webpage easier for an AI system to discover.
But once businesses discover that certain kinds of webpages get cited or surfaced by AI systems more often, what do you think happens next? Exactly what happened with Google.
An industry appears. Consultants arrive, courses get sold, agencies promise visibility, and everybody starts reverse-engineering the machine.
Pretty soon, instead of asking, “How do I rank first on Google?” business owners are asking a new question:
“How do I become the answer the chatbot gives?”
That’s a very different Internet.
And Now We Have The Feedback Loop From Hell
Here’s the part that really gets interesting. AI models are no longer merely reading the Internet; they’re increasingly helping people write it.
People use AI to produce blog posts, product descriptions, news summaries, social-media posts, press releases, comments, emails and mountains of other text. So imagine an AI system picks up a distorted claim and then helps generate hundreds of new pieces of content containing some version of that claim.
Those pages get copied, indexed, summarized and reposted. Another AI system encounters them, and that system helps produce another mountain of material.
Round and round we go.
It’s the informational equivalent of saving seed from a diseased crop and planting the whole north forty with it next spring.
Pretty soon you’re not sure where the disease started. You just know it’s everywhere.
The Smaller The Pond, The Easier It Is To Muddy The Water
There’s another practical point here. Try changing the Internet’s collective story about something that’s been exhaustively documented for decades, and you’ve got a tough row to hoe.
There are books, newspapers, court records, government documents, photographs, archives, academic papers and millions of webpages sitting there as competing evidence. Whatever nonsense somebody invents has to swim against all of that.
But suppose something happened yesterday morning.
Or suppose you’re researching an obscure company, tiny scientific controversy, little-known court case, niche product or breaking political story. Now the information pond is smaller, and when the pond is small, somebody dumping three truckloads of mud into it matters a whole lot more.
This is closely related to what’s sometimes called an information or data void—a subject where reliable, authoritative material is sparse or difficult to find.
Those are exactly the places where careful source-checking becomes critical.
The Machine Doesn’t Have To Be “In On It”
This may be the most important thing to understand. The chatbot doesn’t have to be consciously lying to you, and the AI company doesn’t necessarily have to be part of some conspiracy.
The programmers don’t have to secretly agree with whoever planted the material, either.
That’s almost what makes the problem more interesting.
Imagine somebody contaminates the creek five miles upstream. Your well pump isn’t evil, your kitchen faucet isn’t plotting against you, and the pipes aren’t members of a secret society.
They’re just carrying the water.
That’s why understanding the source matters more than getting angry at the faucet.
How To Guard Your Digital Homestead
So what’s the practical lesson? Simple: never confuse a chatbot answer with reality itself.
AI can be an extraordinary research tool. Used properly, these machines can help you cover more ground in an afternoon than you might once have covered in a week—but they’re still tools, not oracles.
When something matters, ask the machine where the information came from. Open the sources, find the original paper instead of somebody’s summary of the paper, and find the court document instead of the article describing the court document.
Find the government report instead of somebody’s tweet about the government report. Check dates, compare genuinely independent sources, and ask whether five websites are actually five independent sources—or five websites repeating the same original press release.
And here’s one of my favorites:
Ask whether the claim existed before the controversy started.
That little trick can reveal a surprising amount. If 300 websites suddenly started saying exactly the same peculiar thing last Tuesday, that’s different from discovering genuinely independent sources saying it for twenty years.
Watch For The Digital Seed Drill
Which brings me to something worth watching more closely. You’ll sometimes see dozens—or even hundreds—of social-media accounts posting remarkably similar arguments, facts, phrases, framing and punch lines.
Sometimes that’s ordinary human bandwagon behavior. Sometimes it’s organized advocacy or a coordinated campaign, and sometimes automated or AI-generated content may be involved.
But increasingly there’s another question worth asking:
Are human eyeballs always the only audience?
Because today those posts may persuade people. Tomorrow they may become searchable documents, and later they may wind up in archives, datasets or retrieval systems.
Then somewhere down the road, another person asks an AI assistant an innocent question. The answer arrives in two seconds—clean prose, calm voice, no campaign logo, no spokesman standing behind a podium and no obvious clue about the information ecosystem sitting upstream.
That’s the real trick.
The fellow who sees propaganda knows somebody is trying to persuade him. The fellow who encounters the same idea six months later through an apparently neutral machine may never know there was a campaign in the first place.
Check The Well Before You Drink
For generations, people living out in the country understood something city folks could afford to forget. Your water doesn’t magically begin at the kitchen faucet; it came from somewhere, and if something nasty gets dumped upstream, eventually you’d better know about it.
Information works the same way now.
AI isn’t some magical brain floating above human bias, propaganda, advertising, government influence and commercial manipulation. It drinks from the same digital watershed the rest of us created, and there’s now direct research showing that the composition of that watershed can affect what models produce.
Meanwhile, researchers are learning how webpages can become more visible in generative answers, marketers are building businesses around that discovery, and influence networks have already tried flooding the Internet with material intended to reach AI systems.
So use the tools.
Use the heck out of them.
But keep your hand on the gate. Check the sources, follow the links, read the original documents, compare competing accounts and look at the dates.
And when everybody suddenly starts saying exactly the same thing at exactly the same time, ask yourself who handed out the seed.
Because out here, self-sufficiency was never just about producing your own food, storing your own water or making your own electricity.
It also means guarding the one piece of ground nobody else should be allowed to homestead.
Your mind.
