There’s a scene early in Neal Stephenson’s new novel Fall: or, Dodge in Hell, in which a tech billionaire, sick of the way that misinformation spreads across the Internet like Western wildfires, decides to stage an intervention. He spends about a million bucks — he doesn’t need more — to create and distribute digital “evidence” of a tactical nuclear strike on the town of Moab, Utah. He hires actors, CGI experts, everything you need in order to fake a tragedy and make your creation go viral online. The idea is that once people see that completely made-up shit can utterly dominate the Internet, that there are no guard rails or boundaries to prevent that from happening, they will realize that they are continually being snookered and will grow a carapace of skepticism.
This is followed by a scene in which an master programmer creates highly advanced bots capable of relentlessly pushing petabytes of inconsistent and incoherent misinformation onto the internet, thus reinforcing the billionaire’s lesson on an unimaginably massive scale. The ENSU project starts by spewing its misinformation about one woman, who cheerfully cooperates:
If everything went according to plan, the Ethical Network Sabotage Undertaking would now issue a press release announcing its existence and explaining what it was doing. It would include a signed statement, as well as a video clip, from Maeve Braden, announcing that she was completely fine with all of this. Also included were links to servers where all of the code was available in the form of a carefully documented open-source code package, complete with sample projects that programmers could use to modify and extend it in various ways. Following up on an idea that had emerged during the conversation on the jet, ENSU also made public a list of several hundred completely imaginary, nonexistent people against whom campaigns of reckless slander and defamation could now be unleashed, as well as an easy-to-use tool that anyone could exploit to create new such fake persons and reasonably convincing social media shaming campaigns that would make those fake persons the object of real, genuine, sincere obloquy on the part of millions of social media users who were dumb enough to believe everything that scrolled across their screens.
You know the old Justice Brandeis line that the remedy for malicious and deceitful speech is more speech? A version of that is what’s happening here: the remedy for malicious and deceitful memes is more memes. So much malice and deceit and that people will give up believing any of it. Brilliant.
Now fast-forward fifteen years or so:
The Utah state legislature had been taken over by Moab truthers who insisted that Moab had been obliterated by nuclear terrorism twelve years ago. From which it followed that anyone claiming to actually live there was a troll, a crisis actor in the pay of, or a sad dupe in thrall to, global conspirators trying to foist a monstrous denial of the truth on decent folk.
In short, nothing changed. People kept believing whatever they saw online that they wanted to believe. They could actually go to Moab and see that it had not been damaged and was not radiation-riddled, but they didn’t.
Some elements of Stephenson’s anticipated future seem unlikely to me, but not this. This seems not just plausible but highly probable. (Cf. Alex Jones and Sandy Hook.)
Nobody is beyond hope. This is an article of faith for me. But if at this stage of the game, given what we know about how social media work and about the incentives of the people who make TV, you’re still getting your dopamine rush by recycling TV-news clips and shouting at people on the Internet, you’re about as close to beyond hope as a human being gets. There is no point talking to you, trying to reason with you, giving you facts and the sources of those facts. You have made yourself invulnerable to reason and evidence. You’re a Moab truther in the making. So, though I do not in theory write anyone off, in practice I do. It’s time to give you up as a lost cause and start figuring out how to prevent the next generation from becoming like you.
