But Seriously, Let's Slow Down AI

speaking as a concerned citizen

If I am late in this week’s post—and yea, I am—it’s partly because I’ve had an unusually hard time tearing myself away from the news cycle of the past few weeks. Not the Trump indictment, but the bigger news, the AI news. I’m not sure how many of this newsletter’s readers will be following this already, but in case you missed it: GPT-4, the next large language model from OpenAI, was released three weeks ago, and is significantly better than its still-recent predecessor ChatGPT, which was already unsettlingly powerful. And last week two notable things happened: A bunch of people in tech signed a somewhat bizarrely worded open letter calling for a six-month pause on LLMs more advanced than GPT-4. And Eliezer Yudkowsky, a central figure in the small but influential AI safety scene, published an apocalyptic and openly desperate op-ed in Time saying the letter’s demands were too weak. Instead he called for a moratorium on large training runs, a shutdown of all large GPU clusters, and a readiness on the part of (not-explicitly-named) governments “to destroy a rogue datacenter by airstrike,” and said that otherwise we should expect all humans to die.

It’s hard to know what to make of any of this, possibly because nothing quite like this situation has ever happened before. I’m trying like everyone else to make enough sense of things to act wisely, and I guess I’m just going to spend today’s post thinking out loud toward that.

More because it’s relevant to what I’ll say than because there’s a strong reason to be confident about it: My general feeling in this uncertain moment is that the new AI systems are likely extremely dangerous, in the sense that their unchecked deployment will do irreparable harm to our social fabric, accelerate destruction of the lifeworld, and make finding a meaningful life as a human being more difficult than it’s ever been before. My main reason for feeling this way, though not the only one, is having lived through a period in history when the unchecked deployment of AI systems did irreparable harm to the social fabric, accelerated destruction of the lifeworld, and made finding a meaningful life more difficult than it had ever been before. (As Nick Bostrom pointed out back in 2006, “once something becomes useful enough and common enough it's not labelled AI anymore.") I think superintelligence seems worth being concerned about too, but my intuition, which I have no strong reason to trust in these matters, but still have, makes me place that as a more distant concern. Yudkowsky seems too volatile right now and captured by his milieu to know what to do with, but overall, not taking his claims seriously at all seems a lot crazier than taking them somewhat seriously.

I hope my feelings about these great impending harms are not true, and find it plausible that they aren’t. I can’t be at all confident that recent AI falls into the same reference class as previous things that used to be called AI, or will have comparable effects. Before this year, I was not able to have a conversation with my computer, or with any other supposedly inanimate objects. It’s just really hard to know what it means that I can.

What does seem clear-ish is that new AI with or without superintelligence will enable a massive (further) funneling of power into the hands of a shrinking number of people, and that seems… bad, if I had to pick just one word. David Chapman, whose opinion on these things is much more informed than mine, and whose online book Better Without AI I wish you would at least skim, has pointed out that a lot of the questions currently dominating the discourse, about whether these models are “really” agentic, intelligent, creative, and so on, are largely irrelevant to questions of prudent action. It doesn’t really matter if the device I’m using to, say, pollute the information ecosystem understands what it’s doing, if it succeeds in polluting the information ecosystem. The risk is not intelligence per se, but power. And to distract from that, intentionally or not (this is more me talking now) typically aids in the consolidation of power by those most interested in grabbing it. I get the impression that most of the AI researchers who are actually building these things don’t think much in these kinds of lossy abstractions during work hours—maybe later, on a podcast or something—and it hasn’t prevented them from making their creations more and more powerful.

Some people think that because open-source LLMs will soon be available on personal machines, AI may in fact de-centralize power in radical ways. That may or may not be true, I haven’t seen those arguments fleshed out, but they would still seem to be compatible with the claim that slowing down or obstructing corporate AI development is a good and safe bet. It’s worth remembering that these models were not developed to solve any particular problem, except maybe the threat of public disapproval of large tech companies.

One thing I’m noticing in all this, that bothers me, is that otherwise smart people, on Twitter and in conversation, keep dismissing the idea of a moratorium on AI development with something along the lines of, “When have people ever decided to slow down technological progress?” This strikes me as a mostly nonsensical response, in ways that seem obvious on examination, but may be easy to forget in chaotic situations where time seems limited. For one thing, it buys into the view that progress is linear and predetermined—this idea that there is an obvious direction in which “progress” is pointed, and we’ve been moving in that direction, tautologically. The purest counterexamples against this progress-as-inexorable-force vision would be technologies that don’t, by definition, exist, because we chose for any number of reasons not to develop them. Aren’t we always doing this? Aren’t there infinitely more nonexistent technologies than presently existing ones?

You might object that I’m missing the deeper point, which is that people rarely start developing a clearly powerful or lucrative technology and then stop merely because of safety concerns. One thing to say is that this is exactly why pressure from other people matters and is worth considering—because abstract safety concerns may not be enough, but taboos might well be. I think people still have not caught up with just how quickly preference regimes can change in the era of social media, or are considering the degree to which most people could be influenced by having a large number of people thinking and persuasively arguing that they’re causing preventable harms. And if you change the former question to, “When have humans ever decided to regulate and obstruct new technologies such that they developed slowly?”, I think you’ll find a few more examples. Scott Alexander makes this point entertainingly, though from priors more libertarian than mine. (I would wager he has Covid vaccines in mind as one instance of a recent obstructed technology.)

We designed our society for excellence at strangling innovation. Now we’ve encountered a problem that can only be solved by a plucky coalition of obstructionists, overactive regulators, anti-tech zealots, socialists, and people who hate everything new on general principle. It’s like one of those movies where Shaq stumbles into a situation where you can only save the world by playing basketball. Denying 21st century American society the chance to fulfill its telos would be more than an existential risk—it would be a travesty. https://astralcodexten.substack.com/p/mr-tries-the-safe-uncertainty-fallacy

(Why don’t all you Luddite visual artists likely to be put out of work by Midjourney and DALL-E try flooding the memesphere with emotionally affecting AI-generated anti-AI propaganda, as one idea I just thought of?¹)

More deeply, “people haven’t solved this before” is just a bad and unhelpful argument if a problem actually needs to be solved. And it strikes me as a stupid argument when this class of problem arguably only just came into existence. Machines never created images from prompts before, either, or outmoded fields of human endeavor once every other week. Maybe new scales of change can breed new scales of collective immune response, or, it’s silly to pretend you know otherwise.

I guess I’m just wishing more people saw their confident predictions about the behavior of other people as having an effect on that behavior. Arms races are a form of prisoner’s dilemma: the worst and most wasteful outcome occurs when both sides believe that the other guy is stockpiling weapons and therefore you’d be a fool not to follow suit. This dynamic is notoriously hard to break out of, but if you don’t want to help bring it about, maybe shouting about how it’d be crazy to expect anyone to ever not stockpile weapons is not the ideal strategy.

What constitutes a better one? I’m not sure, but I think one part might involve spending a bit of time imagining the kind of near future we each, individually, actually want to live in, and secondarily asking whether or not that future includes, or is likely to stem from, more and more ubiquitous use of AI systems. I’m seeing a lot of the opposite: people assuming AI has to exist and improve and trying to squeeze some allowance for human flourishing into that picture. Related to this, Yudkowsky has admitted² that he may have inadvertently contributed to the present AI boom by making the problems of AI alignment seem really interesting, and inspiring a generation of smart young people to enter the field who might otherwise have done something else. (I considered it more than once, reading his writing over the years.)

What would it mean to take that seriously as a cautionary tale? Something I’m finding repeatedly as I get older and see through a few of my self-defeating cycles is that this is often how unwanted outcomes are brought about. The thing you spend all your time worrying about is often the thing you bring closer; the thing you crave is often the thing you’re actively pushing away. It is almost always better to focus on what you want to see more of, rather than moving to forestall the thing you don’t in hopes you’ll end up somewhere good by default. Perhaps I am falling into that trap in this very post…

But so seriously, I’d ask you to ask yourself: what do you want your life to be like? And does AI really have anything to do with it? (Hint: Did it, before this year?) When you picture your greatest happiness, your most beautiful and meaningful life, is a friendly lil’ AI buddy there in the picture, doing the hard work for you? Or would it perhaps be more likely a distraction, an interference, another habit-forming toy? I wonder are you actually checking on this right now, in your gut where the wanting happens? Are you avoiding checking because you sense you’ll be frustrated by what actually happens in the next few years? Are you avoiding asking the question at all, or looking at what’s happening, or learning how these things work, because you think you’re unqualified to have an opinion? Do you feel as powerless as I do about this? But if you’re already relatively powerless, then what is the harm in knowing, and perhaps stating, what you want to see (knowing that what you want could always change later)?

If you’re feeling powerless, I might also recommend going ahead and feeling powerless—really feeling it—before doing whatever that thing is that would make you feel a semblance of control. Most often, for me, that thing is simply looking away, but sometimes it presents as power-hungriness.

To be frank I am fascinated by AI, and have been since college. It is indistinguishable from magic that my computer can write me a sonnet or generate an image for this blog post. The other day on the new Bing I went down a rabbithole on reverse abdominal breathing that was both enjoyable and salubrious. And as a writer, it’s an unexpected flattery that my command of language is suddenly an engineering skill. I would be lying if I said I’m not tempted to go deep on prompt engineering and see if I can’t make some easy money in this strange new economy where liberal arts types may actually have an advantage. I will probably continue using ChatGPT to generate weird party ideas and convert brain dumps into daily schedules, something I’ve been having good luck with in the last few days.

But when I move to take a wider view, I don’t see the further advancement of this stuff leading somewhere I want to be, and I see a decent chance that it leads somewhere genuinely uninhabitable for humans. I’m already not that fond of the ways this new world has been reshaping my salience landscape. I already feel a bit more machine-like myself after I engage with ChatGPT, and notice that I have a harder time envisioning interactions and ambitions as meaningful, or skills as worth developing, as I sense what could be coming next. I can’t help noticing that in recent days eating the flesh of an animal has felt slightly more like nothing, like just churning the raw data of another thing that pretended to be alive. The breakdown of categories this stuff invites comes with a significant side effect of derealization, that I assume is affecting not just me, and will probably become more noticeable in coming months. Just today I’ve been locating a delayed anger in myself about this, that other entities are able to twist the knobs this far, to make these the questions we’re spending our lives considering, without asking us what else we might have wanted, without having any idea what the second- and third-order effects of their very interesting and clever jobs might turn out to be.

Caution seems like a good idea. Caution seems possible, and like a wise collective habit to develop. Caution seems like something we could feel really good about ourselves for having chosen and advocated for, in the later on. My ability to predict here is tiny, but everyone’s ability to predict here is tiny, and that’s the point, that seems like the kind of situation caution was designed for. I wish I had something more conclusive to say at this point; I could suggest you read this book or listen to this interview. I could advise you to start meditating, to take this step now to reduce the manipulability of your attention. While you’re at it, develop a distrust of cute animals. I don’t know. But maybe it’s worthwhile enough just to say here’s my sense of things today. And to suggest that fortunately for all of us, the near future is much too weird to be foreclosed.

1. … that could not conceivably backfire.

2. I’m having a hard time confirming that he said this himself (it does say so here), but it’s uncontroversial that his writings were an inspiration for OpenAI and DeepMind—see https://www.nytimes.com/2023/03/31/technology/sam-altman-open-ai-chatgpt.html.