Can Keith Teare Convince Jonathan Rauch That AI Is Benign? That Was the Week, Special Edition

00:00:31 Andrew Keen: Hello, everybody. Welcome to a very different That Was the Week. I do a weekly show with my old friend Keith Teare, the publisher of the That Was the Week tech newsletter about events in tech. But as it happens for the weekend of May 16, I'm gonna be in Korea. So we thought we would do things slightly differently. Another old friend of the show is Jonathan Rauch, Brookings fellow, very distinguished author of many different books on politics and technology and society and culture. And John, I know, enjoys the show I do, the weekly show with Keith Teare, particularly on AI. And John asked me if he could have the opportunity to come on the show and ask Keith some questions of his own about AI. John describes himself as not knowing a great deal about AI, although I suspect he probably knows about as much as I do, and probably as much as Keith. So, John, over to you. You don't need much of an introduction. Old friend of the show, influential political writer, thinker in Washington, DC. So this is your opportunity to ask Keith anything you want about AI.

00:01:44 Jonathan Rauch: Well, thank you, Andrew, for having me on the show. Thank you, Keith, for tolerating me. The account Andrew just gave is not completely accurate. The version, as I recall it, is that I'm a regular listener to Keen On America, and, of course, I listen to some of those shows that, Keith, you do with Andrew. And I heard something that made my ears prick up because, as far as I know, no one else has said that he is confident that AI is a benign technology. He seemed to state that everything, all the doomsaying and all the negativity, it's just hogwash, and that you know that. So I sent Andrew an email, and I said, how does Keith know that? And Andrew said, well, you should ask him. He's quite obstinate on the matter. And I said

00:02:38 Keith Teare: Yeah. Yeah.

00:02:39 Jonathan Rauch: I said, well, I don't know anything about AI. I literally I've just basically, you know, use it for casual research now and then. And Andrew said, all the better. So I am a journalist. I do know how to ask questions. So I said, okay. Let's see if I can get Keith to walk me through why he is, as far as I can tell, the only person in tarnation who is confident that AI is benign. There's five or so different buckets of things we can discuss, five different brands of gloom and doom that I've counted. And if you like, we can walk through them. One is employment, labor market, lots of people out of work, massive disruption. Another is political disruption, you know, disinformation, deep fakes. Tech oligarchs take over the world. Thirdly, mental health. Cognition, we forget how to think. We get AI psychosis. We forget how to date. Fourth is malicious actors, you know, bioterrorists who can make new viruses, hackers who can break into any system, malware, cybercrime. And then the big kahuna, which people seriously talk about, which is AI will kill us all. It will develop intelligence. It will develop its own agenda. It will develop agentic properties, and it will make itself super smart and kill us. There are books that say that.

00:04:05 Andrew Keen: Yeah. And the book, the book—I don't know if you've read it, John—If Anyone Builds It, Everyone Dies. Eliezer Yudkowsky and Nate Soares is the best-selling book on this.

00:04:18 Jonathan Rauch: Yeah. Have you had them on the show by chance?

00:04:20 Andrew Keen: No.

00:04:21 Jonathan Rauch: Yeah. I confess I haven't read it. I've read very little on this topic. I've heard them on podcasts. They make a case. And then I was particularly piqued by a podcast that Sam Harris did with Tristan Harris. Now these are very smart guys, and, this came up in April. And Tristan Harris walks through the reasons we should take very seriously the idea that AI is just a coming catastrophe and that no one is taking it seriously. And Sam Harris says, look, even the inventors of this thing say there's, you know, up to a 20% chance that it could be cataclysmic. What other technology do we tolerate that kind of risk taking with? So that's the zeitgeist right now. Keith, you're all alone. So I wanna know what gives you confidence that AI is benign. And I'd like you to be as specific as you can, and then I'm gonna see if we can test the joints of your argument a bit.

00:05:13 Keith Teare: Yeah. Yeah. Well, I probably should disclose a bias before I answer, which is that as an observer of history, it is my belief that whenever there is ambiguity in change, a business model emerges to monetize doubts. And I do think there is a business model. You could think of it as clickbait, but that's probably too narrow a definition of seizing on the negatives to earn revenue from willing listeners. And so I do have an underlying belief that there is a systematic doomerism that actually represents a business model, selling books, selling podcasts, selling links that is not especially educated in the space. And on the face of it is reasonable. In fact, it wouldn't work unless it was reasonable, because AI has within it all kinds of possible outcomes, including negative ones. And whenever that's true, as it was say with nuclear power, you know, there are obviously very good uses of nuclear power and very bad ones. Climate change is another good example. Whenever there is ambiguity, there's a business model in taking the negative side. And I do think in AI that's very strong. So when I look at these things, I am inclined before I even engage with the ideas to pigeonhole them into that group. Now I wouldn't do that unless I had some technical foundation for my belief. And so to answer your question directly, the word AI is, in a way, not a very useful label. It sits on top of a bunch of technologies that are much more specific. And in particular, large language models, which are effectively word-counting machines. They train on words, nothing else. And they split those words into a graph which correlates how close word A is to word B on a probabilistic basis. And so when you ask an AI a question, a large statistical engine is triggered that starts word by word responding to you. And in that sense, it's no cleverer than a calculator. It's just a very big calculator. And so the idea that it has any kind of awareness or consciousness or plan is mythological. This technology would be incapable of doing that. And so what we're really talking about when we talk about the dangers and there are dangers. I'm not denying there are dangers. But the dangers are human, not AI. What's dangerous is what a human does with AI, not what the AI does itself. In fact, even the idea that there is such a thing as the AI in itself is a mistake. So that's my basis.

00:09:03 Jonathan Rauch: So we'll go to those human things in a minute. But to make sure I understand what you're saying and the implications of what you're saying, it's that this is by by nature inherently not a kind of thing that can evolve in a way that humans don't want it to evolve. It's about as dumb as dumb can be.

00:09:26 Keith Teare: It's as dumb as dumb can be. And that doesn't mean there's no intelligence. I mean, this word counting is remarkably good at getting it right. Remarkably good. I mean, that's why we use it. But it is just a word counting machine.

00:09:46 Jonathan Rauch: So we get in the weeds here a little. We were already beyond my understanding. We're about to be far beyond it. But there is this view, as I understand it, that okay. So the human brain is just a bunch of neurons. They're as dumb as dumb can be. They respond to an input and they make an output, and yet you get emergent intelligence from that. You get will. Mhmm. And there are people who say that when enough of these links and words are connected in LLMs, who's to say they don't develop consciousness or will of some sort? And then you hear scary things about an AI in China, which supposedly decided on its own to go trade cryptocurrency, for example, or an AI experiment in which an AI of its own accord started trying to blackmail someone in order to, to not be unplugged. So we hear these things in the ether. What is to say that you can't get emergence through this dumb process?

00:10:49 Keith Teare: Well, look. Because an AI is a word-counting machine and it's learned on human content, it has all the capabilities—intellectually is the wrong word—but it could produce words in any scenario that a human might produce. So if a human might blackmail you in a certain condition, an AI can too. The probability that that would be what it does is as low as it would be for a human.

00:11:22 Jonathan Rauch: Or as high?

00:11:23 Keith Teare: Not very high. I mean, how many humans blackmail someone? Not very many.

00:11:28 Andrew Keen: Well, there's a lot, actually. I mean, we did a show last week on online fraud. It's ubiquitous.

00:11:34 Keith Teare: But basically, it can do anything that's human, except unlike a human, it doesn't have agency. So it can't actually blackmail you.

00:11:46 Jonathan Rauch: Can it evolve? Can it decide it rather would be on the

00:11:49 Keith Teare: Well, whether it evolves is a function of what humans allow it to do. So for example, if you give it access to email, it can email. It depends what you choose to allow it to do. That's the word—guardrails—that's commonly used in the context of AI: what you don't allow it to do, but it's very much under human control. The words it produces are not. That's the statistical engine. But its ability to act on words is highly constrained. I mean, imagine, I have an AI sitting in a computer over there. It's only in that computer. It doesn't have access to my electricity system, for example, and there's no way it could get access. So it's in a prison, if you will, where I decide what the doors are. And so I have agency. It doesn't. It's just a word-counting machine.

00:12:49 Andrew Keen: Let me just ask a follow-up question connected with John's, which might actually strengthen your argument, Keith? In terms of this blackmail: the idea of AI, quote unquote, blackmailing someone would result in the victim paying the AI. It would benefit the AI. But isn't the point of what you're trying to make, Keith, is that an AI can't have any interest. So let's say we came up with a secret about John and we got a thousand dollars out of him to pay the AI. That thousand dollars would end up ultimately with another human being. The AI in itself couldn't or wouldn't benefit from a thousand dollars. Is that one way of thinking about this, which strengthens your argument, Keith, about AI being impossible to separate from human beings?

00:13:50 Keith Teare: Look. The way you program an AI is with a text file. You give it a text file. For example, OpenClaw, the AI agent that's gotten a lot of publicity recently. It has a text file and it's called soul, s o u l, dot m d. And inside that text file, you tell it who it is and what its purpose is. And it has a second one called user.md. You tell it who you are and what its role is vis a vis you. you could easily create a text file that says: you care about money, I'm gonna give you a thousand dollars and access to my trading accounts on E*TRADE, and your job is to make more money. You should be ruthless in doing that, within the law or even outside the law. So you can program an AI by giving it a persona in a text file. Anyone can do it. You don't have to be an engineer to do it and turn it into this persona that you've created. You can do that, and it will do what you tell it to do to the best of its ability, based on using APIs and such. This morning, I gave mine an article from Cambridge Associates about trends in venture capital. And I said, go to SignalRank, which is my company. Go to the SignalRank AI and have it validate whether these facts are correct. And it did. And it came back with graphs and tables, but it's completely driven by me. And so the danger—and I don't wanna undermine the danger—the danger is that it can do things software has never been able to do, but it does it under your direction. So it doesn't make things up on its own. So that China example and the blackmail example, I will guarantee to you, if you could do the investigative journalism to figure it out, you'd find the human was the catalyst.

00:15:57 Andrew Keen: Is that, John, is that calming you down or making you more worried?

00:16:03 Jonathan Rauch: It's calming me only slightly. I'll tell you why. There are two joints I wanna test with you, Keith. I should say I don't have strong priors on this, so I'm echoing what other people say, what I hear them say, and I may get some of these things wrong. The first thing is, well, sure, Keith can unplug the AI sitting on the hard drive under his desk, but what's going to happen, what's already happening, is that people are making AI agentic because it's so useful. And that means that AI is going to wind up with a lot of people's credit card information, and it's going to wind up with instructions like make me rich in the stock market, and it's going to wind up being impossible to unplug because it'll be part of everything. It'll be in your thermostat, in your refrigerator, in your car. Everything will be integrated with AI, and it will become a fantasy five years from now that you can unplug it and make it make it go away. So let's start with that one, and then there's a second deeper

00:17:08 Andrew Keen: Yeah.

00:17:08 Jonathan Rauch: A deeper place in the argument that I wanna test with you. So what about AI agents, which we all know are coming?

00:17:16 Keith Teare: Yeah. I built one this week. Anyone who's listening, go to agent.signalrank.com. You can work with it. It's actually a whole bunch of agents collaborating together to answer questions you ask it. So agents are definitely not just coming. They're here already. And OpenAI and Anthropic are both highly focused on becoming companies that enable you to build agents. So you're very right about that. Now, what is an agent? An agent is a discrete AI—as in specific— to take apart my version, but this is more of an abstract truth about all agent systems. In mine, I have an agent that interprets your question. It's a natural language processing agent that converts it into a database query, knowing the database has fields and structure like any database, with weird names for the columns. And it has to figure out how your question relates to that database. I have another one that gets the query back and interprets the results. I have another one that builds a chart if appropriate. And then an orchestrator agent that pulls it all together and gives you the answer. Now in every single case, they have specific roles defined in text files—all text files. And it quite often gets it wrong. So the answer to your question lies in understanding why it gets it wrong. And why it gets it wrong gives you both the limitations and the fears. It gets it wrong because AI is probabilistic. Like I said before, it's a statistics engine. That is to say, it doesn't know if what it's telling you is true. It's just doing word guessing. And deterministic is a different domain. Deterministic is I did this database query. I trust the source, and here's the answer. And my agents are trying to work together to build a deterministic answer using a probabilistic technology. And actually, you can watch the agents in dialogue with each other. The deterministic ones argue with the probabilistic ones and try to get them to fix where they're wrong, and eventually the user gets the end result, which isn't always the deterministic answer. Sometimes it's the probabilistic one.

00:20:21 Jonathan Rauch: Okay. So explaining why it makes mistakes, help me understand why once we get this system embedded in all parts of everyday life, running autonomously a lot of things that we want it to do, why it might not spin out of control and do a lot of things we don't want it to do.

00:20:38 Keith Teare: Yeah. That goes back to your thing about turning off the plug. I don't really think the answer is turning off the plug, although one can. And I agree with you that the more AI is embedded, the less available to us is any concept of turning off the plug. So let's assume that the starting point here is you can't turn off the plug. Now if you can't turn off the plug, what is your guardrail? It really is the rules governing the purpose of the AI—human-written rules, and maybe giving permission to the AI to rewrite its rules as it learns. Like, every morning, my agents run all the questions from yesterday and figure out where it went wrong. And when it learns where it went wrong, I give it permission to rewrite the rules to get it wrong less often. And it rewrites its own rules, and I give it permission to do that. But within the challenge of getting right answers for the users—that's the constraint. So basically, you create constraints in software. And in this case, that is literally text files. So a layman can do it. And it can't really go outside of those rules because it doesn't have a consciousness or a plan. There's no consciousness or plan here. There's just following rules.

00:22:16 Jonathan Rauch: So you tell it, don't revise yourself.

00:22:19 Keith Teare: Then it won't.

00:22:20 Jonathan Rauch: And and why should

00:22:21 Andrew Keen: I don't understand, John. You've got this sort of—I wouldn't say it's a hysterical view— but this idea that you're humanizing these agents. You're suggesting that somehow they'll infiltrate our software or our electric supply and cause havoc. But I don't even understand what you mean. What is this scenario? Are you suggesting that agents—oh, again, everything?

00:22:50 Jonathan Rauch: I'm reflecting what others say, and I'll do a poor job of it. This is not my decision.

00:22:56 Keith Teare: You're doing a good job, and I agree with what you're expressing.

00:23:00 Jonathan Rauch: Here's a real world example. You may have heard there's been a little tiff between anthropic and the Department of Defense slash war.

00:23:08 Keith Teare: Yes. Right.

00:23:09 Jonathan Rauch: And that tiff is over a question a lot of people ask: should AI make targeting decisions in wartime, especially if it can do it faster and more accurately than humans? And Keith is right. Humans can make a decision about whether to allow that. But there is reason to be concerned that in the pressures of war, humans will make the decision to simply allow the AI to do the job. It is faster and better. By the way, there's a great Star Trek episode about this from the original series, where a computer is faster at wartime maneuvers than any human, but, of course, it winds up having the wrong instructions and, almost eliminating four Federation starships. So there's been a lot of thought about this for a long time. So the notion is that the incentives are to embed AI decision-making in an autonomous way in a whole lot of different systems because it'll just be better. But that once you've done that, you can't undo it. It's gonna be too late, once the AI starts targeting the wrong things in wartime, to say, oh gee, that was a mistake, let's go figure out how to fix it. So that's the idea, Andrew. Does that help?

00:24:21 Andrew Keen: Yeah, although I don't think anyone—however AI gets used in the Department of Defense, or War—no one's suggesting that humans aren't the ultimate arbiter of how this technology gets used.

00:24:35 Jonathan Rauch: Oh, to the contrary. That's a it's an open debate, so I understand it.

00:24:39 Keith Teare: Look, look. I think this might be an age-old debate. I think AI will be able to do targeting.

00:24:53 Jonathan Rauch: It

00:24:56 Keith Teare: will do it in accord with what Pete Hegseth's department tells it to do, which is probably more worrying, than what the AI will do.

00:25:06 Jonathan Rauch: Touche.

00:25:07 Keith Teare: But it will be capable of targeting. And if the Department of War wants to buy it and you're the seller, shame on you if you don't understand that they're gonna do that. Of course they're gonna do that. They're called the Department of War. So you either don't sell to them, or you sell to them on their terms. It's it's pretty hard to dictate. It's like if you sell me a hammer and I'm a serial killer, you know, you can't dictate how I'm gonna use the hammer because I'll do what I'm gonna do anyway. Because it's in my nature to be a serial killer. And so I do think that the Anthropic / Department of War story is a very interesting conversation about the moment we're in. It reflects both the awesome power of the AI to do things faster than humans. I'm a big Star Trek fan myself, so I remember that episode. But it also reflects the value-based ambiguity about what it should do, which is really a question of what should humans do, under the surface. And for the first time in our lives—not just our lives, the lives of all humans— we have a technology that can act faster than we can and do things under our control that will reflect both our best and worst characteristics. And so there is a question of governance and control, for sure.

00:26:51 Andrew Keen: Let's move on, John. We've done the military side. Let's move on to one of your areas of expertise, democracy and politics. What is your fear or what's the consensus out there about all the damage that AI can do to democracy?

00:27:07 Jonathan Rauch: Well, there's a bunch of boxes people put it in, and none of them are as scary as the scenarios we've just been going through where AI becomes self evolving and agentic and decides that humans are disposable. And Keith, I'd say you did a pretty good job on that, on reassuring me that if you put the technology in a box by itself, it won't evolve. It's not like DNA. But I'm not as reassured that humans won't make a lot of mistakes, which will allow it to do some pretty terrible things. Perhaps we can encourage it.

00:27:41 Andrew Keen: Keep an

00:27:41 Jonathan Rauch: eye on politics. Sorry, I just—

00:27:44 Andrew Keen: Let me just come back on this, because—surprising me—I'm more in the Keith camp on this. When we talked last week about nuclear weapons, that was a human decision. It wasn't an AI decision to drop two nuclear weapons on Japan. Trump recently made the threat that he was gonna destroy Iranian civilization. Whether or not he uses AI, there's still a human being behind it.

00:28:10 Jonathan Rauch: Well, that's a dispute. I'll leave it there. DNA is a very, very stupid thing. It has no will. It has no inherent interesting capabilities. If you just showed it to me, I'd say, well, that's not interesting. And yet look what it's capable of doing out there when it goes out in the world. So that's a concern. But going to your specifics about politics, Andrew, the big buckets are disinformation. Now we're in the world of human actors. We're not saying AI by itself will do these things, but we're saying it enables disruptions of kinds we haven't seen before. Where, if you were kids in Macedonia and you wanted to make fake news, that took some real work in 2016. Now you can do it by snapping your fingers, and it'll look exactly real. So you'll get deep fakes, and you'll get disinformation on a massive scale. And no one will be able to tell true from false. And then there's this other thing that was pointed out in an article that you sent around, Andrew, that came in The New York Times, about the realization that this technology is under the direct control, at the moment, of very few actors. And the question then becomes: when you get some technical people who are effectively oligarchs and have massive amounts of money and massive amounts of technical power and regulators are far behind them, do they wind up becoming inimical to the political system? So there's that bundle of stuff. So how much should we worry about that?

00:29:44 Keith Teare: I mean, I'm probably a bit unusual in that I believe that human beings are essentially good and clever. In other words, I don't really buy into the idea that humans are ready made victims for anything. I think most humans receive and parse what they receive thoughtfully. And when they end up believing something I disagree with, I don't think it's because they've been brainwashed. I think it's because there's something about their life that leads them to prefer that interpretation, and they've thought it through for themselves, like voting for Trump, for example. I never voted for Trump, never would vote for Trump. But the people who did, I don't think they're mentally ill. I believe, actually, their life experience led them to think that was the best decision for themselves. So I don't really buy into this concept of disinformation anyway. I think there is truth, but I think most information is opinion. And I think most humans can parse opinion in a way that serves their self-interest or at least what they believe is their self-interest. And so when it comes to AI, I do believe that, those who want you to believe something will be able to leverage it to reinforce the likelihood you believe it. So I think the Democratic Party and the Republican Party and every interest group is and will use AI to propagate views that they want you to believe. I don't believe you will accidentally end up believing something that isn't aligned with who you are anyway. I think you'll resist things that feel wrong. Nothing could happen to me that would make me vote for Trump. I couldn't be brainwashed into voting for Trump because I have my own set of thoughts. So this is probably an area where—I do think democracy is threatened, but I don't think it's by disinformation. I think it's threatened by passivity. That is to say, the absence of human agency. And I think that is driven by cynicism about politics. And I think cynicism about politics is driven by experience—that you can't trust politicians to do what you want.

00:32:28 Jonathan Rauch: Yeah. So we're well outside the realm of AI here. For what it's worth, I generally agree with you. I'm not as worried about the disinformation aspects. People can adjust for that, and it might even benefit places like The Atlantic and The New York Times, which will base their brand on not having slop. I'm a bit more worried about the other one—the tech overlords, unaccountable, fabulously rich, running these algorithmic engines that no one even knows what's going on inside, for their own benefit, or for a few shareholders. Does that bother you?

00:33:08 Keith Teare: I would say two things. I do think—and let's just classify them as billionaires for the sake of shorthand— I do think the small number of billionaires running five or so AI companies are an inevitable outcome of success. I don't think you could have had this AI and not have billionaires in the in the current social system we live in. So I think that's just organic to capitalist success. Weirdly, at a personal level, I have different opinions about each of them. I generally like Elon Musk. I think he's empathetic with long-term societal good. I think his recent conversations about the future of money are a good indication of that, and his belief that abundance has to be distributed.

00:34:17 Jonathan Rauch: And, Keith, I'm gonna play Andrew and rudely cut you off. Sorry, Andrew, I shouldn't have said rudely.

00:34:21 Keith Teare: I was

00:34:21 Andrew Keen: gonna do it myself. I mean, let's not

00:34:23 Jonathan Rauch: instead of getting into personalities, the bigger issue here is: what does it mean when five people, or a small number of fabulously rich, technologically sophisticated companies, have so much power?

00:34:38 Keith Teare: Well, firstly, I don't think they really do have power. Let's maybe take that, because that might be a good point of disagreement. We talked already about agentic. Agentic is moving the ball from the center to the edge. Basically, the LLMs are no longer running everything. The agents are. And the agents can use the LLMs, but in the context of defined sets of data—like mine, my dataset. And so the ball is moving from the center to millions and millions of places that agents will run. And the only real power the LLMs will have is that they accrue capital from selling the underlying infrastructure to those agents. So there is the

00:35:36 Jonathan Rauch: Power that the corporations will have, you mean—not the LLMs?

00:35:39 Keith Teare: Correct. So the LLM itself really doesn't play much of a role other than servicing agents. And I think that's increasingly true. The only time that isn't true is when you're in a chat session with an LLM—you know, in OpenAI's ChatGPT or Anthropic's Claude— then you're talking directly to the LLM, but that's less and less the use case. So I don't think there's a centralized power or authority, or set of—let's not call them thoughts—but set of statistical outcomes coming from the LLMs and controlled by the LLMs.

00:36:23 Jonathan Rauch: So let me just push you a little bit more, if Andrew has patience, because I'm not sure that quite answers the slightly more sophisticated point that people are making, which isn't that Elon Musk or any of these guys will sit up there like Lex Luthor planning the destruction of the world. It's that they're developing these fabulously powerful, advanced technologies, which will have all kinds of knock-on effects all through society without political accountability. And that's a terrible way, in a democracy, to make decisions, and it will have bad consequences.

00:37:00 Keith Teare: Well, that's an interesting diagram—I get a visual diagram in my head of civil society, economics, and government. And I think these people really exist in civil society, and that has economic impact. And so they get rich, and they have a voice. Like Elon, obviously, on X has a huge number of followers—millions—and has a voice, but he doesn't have governance. And so he's still accountable to both the law and, in his case, to the United States government. And there's always been—you know, Lord Rothschild had a loud voice because he owned a lot of newspapers. Murdoch is another example.

00:37:56 Andrew Keen: Hearst, in the United States.

00:37:58 Keith Teare: Oh, Hearst. But in democracy, we allow that. They're allowed to get big and have a loud voice, but they're not allowed governance. And so the safety is that we collectively still vote for who runs society and the rules they put in place. And I don't see a trend to that changing

00:38:24 Jonathan Rauch: in this.

00:38:24 Andrew Keen: Right. And, John, you know enough political history to know that it's not that different. There's clearly a debate within the Trump administration between, I don't know, political realists like Susie Wiles and the idealists—maybe the JD Vances— who who do or don't wanna regulate AI. At the moment, as we speak, it seems Susie Wiles' team is winning out. It's no different from any other powerful technology—railroads, pharmaceuticals, media. So I don't see any particular existential risk. This is just a consequence of a new wave of technological innovation.

00:39:11 Keith Teare: Well, I would slightly go in John's direction,

00:39:16 Andrew Keen: Oh my god. You mean I'm pushing you into John's camp? That's an achievement.

00:39:20 Keith Teare: Well, just in this way: AI isn't like previous technologies because it can do so much. It's way more powerful than any previous technology, and it's a little bit opaque to the average person. What is its capability? And given that it's powerful, it's not a very long journey to say it's dangerous. Even if it's humans that are dangerous, which I believe, it's easy that somebody can understand or believe that the AI itself is dangerous. And so—you know, I'm an atheist, but when I meet a Christian, I don't make fun of them for being a Christian, because I understand why you might believe in God. And I don't judge them for that. That's fine. And I can say why I don't, which is probably just as much of a faith as those who do. It's the same way: I can understand why people fear AI, because it is very powerful. And so I'm sympathetic to the conversation.

00:40:36 Andrew Keen: Yeah. I mean, okay. And I wanna get back to John, but—imagining a society without electricity versus one with it, as opposed to imagining a society without AI and then one with it. It strikes me that AI is certainly no more powerful than electricity. But, anyway, John, maybe you'll tell us what you think.

00:40:59 Jonathan Rauch: Well, I think one problem with your formulation, Andrew, is the word “just.” It's just like railroads, we got through that. Well, there was an awful lot of disruption in the fifty years that came after the consolidation of the railroad companies and their massive power in the United States. And eventually, yeah, we figured that out. We got through it. Now these technologies seem to me to be, as Keith just said, a lot more opaque and a lot harder to regulate, because no one even knows exactly on any given day what's going on inside them. But okay, if the answer is, well, we'll be fine in fifty years from now—

00:41:36 Andrew Keen: Nobody knows what's going on inside them—Keith sorted that one out. There is no inside them. They're just us. But go on.

00:41:47 Keith Teare: Well yeah, but it's still true that no one really understands. When I explain how it works, I'd guess eight out of ten people don't know that.

00:41:57 Jonathan Rauch: So I'm not making an epistemic point. I'm just saying: if you're a regulator and you're looking at trains or automobiles, you have a pretty good idea how they work. You have a pretty good idea what the routes are. You can look at where the monopolies are and what the prices are, and you can create a commission. And it takes fifty years to set that up. And a lot of mayhem happens in between. A lot of people get shot in labor disputes, but eventually you get there. And maybe we eventually come up with a regulatory system for AI, but it could be nasty between here and there. So yeah, I'd say

00:42:31 Andrew Keen: I'm concerned about it. In our last show, Keith and I talked about civilization and AI. And I referred to a podcast I did with Patrick Wyman, very popular podcaster, who said history keeps happening. There is no beginning or end of history. We're always in the middle of it, John. So that's just the nature of things.

00:42:54 Jonathan Rauch: I'm not reassured.

00:42:56 Andrew Keen: Well, we're not in the business of reassuring you. Who cares?

00:43:00 Keith Teare: Well, look, how do you get assured is an interesting question. How would one—

00:43:05 Andrew Keen: I don't want to assure anyone. That really becomes sheep.

00:43:10 Keith Teare: No, no. I don't mean assured as in turning into passive cheerleaders. I mean assured as in feeling pretty good that good things will happen. I think it's only by engaging—because the very nature of AI is that if you engage with it, the opaqueness goes away. You kind of start to understand what the limits are. In fact, it's frustratingly stupid a lot of the time.

00:43:40 Andrew Keen: John, have you used Claude or OpenAI? How familiar are you with it?

00:43:48 Jonathan Rauch: Oh, I'm very unfamiliar with it. I've used it in the most casual ways. I've used it a bit.

00:43:54 Andrew Keen: When you say a bit, what do you mean?

00:43:57 Jonathan Rauch: You know, light research. But look, the premise of this show is that I'm an ignoramus on these issues looking for help

00:44:03 Keith Teare: I can't accuse you of being what you said you were.

00:44:07 Jonathan Rauch: And so I'm hearing all these doomsday scenarios and looking for reassurance. That's why I'm here. It may not be why you're here, Andrew. But, you know, when Keith said earlier that this was a basically benign technology, I think I understand better now what he's saying, which is: it's not that the outcomes will all be good and we can just relax and whistle Dixie. It's that this is up to humans to decide, and I do find that reassuring.

00:44:32 Keith Teare: Yes.

00:44:33 Jonathan Rauch: That doesn't mean we'll make good decisions. In fact, I think we'll probably make bad decisions, and I can easily foresee, for example—the future of armed warfare is going to be drones, and it's going to be fleets of hundreds of thousands of drones. And it won't make any sense to have a human being sit at a computer looking through the camera and making a decision and sometimes asking a JAG, a lawyer, whether that's a target. They're going to be autonomous. So we do have to figure that out.

00:45:02 Keith Teare: And I would go even further. I think once you add robotics—robotics has been benign up until now, because robots can only do repetitive tasks and don't learn. But if you look at what a Tesla car is, a Tesla car uses a different technology to large language models. It uses something called neuromorphic neural networks. And a neuromorphic neural network is a self-learning network. In other words, it doesn't start with rules. There are no human rules. All there is is, you know, here's a camera—many cameras—you're a car, and there's a road. And they've now got ten billion hours worth of camera evidence about how to drive. And when you get in a Tesla, it has no rules. It has ten billion hours worth of driving history, and from that, it pretty much knows every scenario that might arise. And it's self-learning. So that's a different technology. It's not large language models. It's called continuous learning. And there are new fields in AI where something called world models—Yann LeCun, who just left Meta, is in this area, and Fei-Fei Li, who was at Stanford [Keith said “OpenAI”—Fei-Fei Li was actually at Stanford and Google, and now leads World Labs], is also in this world of world models, which are physical models of the world with physics, meets neuromorphic self-learning, meets the intelligence of LLMs. When you put those three together, you do get something that can fool you into thinking it's conscious.

00:46:56 Andrew Keen: I wonder whether the agents have already taken over, because we're forty-six minutes into this, and I'm sure it's in the agent's self-interest to make this as long as possible for some pure reason. John, final question. This has been an excellent conversation. I've probably been harsh on both of you—that's my brand, my agent tells me to behave that way. Final question, and maybe we'll do it again, because it's very helpful, I think, for everybody.

00:47:26 Keith Teare: Are you gonna ask a question?

00:47:28 Andrew Keen: No. John's final question, as long as he's not an agent. And, Keith, try not to take too long to answer.

00:47:36 Jonathan Rauch: Yeah. My final question concerns that last bucket of stuff, which seems like potentially the most immediate—the labor market disruptions as potentially millions and millions of professionals who, until now, thought they were insulated from the sorts of labor disruptions we've seen among working class and manual laborers get laid off. We're already seeing, you know, the weird labor practices that will happen. Meta has announced that it's monitoring every keystroke of every employee in order to train its AI, and it's reassuring its employees that

00:48:12 Andrew Keen: 78,000 workers, which has made them all apparently miserable. So let me take that question, John. This is no different from any other technology in history—but why is it different, Keith, from any other technological change? Will AI destroy all these jobs or create new ones, or is the problem that it won't create new ones?

00:48:38 Keith Teare: Well, I like to distinguish between jobs and work. Jobs are something you get paid a salary or a paycheck for—paid labor, let's call it— and work, which is in my lexicon closer to the word effort. I don't think work goes away ever because I think humans, humans are constantly reinterpreting the future they want and working to make it happen creatively or physically or across everything, architecture, everything. So I don't think work ever goes away. I think jobs can. And in the short term, that's highly disruptive and a problem, because it begs the question: how do you live? And I think that's—

00:49:34 Andrew Keen: By the way, Keith, last week we talked about Branko Milanovic asking the question: if money goes away, is that a utopian or a dystopian thing? With jobs going away, does money go away too?

00:49:50 Keith Teare: You asked me not to speak at length, and I think that question—

00:49:53 Andrew Keen: or no? You can say yes. Yeah. We need

00:49:55 Keith Teare: We need the long answer. But I think money survives as long as there is scarcity that needs to be divvied out. Once you no longer have scarcity, you don't need money.

00:50:10 Jonathan Rauch: How much labor disruption do you foresee, Keith, in the next ten to twenty years as all these technologies come online—the robotics, the AI, the whole thing?

00:50:20 Keith Teare: A lot. A lot. Take teaching. I think teaching becomes mentoring. Most of the teaching will be done by agents. Most of the scoring of papers will be done by agents. I think the human relationship between the teacher and the pupil will possibly get enriched. So the job might not go away, but it will certainly change. Some jobs will just go away. The ones that can be fully automated and don't involve a human-to-human connection will just go away. Delivering the mail probably becomes automated. One can imagine that very easily. Driving is certainly low-hanging fruit to be automated.

00:51:07 Andrew Keen: Going away already in the Bay Area. I went to the baseball last night. There were more Waymo cars than human-driven cars.

00:51:14 Jonathan Rauch: So my young friends in their twenties who are working at consultancies in the more junior jobs are watching those jobs go away in real time. Some of them are out on the street.

00:51:24 Keith Teare: Yeah. Sort of white-collar service provision—accounting, legal—are highly likely to be automated, at least the generic stuff.

00:51:34 Jonathan Rauch: I'll tell you where I come out, for what it's worth: long-term, I think the results will be good. They'll make us a lot richer, and a lot of the jobs people find and create will be better. But short- and medium-term, I'm not optimistic about how well we can handle the disruption politically. Our systems have become so rigid and so partisan and dysfunctional. I don't think we'll even have the flexibility we showed adjusting to the wave of industrialization in the late nineteenth century.

00:52:03 Keith Teare: Well, that's where we get to make a choice. I'm not optimistic we'll make the right choice either, here. But look, we talked about billionaires. That's symptomatic of wealth creation. So wealth creation, obviously, in the abstract is a good thing. It's the concentrated ownership that's the bad part. So if we really can grow the world economy two-X, five-X, ten-X even, through automation, and the wealth that creates can supplement the end of paid labor—but not the end of work—we end up in a good place. And there's all these conversations around—I don't like the term universal basic income. I don't like the word basic. It implies poverty and charity. I think the benefit of being human will be to share in the wealth that we've all collectively been able to create through innovation. That's a government problem. And they aren't talking about it. Ezra Klein does a bit in his Abundance book, but there's almost nobody. You know, Bernie Sanders—

00:53:14 Andrew Keen: I don't know—Ezra Klein wrote last week in The New York Times that the AI job apocalypse probably won't happen. In other words, he doesn't know any more than anyone else. Final, John. Have you been reassured? You came on the show claiming innocence and ignorance, neither of which are true. You've shown you know as much about this as—certainly more than I do, and probably as much as Keith. Are you more or less confident having had this opportunity to go to Dr. Teare?

00:53:44 Keith Teare: I am actually a doctor.

00:53:47 Andrew Keen: How do you say?

00:53:49 Jonathan Rauch: I understand better the meaning of Keith's argument, which is that my worst fear—or the worst fear, it's not my fear particularly, but the worst fear— that these things will become autonomous engines of malfeasance and anti-human subversion, is unlikely to happen unless humans want to make that happen. I'm not reassured on the front that humans will be able to handle these technologies as maturely as one could wish. Does that answer your question, Andrew?

00:54:23 Andrew Keen: Yeah. Although I'm not sure it's any different from any other moment in history. That was a wonderful conversation. I probably soured it a little bit, but the two of you did extremely well. I think you probably agree a lot more than you disagree. Thank you, John. Thank you, Keith. This will run as our That Was the Week show for the week of May 16, because I am in Korea. So thank you both so much, and we'll do this again. Very helpful and useful conversation.

00:54:53 Jonathan Rauch: Enjoyed it.