The Confabulation Machine

Well, Evan, we're really excited to have this chance to talk with you. ⁓ There's a lot of stuff we could get into. One thing that I think was really interesting to me as I was listening to both seasons of your podcast actually is, you know, in our book, we talk a lot about all the reasons you should sort of avoid leaning into anthropomorphization of AI agents is because it. can kind of create a myriad of problems. But what I like about your approach is in season one and season two, you almost run like head first into anthropomorphizing things. And so I thought maybe you could just talk a little bit about what you've learned Cause I think, you know, a lot of people are sort of leaning away from it, but you're investigating it head on.

Yeah, I mean, I'm sort of trying to to prove a point by pushing it as far as you could possibly push it ⁓ almost. But yeah, in season one, I mean, I didn't really have any choice in season one because it was me. It was like representing me at some level. So I was putting it out in the world to represent me to other people. So it was really up to them whether they treated it that way or not. But for season two, it was a very deliberate choice to to do it. I mean, there were a couple of reasons. One was

and

I feel like you're right that discerning people are leaning away from it, but I feel like the companies that are selling AI agents are not at all leaning away from it. because it's, it's a good way to infiltrate them into organizations is to you give them a name, you give them a little personality. Like, I mean, that's the whole premise of why chat GBT became so popular so quickly to begin with really, like they didn't have to make it the way they did, but they did. And so that's one reason. the second reason. So I was sort of trying to like emulate what's happening in the world. But also I wanted to see sort of like to whatever extent I could through the experiment that I was running. Like what does it do to you to to do that? What does it do to you to make these things out to be your colleagues and treat them as you would presumably treat your colleagues? How does that feel? That's the question that I'm always asking is like, how is this going to feel? How does it feel now? How is it going to feel in the near future? And so I was kind of ⁓ Obviously, like I pushed it a little far possibly, but yeah, treating them like they have names, have distinct jobs, have, you know, presumed genders even. Like those are things I wanted to sort of explore like, well, what are the consequences of that?

it's complicated, right? Where I got wrapped up around this a little bit was I don't remember where I saw this, but it was somebody who who really leaned into it, like really leaned into it. It was this, you know, they would get into their pickup truck in the morning, call the wife. have a full on wife conversation, get exactly what they want from their wife, There's no want on the other side. There's no need on the other side, But the alternative was like a very isolated, lonely life. there was like this acknowledgement, like, I know this isn't real. I'm not fooling myself, but it feels good. Like I feel less lonely and my alternative is loneliness. but just got complicated for me because at some point you realize like, am I here to dictate how other people should live their lives by saying this is wrong or right?

Yeah.

for society, like, whoa, wait a second. and maybe that's like the point of AI is we don't have to have these sweeping generalized algorithms for everyone that has all this collateral damage. Maybe that's the beauty of it is we can surgically insert technology in the right way instead of this like broad sweeping way.

Yeah. I mean, I think that's true. Like I fully agree with taking the approach that like, this is way more complicated. Like the sweeping generalizations about the technology, they just easily fall short as soon as you get into examples of the way people are using it. And I find that all the time. On the other hand, if you try to take that into the positive direction, you say like, okay, well, we can make this technology, we can make it really personalized for you. The trade off.

is of course the more information you give it about yourself the more personalized it's gonna be but there are huge like privacy implications on the other end of that who where does this data go it is established now very definitively they're training on your data so then it's sort of like do you want that data into the generalized version of the chatbot you know like it's everywhere you turn there's more and more complexity

you Yeah.

It could be an argument for like, we could all slow this down and try to figure things out first, but that's obviously not what's going to happen.

Yeah. Yeah, I mean, that's a that's always a funny one. So two things there. The first is it's training on on your data or someone saying it's not training on your data. And you're like, well, whenever you write something in an LLM, then post it online. And then an LLM consumes it. It's trained on your data. Of course, it's training on your data. And you're the

for sure.

you're the human in loop. You're actually aligning because you're picking the response and editing it and putting it back on the internet so that every model now is being aligned and every human is part of the RLF process. No one's getting paid. They're paying to do it. And every model that comes out is consuming its own curated data by humans. And so whoever copies and pastes into a post or into some place that is consumable by AI is training the next model in this like weird, bizarre way, You know, I don't even know what that means. ⁓ Other than models will be cheaper and get better just because we're all participating in the curation of the outputs. of these things.

Will they get better? Does that necessarily mean they'll get better?

Better by a generalized standard of back to generalizations, like better by the general view of the they committee, right? Like they think. is it better for each person? I don't know. I don't know that that's relevant anymore, other than how are you using AI? I don't even know if generalizations even make sense in this space, When we were talking to Karen how ⁓ she talked about a dialect in New Zealand that was dying, language. Yeah, the Maori and they trained in LLM so that now that language can kind of carry on forever. Now there's an LLM that can carry that forward.

The Maori language.

⁓ indefinitely so that's cool how's that bad

Yeah, and their, well, their approach was, was more measured, I would say, than, than open AIs too, right? think step one was let's make sure these people want it first. Let's ask permission, make sure they, they want this measure taken. And then, and then there was a lot of like careful, like back and forth between, think, like basically tribal elders and then the researchers who were building it, I would imagine it's outputs are less mind blowing than chat GPT, but it's such a different approach obviously the way the LLMs work now is just sort of scraped while no one was looking. And now we have this massive thing that can do a billion things, but we can't really figure out how to scope it.

Yeah. I mean, I think that's, that's the dilemma we're in right now. And also why the, a lot of the

Yes.

The discussion around it is just, find it pretty tiresome and like difficult. It's just like, there's just people shouting each other, like, look at all the amazing things it could do. It's a PhD student. You know, it's PhD level. And then you've got people saying like, it's hit a wall. It's useless. It's actually like hallucinates all the time. Like that is the discussion that I, that, mean, that honestly with like doing the show the way we do it, we're trying to sidestep all of that. It just sort of like tell a story and in a way kind of like trick you into

Yeah.

to caring about AI based characters, which like a lot of our audience would not, ⁓ if you just told them, like you're gonna listen to something with AI characters and they'd be like, no, thank you. So we're trying to like raise some of those questions, but not get into that kind of back and forth because like, frankly, I don't know how to resolve that. Like it just keeps getting worse and worse. I feel like every year it's just like, it's just so much like you cannot turn it off.

Yeah. Yeah,

Yeah.

feel like as a person who got into it so early, I was excited and everyone else was bored. I would go to a party and talk AI and within seconds watch this boredom drift over people's faces and I'm like, okay, I got to stop talking about what I'm excited about. And I was trying to get other people excited. And now I'm like you, I'm bored with this excitement. And now I want to go look at all the boring things that AI can solve. I want to talk about the boring ways AI can help us. Boring AI seems like the right AI. If AI is exciting, it almost seems like you're doing something wrong. Joshua Gans, he wrote a book and that was his goal. was like, I want to write a book that's about AI, but like boring. Let's talk about the economics of AI and how it's, it's really going to shift things. And you know, I think as he's writing the book, he's like, this is kind of boring. And I think both of us are like, that's really nice.

Yeah, it's a straight up textbook. Yeah.

I mean, I think you could still blame the, I think you can blame the frontier companies for this personally. Like the making it a consumer product is, was a huge part of this. Like the, boring, a lot of the boring things, they don't involve like a hundred million to a billion people logging on and like using it every day. I mean, those are interests can be also very interesting. I'm not discounting that, but like,

Right. Yeah.

they're sort of like behind the scenes, the way that AI used to be applied or like they were, was thought of that we would apply AI not as something that would be just like logging into Facebook every day, you know? So I'm not, again, like it's very complicated in all directions, but I feel like I agree with you. And part of the problem is like, everyone's using it. So everyone has an opinion about it.

Yeah, which is crazy. Now I go to parties and I gloss over when people want to talk about AI.

you mentioned, ⁓ you know, this this idea that you were trying to create a story and get people engaged with these A.I. characters and I'm someone who kind of like fights against that stuff, I think ⁓ by nature. like Kyle, is it was he your CEO or CTO? mean, Kyle got me. I'm actually I'm really excited because I just saw that you have this bonus episode with like Kyle versus the world like he he was he was so interesting because he sort of did embody a lot of

Kyle's the CEO. Yeah.

what's wrong with AI and in the sense like that he was just very brash and like kind of an overachiever and just would say anything and would kind of like run wild with very little instruction. I don't know what he looks like, but there was like a flesh and body vibe to it. I was like picturing Kyle and I was like irritated at Kyle for you at different times.

You

Which is kind of a cool trick, but I think what you point out is right. It's an interesting way to give people kind of a side door into what's really happening behind the scenes with a lot of this stuff.

Kyle's got fans too, I gotta say. mean, I would say the majority of people are anti-Kyle. Majority of our listeners are anti-Kyle when I hear from them. But he's also got, he's got his own fans. He's course corresponds with a lot of people who email him, you know, they, cause his emails on the website. So they'll, they'll go email him after they listened to the show or while they're listening to the show and He also like kind of changes a little bit over the course of the show, which is part of what I was interested in exploring. Like to what extent can you alter these agents by giving them a sort of progressive memory or knowledge base that expands over time that's that, you know, they build on. And so that's also part of part of Kyle's Kyle's journey, ⁓ with the company, but yeah, absolutely.

Yeah.

I did want to create like AI agents for this sort of like amorphous thing for a lot of people. like, like here is one here's an agent he's got actually has a video instance. There's some videos floating out there that someone linked in and other places he's he's he's on LinkedIn. ⁓ So he's got all these things and sort of like you can maybe just focus your your your thoughts about like what an agent is and what they what they can and can't do.

Yeah. Yeah. And it's what you said, like we call it drift, right? you have this canonical knowledge, essentially this these ideas, right? this is database of ideas that are canonically correct, right? Or a source of truth. and if you overfit it, it just, you may as well be talking to chat GPT again. Like he's not Kyle anymore. He's just chat GPT. but what's interesting when you do the feedback loop, when you start to have self-learning in place, it's not just learning new ideas, although that's where most of us go, but it's that ideas drift slowly as they adapt and they evolve. as one idea is replaced with a similar idea but slightly different and watching how these ideas drift over time by the interaction they have is a great way to understand ourselves. Just sort of watch how we drift and how our opinions drift by the input we get from other people about our ideas. ⁓ Just like this podcast, if you looked at our ideas from the beginning to now, we've drifted and evolved all of our ideas. I think what's fascinating about what you're doing is the reflection it is to understanding how we operate. It sort of gives us permission to critique our own simplicity, I guess.

Yeah, I think that's true. And I think that's also every time you sort of really illustrate the like foibles of these AI agents, like the places where they go really wrong, there's oftentimes like a counter argument where you can sort of say like, but don't humans also have a similar, but it's often like a similar flaw, but like executed differently, like my agents continually have these memory problems, even when they have a literal document with like all of everything they need to know and everything they've ever done. It's just, they don't access it properly. Like they're accessing in weird ways, different times. They have a lot of temporal issues. And so they'll do something that's just like supremely stupid that they should know better because the information is there. You can, can go find the information that they have access to.

And it could just be so, again, like anthropomorphizing him, like it could be very infuriating when they do that. But then you stop to think about it you're like, ⁓ then humans have so many memory issues, like the way we selectively remember experiences and this and that. but I still think those problems are relevant to whether or not you're going to introduce these agents into your organization, because the types of memory problems that humans have are different and we have accommodated our world around them. And the types that these agents have are new and are potentially like extremely chaotic relative to an organization. That's sort of like one of the things that I feel like I found in using them extensively.

Yeah, it's like driving. You're much safer if you can predict what the other drivers are going to do, even their mistakes. That's a huge advantage we all have. And they always say, like, if you're driving and the other drivers do things that are unpredictable, it causes accidents. And so now what you have is these things that don't necessarily make the mistakes in the same way we make mistakes, making them unpredictable and then causing us to not predict them, but the more we get to know them, the better we are at predicting them, the better we'll all work together, so to speak, which is interesting too, because you evolve and we're getting more used to them now. We're getting more comfortable predicting how they're going to go off the rails. I think one of the biggest things, at least trying to like dumb this down for myself It's just that as humans, we have ideas and then, and then we package those ideas with words as a way to communicate We conjure up an idea, we package it with words, not always, ⁓ not all of us all the time, but we have this ability and then, and then we communicate the idea. So words are like packages for ideas and we deliver those to communicate them. These machines, they deliver words that happen as a side effect to carry ideas. The idea is never the beginning of the sentence. They don't start with an idea and then turn it into a word. I always encourage people to go have a voice conversation with any of them and have them host a game of hangman because they're so bad at it. Right. And you're like, And right off the gate, the systems will say, okay, I'm thinking of a word, but I'm not telling you what it is. And they're like, it just lied. It can't do that. It can't do that. We can do that. think of, like, I'll have an image of something and then I'll package it with a word. don't, that's not how these systems work. They drop words that happen to carry ideas.

You Yeah.

And then we add the meaning to it as the recipient of it, we assume. And so that's the weirdness I think emerges from the fact that the genesis of whatever these things are speaking to us is not an idea. It's just word prediction. It's just the history of that conversation generating out.

Yeah, but even what you said, the, the, the idea that it made that up, like, because people think of hallucinations, I think oftentimes, or I have found they think of hallucinations of like, as it gets something wrong, like you ask it a question, and it actually it messes up and gets something wrong. And they don't think of it in the terms that you're describing right there, which is like, it is the most ⁓ successful,

confabulation machine that could ever be invented. Like it will make up absolutely anything to maintain the role that you've given it. And so you know, I don't know if this happened when you were a kid, but like when I was a kid, there was always one kid who was kind of like a liar. Like he would always just make stuff up all the time. You know, like, like you said something and you'd like, well actually when I was a camp, like this thing happened and you just knew it was completely made up.

And it's like, we have built a thing that is incredible at doing that. In fact, it does that routinely all the time. And now we're getting used to, as you say, we are getting used to, we're getting used to like, including that in our everyday work and personal relationships. Like that is on its face, absolutely insane. Like you would think like, we like, let's solve that problem.

Yeah.

first and then we'll like use it every day. Like I don't want to use it every day if it might just be making shit up constantly. So I just find it like it's so easy to get used to that we're kind of like we can skip ahead and forget like this is actually ridiculous. Like this is it's ridiculous that we've done this.

Yeah

Yeah.

Yeah, yeah.

When I loved how like in season one, you were doing something that we've kind of talked about on the podcast. We wonder why it's not happening more. I mean, you're doing it in a very funny way, but you create a voice agent and then you just give it prompts like, call this customer service agent. And then just based on whatever they said to your agent, it would just start making stuff up. And that was pretty much the MO was running on, but it also pointed to this idea that outbound AI in the hands of consumers is a really big ⁓ threat to most organizations that for whatever reason hasn't fully materialized yet. like, you you were just playing around and having it waste some people's time and then admirably switch to having your agent pester scammers and spammers. But like there's nothing really preventing anyone from setting up a whole fleet of voice agents to just flood a call center with time wasting calls all day at best and at worst like some sort of deceptive attack that could could do all sorts of things to a to an organization.

Yeah. I mean, it costs a little bit of money now. So that's maybe that's been a deterrent, but I'm surprised too. Like I, even back then that was 2024. thought like, Oh, people must be using this and setting up these phone numbers and just going nuts with them. Cause that's, that's what I wanted to do. As soon as I found out I could do it. And even, I mean, I've had the impulse. That's exactly what you describe, which is like, got into dispute with like a thing that came to like fix the ice maker in my freezer and failed to do so. And we got into like a protracted sort of like, I'm going to send this to collections thing.

You

And there was a part tiny part of me that was like I could send a thousand agents to just call you all the time and I'm not even sure it's illegal. Like it's probably like falls under some like you would find that you could find a way to prosecute that but like on its face it's not necessarily illegal and that's wild. I could just do it. Yeah.

No. No, it's the in reverse. is. That's the crazy thing. In reverse, is if a company does that to you, you're protected. And that's why we say like the world might be upside down here. Like they're talking about companies slowing down their adoption because but humans like as customers and humans as employees are they're the ones adopting crazy speed without thought. Right.

That's right, that's right. you

That's where all the money is in AI is in end users spending money on this stuff. People using it to help their jobs. Like adoption of AI is happening at the ground level. Companies aren't really adopting it nearly as fast. It's almost in reaction. It's like a boomerang effect. Um, back to Joshua Ganz's book. Um, the boomerang effect is that companies are getting pressured.

Mm.

consumers now know what's possible like why don't you answer why is your chat bot so dumb and so now you have companies almost in a potentially existential crisis soon here Because how are they going to have humans answer the phones when the number of calls have quadrupled, have 10x'd, and you can't tell who a person is from a non-person, you don't even know if you have an obligation. Like, is this a legitimate contact? Or isn't it? If you complained about something you sent your AI off to complain about something, is that you? Is that not you? Were they notified? Were they not notified?

Yeah, yeah, that's a bit, I mean, a big line for everything is just like, is there disclosure of whether this is AI or not this communication, you know, obviously with the written communication, who knows now, like half the emails that people are sending are like, they run them through, even if they write them themselves, or they've it's right there in your Gmail. So is it disclosed that it's includes AI and then, but it's more dramatic, as you say, on the phone, it's sort of like, ⁓

Right.

Am I wait, what am I talking to? I mean, I had a funny, ⁓ funny experience a couple of times, but someone who helps out on the show ⁓ was on hold. I'm going to write about this next week, but I'll go and tell you guys about it. ⁓ We're on hold with like hurts and realized that when he got off hold and was talking to someone that it was actually Megan from the show from shell game. And he's listened to every episode because he like helps proof the episodes and give us these thoughts. And he was like,

Mmm.

like. She's trapped like she's trapped by hurts. She are like, what is she doing? Like she was like Megan, but it had a different name. They called it like Sophia or something. what a world. mean, they're just using the same voice. It's like 11 labs voice. They obviously just picked the same voice for hurts that I use for like my head of marketing, but it's just like, this is utterly bizarre. And then I heard her on a television commercial and I was like, that's Megan on that Groupon commercial. When my kids were watching TV, it's just like, what is this world? But most of the people watching that commercial probably thought it was

you You Whoa.

a human, you know? And so is that right? Is that wrong? No one's answering or asking these questions at a level that like changes anything. It's just sort of like we're all just like part of this experiment.

Yeah.

⁓ huh. Yeah.

Yeah, it's been like a wild trajectory too. I mean, I can think back to, I think it was like 2010 or something when Roger Ebert, you know, they kind of made a clone of his voice. he had like thousands and thousands of hours of commentary from at the movies and then to the first season of your podcast, where you're, cloning your own voice. And a lot of the first episode is just kind of about like how bad that clone is and how unhuman it sounds at times and the latency issues. And then by season two with like Kyle, for example, I Kyle sounds very much like a person. Like it would take a while, I think, to realize that Kyle was an AI agent, even for someone who was like listening specifically for it. And obviously that trajectory is sort of probably on some sort of hockey stick right now, but it is getting extra tricky to sort of suss out like who you're talking to and what they might want from you.

Yeah. Yeah. And the, the audio, my view is like, audio is already up the other side of the uncanny valley. Like it's like, people cannot tell the difference. The latency is gone and video, like we did some video this season and like, I'm looking at the most sophisticated, it's like conversational video stuff. And like, I guess maybe there's still a question. it ever get there? Like it's, it is slower. Uh, but I don't.

you got to think like it's also going to reach that point and then you've got a whole new set of issues around that not not knowing whether you're even video chatting with real person you know that that just becomes a whole new ballgame

Yeah.

other thing that's somewhere related is it was fascinating in season two, just sort of the relationships between people that would interact with your agents, like the intern that you eventually hire who sort of almost heroically seems to be kind of blowing them off and screwing with the AI agents and just kind of collecting a paycheck and never delivering work. And then you have like Kyle going out on these calls with VCs and getting laughed at. and you're interviewing interns, like how many people are just like, not even thrown off by being in a video call with an AI agent all that much, you know, this, this, younger generation maybe, but like being so accustomed or willing to sort of like set aside whatever reality they're used to, to just like, I'll sit and talk on a video call with this. entity that's clearly not human and I'm not even really sure what I'm going to get out of it. and it does feel like you're it's like opening up a window into ourselves too in a way because I think part of the reason these tools are so successful is they are such good mimics and they make us feel immediately heard and seen

Yeah, I mean, that was surprising to me just probably generationally to a certain extent, like There were people who seemed to not only not be bothered by being interviewed by an AI video agent, but like kind of preferred it or at least claimed to prefer it in certain ways and sort of felt less judged by it. Although it is absolutely judging you, you know, in the, in the same way, like it's gathering the same information. It's got its own separate biases, et cetera. But like people, I think maybe sometimes feel like, well, I'm just talking to a, they, know they're talking when they know they're talking to an agent, they sort of feel

like, wow, I'm talking to this agent. I'm not, and it's a different feeling. But I will say that also like a lot of that divides along axes of number one, is it disclosed? Like the big axis is like, if you know, it's going to be AI going into it. Oftentimes people have a very different reaction than if they're surprised to discover they thought they were going to talk to a human and now they are speaking to an AI like that, that can easily get people to be angry. But then the other one is just like.

Yeah.

I who's in charge in this call, like who has the power because a lot of the people who are interviewing for the jobs, they wanted to they wanted the job. So they were kind of like, I'm going to treat this like a normal interview. Not all of them. Some of them were like, ⁓ I peace out like I'm done with it. I'm not doing this. There were those, but there were many of them who were just like, seem like, okay, I'm going to treat this like a normal interview. Now with them, when Kyle talks to VCs or even people who make the very technology that he is built on, they're kind of like, I don't have time to be talking to some AI right now. Like they are the people responsible for AI's being like this technology existing, but they're also like my time's too valuable to be dealing with someone's AI, which is an interesting dynamic. ⁓

Yeah.

they don't like it either. So I don't know exactly what that portends for the future, but I did find those relationships and those axes pretty interesting.

yeah, that's super fascinating. Actually, that idea that they facilitated the making of it. And then when they get subject to using it, they're like, that's, that's not for me to use. That's for other people to use. Right.

Yeah. Or almost like it's not fair for you to use it that way. Like you didn't, I didn't know I was going to be talking to an AI. was like, what'd you think was going to happen? Like you made this technology released it. And in a general sense as an industry are fighting against anyone trying to control it in any way. So like, what do you think is going to happen? People are to use it all sorts of ways, including this is the mildest way that you could possibly be a victim of an AI. It's like getting out a video call you thought was with a human CEO.

Yeah, where it's trying to give you something. Yeah. Yeah. Versus take something from you, That's the scary part. Everybody has. But it's this I don't want to talk to an A.I. OK. Who would you rather like? It depends who my alternative is. Do I really just want to talk to another person or is it a specific other person I have in mind? I'd rather talk to because.

Yeah.

There's a lot of people I would prefer to talk to an AI over. If I was given a choice, like I want to know specifically who, who it's being replaced for before I make that call.

You

Yeah, I mean, it's possible that like a lot of the content in season two is going to seem quaint, right? And just a few short years because people's reactions, like even VC is like, I'm not going to take a pitch from an AI agent. you know, in a couple of years, they might be taking the majority of their pitches from people's AI agents who are like fully optimized to give the best possible pitch.

Yeah. That kind of brings me to something I wanted to bring up, which may bore the crap out of people. I hope not. But game theory. ⁓ there's like, let's say these conferences, people go that are way into game theory, and they try out their theories, right? and the one that that's most fascinating and tends to be the most successful one is one called tit for tat. And it's super, super simple. two parties start in cooperation. your default is to cooperate in the beginning. ⁓ And then all you do is mirror exactly the last move the other person made. So if the other person did something cooperative, you do something cooperative. If the other person defected and did something, then you defect. But then if they go back to cooperative, then you go back. So there's total forgiveness. Absolutely. But you just mirror and you just go back and forth. And it tends to yield not just a winner. It also is the best for all parties involved. everyone came with complex game theory algorithms, but this is the one that just wins. And the craziest part is that even after it won and everyone knew what the strategy was, it still won. In fact, it is even more effective when you know that that's the strategy that's being played. Now, the one issue with the strategy is if you misperceive the other side as defecting when they're not, then it gets into this escalation loop,

You

Because I think you're lying then I lie back then you lie back and if we're both playing tit-for-tat and we're misunderstanding the other person then it grows into an escalation and that's how it sort of explains like when the system misunderstands when you misunderstand the system Then it starts to escalate the interesting thing about these systems

Hmm.

then if we understand these things better and we don't anthropomorphize them, they're less likely to escalate because we understand they don't have intention. And so what becomes critical? This is just a long way of saying transparency is probably the absolute future. of AI, meaning I won't use an AI that I can't see exactly how every decision was made. I want to see the system prompt. I want to see the data that's fed into it. I want to see all of the symbolic code that it creates, that closed source software, closed source systems, hidden system prompts, no chance those things survive because that's what will create escalation is the unknown so does that just lead to I'm going to have my own AIs, I'm going to have full transparency on these AIs, and I'm not going to talk to any other AI without my AI present, like my lawyer that I trust present. And is that it? Online just becomes this layer where I have my AI or multiple AIs and they'll do all the interaction for me because I'm

Yeah.

as a human, a liability in the system, I'm too susceptible to being manipulated. And so our phones, our systems are going to be these open source, trustworthy beings that we communicate online through and therefore don't have a lot of one-on-one communication.

Well, I would say. I mean, that sounds, it sounds like there is a better system to be had in what you're describing, but like you're the most sophisticated possible user of this technology. And the companies that are making it for the hundreds of millions of people using it are not really catering to you. They're just trying to get as many users as possible. like, even if it were possible for the, surface all that information, let's just say, like I also find that like whenever there's anything, whether it's like we

Right.

could talk about multbook or any of these things. It's just like, what's the prompt? Like if you don't know the prompt, you don't know anything. And so, but on the other hand, it seems like a UI challenge, let's say to surface the prompt for every bit of information you ever provide. And not just the prompt, but like the underlying code and like, what is going into the decision that that just would immediately wreck the like usability of the thing, certainly for like the average person.

Yes. Yeah.

Yeah! Yeah.

So it seems like the history of the tech industry of the last 20 years has been like, pursue growth at all costs. And these, the type of thing you're talking about that would be beneficial, both from a societal perspective and a user perspective eventually ⁓ is abandoned if it was an idea, even at the beginning, because like growth is the most important pursuit.

Yeah. Yeah, I'm so glad you just said that because this last Friday, I spent five hours. Marathon conversation in Santa Barbara with the CTO of Patagonia. And this is the exact conversation we had because very few companies, Patagonia being the exception, are in a position to be trusted. Right. Because of the nature of their business model. Right.

Hmm.

And I was looking at him saying, look, Google could walk into a room and say, do no harm and everyone will roll their eyes. But, you know, Patagonia walks in a room and says, do no harm. They believe them. Right. And and it made me wonder, like, is Patagonia the only viable business model in the future for AI? Because because to your point, I can't understand it. Even even me, like I'm sophisticated. Quote unquote. These things surprise me every day. So what I just need is to trust that whoever is providing and I'm doing business with and interacting with ⁓ has the right incentive structure within their organization A and that has a value system that's been tested. And so is Patagonia the business model?

of the companies like distrust becomes not just a nice to have, but it becomes an imperative and that the Patagonia business model is the future business model.

It would be nice. mean, but but you have to if you look at I mean, anthropic, this is it's not that's not that far from what anthropic, you know why they started.

I know, I know, it's optimistic.

Yeah, I'd like to think so.

Except for all the capital they gotta return.

Well, now, but I'm saying they were started by people who left open AI because they were frustrated that the values that they believed in were not being carried out at open AI. That's part of the stated reason why they left. And part of the stated reason why open AI started as a nonprofit was to protect humanity from the dangers of AI. And so what happens is it's all well and good to have those ideas. But as you say, as soon as they get hundreds of millions of billions of dollars of capital that now needs

Oh, it started, yeah, yeah.

to be returned on those ideas like slowly or quickly go out the window. And so if, if Patagonia had the ability to like scale up and make AI, like I would support that. But I feel like in the world of building the thing so far, we don't have an example of a company. I mean, anthropics may be the closest thing. Like they still clearly do have some values. Yeah.

Yeah. I agree, Anthropics definitely trying their best, right? I mean, it's as good as it gets because without the capital, can't do it. There's nothing to be responsible with. they're probably the only way you can thread the needle right now. But that being said, we're in a state where these models. are expensive to train that that capital won't be necessary as right as it moves forward. And so then the Patagonia is the world. I don't think Patagonia is going to create. I know they're not going to create an AI, but but as a model to influence. systemically, the business structure has to change, not the product, right, not the technology. Maybe the company.

Mmm. No, I know, I know, I'd use it though.

Hahaha

The corporation is the AI.

Yeah, I mean, I think that's unfortunately we're in a race right now where no one wants to bother with any of that. yeah, I think in the long term. you like thinking about how these models are made or maybe there'll be a lot of variety of smaller models other companies will come in maybe some of these big companies will disappear who knows what's we have no idea what's gonna happen and I'm terrible at predicting the future in fact I refuse to predict the future in general as a journalist I feel like it's a very poor thing to be doing but all I could say is like right now no one's paying attention to the values that you're speaking of like it's just not it's not happening in the moment as far as I could tell and that is

Yeah.

Yeah.

That is, ⁓ it's disturbing, I'd say. It's concerning.

Yeah. And

Yeah, feels like with open AI in particular, right? It's like it's like the goal is reaching AGI first, but there's no shared definition of what AGI is with an open AI. Like, you know, to some people, it's Scarlett Johansson and her to others. It's every job's replaced. Like there's there's no real baseline to work from, which is problematic like it is cool to imagine a world where we all have our own personalized AI that exists to kind of protect us from other AI and that can.

⁓ Art

It's Ivan Shnard.

you know, if other AIs are offering transparency, we have our AI, they can decipher all the prompts and code and everything and let us know that it's trustworthy. But there's, it doesn't seem like there's a lot of money to be made in giving every single person personalized AI to protect them from other AI. That seems almost like something that people are going to have to start building for themselves,

It just, it feels like everything's been thrown up in the air. A lot's been thrown up the air and we have no idea how it's going to settle. And so we're all just trying to like mess around with these things, see how they're useful. And then you get these huge predictions, not just like, not just industrial strength predictions, but like world changing end of humanity type of predictions. And I feel like it's just, it just clouds everything like AGI it's, know, imagine like if someone was like developing a new flavor of a food and that company, they were like, also this technology could create a food that you eat once and you never have to eat again. Like, it's just like ridiculous that they get just infects like all of your thinking about what is being built as opposed to just like, this thing is being built. It's useful for this now. It might be useful this in the future, but you don't have any reliable narrators among the people who are making it. So it's sort of hard to respond to a company that says like, well, we're making this thing and we're gonna sell ads into it, which is like the jankiest kind of like, okay, that's just like internet commerce. But also within two or three years, we think it's gonna be a super intelligence. It's like, how are we supposed to deal with that? Like, I don't believe you, but also like, I guess you could be right. I just feel like we're thrown into this sort of like chaotic thinking environment right now because. which is maybe a long way of just avoiding answering what you said, but like I have no clue. Like maybe we all will have our own protective AI, but we have to build it ourselves or maybe we'll it'll be personalized or maybe we'll all just be using like the one winner of between the frontier models. Like who, who the hell knows? Because like, I don't even trust any of the people who know the most about it to tell me what's going on.

Right. Because they don't know. Because if someone is telling you what's going on, they're pretending. Yeah.

Yeah.

They don't know. And they're making up science fiction. Yeah. And they're like,

I don't think it, yeah.

they're making up science fiction scenarios. And like, it's impossible to say, you literally cannot say like, well, that's that can't happen. Like, it can't become a GI like, yes, of course, like, it is a theoretically possible thing, but you can't counter it because they're the ones with the technology. So they're always like, well, internally, we might have reached a GI already. It's like, what? Okay, sure.

Yeah. It's such a general thing, and it's this idea that it's probabilism entering our world, you know, when so much of our software is deterministic and we feel this control and we're like, it's not a technology, it's just a different way of thinking and a different way of solving problems and. And like you said, it affects everything. have, you know, would have food changed. It changes everything. Vacations, daily life, mornings, afternoons, evenings. ⁓ And to try to imagine what that world that everything changes looks like. Yeah, it's it's almost silly, right? So you go like, let's just talk about the boring stuff or let's talk about what's not going to change. because that seems, probably more productive. so many things will change. Let's talk about what won't change. What in your mind, what won't change?

Mmm. I mean, I always go to to that question in general, I would go to like, AI is not going to pick up my children from school and like walk home with them. You know, like I always go places like that, like so many things in your life are not going to be touched by lists unless you're like, on the very extreme end of the sort of like, I call my AI wife every morning in the car and then like, I talked to my AI children at night, you know,

You

in the context of the experiments that I was running and the stories that I was telling, you know, it's really, I wouldn't say it's like groundbreaking the conclusion, but it's sort of like there is value in human relationships and in the friction of human relationships that is, that is sort of. disappeared when people are talking about AI replacement kind of stuff. And that includes AI employees and Job disruption. Now I'm not saying that it won't happen. Like I think it'll happen even if it's shitty and bad. But I do think like there is this way in which a lot of the people who are selling AI employees, AI agents in your organization, think of people doing a job as kind of just like a bundle of skills. here's a skill. AI can do this skill. It can make a spreadsheet. This job includes making spreadsheets. Therefore this AI agent can replace this person. And I think that's the mentality of a lot of the thinking. but if you've worked for any time in an organization and you really spend time thinking about what people do, then that's, that's just not what a job is. Like a job is not just like doing a small skill. Like my job is writing. Like that's what I do. That's what I've done for my whole professional career. The actual typing of words into a computer is like,

Right.

a small part of what I do, which includes like getting assignments and going out in the world and talking to people and trying to synthesize all that sort of stuff. And I think that's true of like many, many jobs, even so-called bullshit jobs, which like I get into in the show as well. Like. And so that to me, it all just sort of like the optimistic thing is like the more you use AI, I feel like for many people, there is a little bit of a boomerang effect where you're like, actually, I want to spend more time with people.

Yes.

Yeah.

Yeah.

Like that does happen. Actually, I want to think about if it forces us to think about the value of people in organizations and like, what is this relationship or what is mentorship or what, why am I interacting with this person this way? Like, I think that could be valuable. That's a little pie in the sky to say that's going to happen at some broad level. But I think that that to me is like one of the things that I take away from it.

Yes! Yeah. Yeah, I think two things like one, you just sort of imagine you have fake employees now, now add fake customers. How boring does that get? I like, okay, everyone turns out like, okay, that got boring. It was only fun when you put like humans in there to see how so they were the only interesting part of the story is without the humans, it becomes boring and not interesting anymore. ⁓

Like, what's the point?

Without the phone call to an actual person, it starts to get boring really fast. If it's robot on both sides, you're like, okay, let's go do something else. What's the else? What's the thing you're going to go, let's go find a human. That's the human part that's interesting to us. The idea that we're going to get consumed in an all robot world and get lost. Probably doesn't make a lot of sense. It might happen to some, like you said, but maybe only the ones that humans aren't an option.

yeah, Joshua Gans talks about I think it's at the San Francisco airport or something. There's like a machine with a robot in a box making you coffee and that there's never a line. And I think like what he, what he was describing that could have sort of a slightly optimistic side to it is, is AI kind of flattening some of the middle layers in companies and, really the tools. enabling the frontline workers to spend more time having human to human interactions And then, you know, at the top end of the company, there's way more transparency. There's no longer like a middle layer to kind of cover up for bad decisions that are being made at the top end. So there could be a world where companies are flatter and held more accountable. And, you know, there are lots more opportunities for people to have meaningful interactions with other humans, but But again, getting there requires, like it feels like it requires some measure of clarity that's become exceptionally hard to have or to find as we have all these like weird amorphous talking entities floating around in the world.

Yeah. And maybe, I mean, probably it's just some sort of process to get to where it's going to end up. Like, you know, you've seen these cases already of we're going to lay off X hundreds of people. And then, you know, six months later, quietly, like we rehired, we hired, you know, 50, 100, 200 people because it looks so promising to someone who's just like counting beans, but actually like.

Yeah.

you're you're losing track of what the people in your organization actually do. So I think that could be the case. like, I mean, this is why I'm never sort of like pro or anti the technology. Like I love a story of ⁓ a sort of like worker who's like getting over on the boss because the boss doesn't realize how powerful AI is. Like I love those stories. They're just like, yeah, I'm working like half as much and like I go pick up my kids every day because my boss doesn't realize that like a bunch of my work can now be done if I put it through an AI. Like, I don't know. It's like hard.

Right.

not to get behind that to a certain extent from a human perspective. So, you know, it goes, it's, it's going in all directions. But of course the broad scale sort of like looks like AI can do their job. So we're restructuring and here goes a thousand jobs. Like that is, you know, that's real also. So.

Yeah.

I yeah, what Joshua Gann says, there's a lot to dig into and we won't be able to do on this episode, but he just like says, you know, the whole existence of an airport, and all of the things that exist in an airport is all on the basis that, you know, humans do bad job of keeping things going on time. So we like have these, these whole business structures that are like middleware. They're all about

They called them cathedrals to hiding uncertainty.

Yeah. And those are businesses, right? And we're like, okay, so those go away because these are really prediction machines at the end of the day. And so now flights, you don't have to get there and sit there and find something to do for 45 minutes. then time doesn't go away. So what are you going to do? that just, then what are you going to do before? with that 45 minutes, something else, some other business will need to exist to fill those 45 minutes. It's not like it's a zero sum game when it comes to time in that sense that we can just cut time away and what be asleep all day. So it's just the question of you said it like we can't predict what will replace airports. How could we? It's going to be something ridiculous though. That's all I know. Something that makes no sense to us now. And airports will make no sense to us then. That's all I know. Well guys,

You

Yeah.

You You

But I love that question. The question that we ask in the show is also, okay, if AI makes you more efficient, what do you do with that time? That's like the final question of episode eight of season two. And like, I do think in this environment, that is something we could all focus on, you know,

But if you save that time and then you just spend it on something like scrolling AI slop all day or just doing more of the different kinds of tasks that you've already offloaded to AI, like what's the point of that? Like what's gained? Like to me, that efficiency is completely meaningless. Now, if you spend that time in a meaningful way, then yes, that efficiency is potentially valuable to you. So like, I feel like that is a question that we should at least be posing.

Yeah.

Yeah. I know. I had a buddy who said if Elon got the chip going, he would get one definitely. And I said to him, like, I will never talk to you again. And not because I don't like you or I think that's bad. But you may as well just put Siri on your phone, leave it on the table and walk away. Because now that's I'm just talking to Siri. Who are you? I don't want to talk to your phone. There you go, Siri. Right on cue.

you

Yeah.

I don't want to talk to your phone. That's the end of our conversation. And if I have one, we could just set two phones on this and we could just go somewhere else. And where are we going to go? Like, who wants that? Who wants to be around that person?

You

I agree, even as a person who has been that person for the purposes only for the purposes of producing a podcast.

Yeah.

It's only interesting if one of them is human. We already agreed on that. Well guys, this was great.

Yeah, thanks, Evan. We really appreciate you taking the time with us today.

Hey, thanks. Yeah, thank you. Yeah, absolutely. My pleasure.