When Agents Have Wallets, Trust Is Currency

Well, Fed, so last time we talked as we were preparing for this episode, you were mentioning that in your role at MasterCard, you sometimes get like, I think you said like a thousand requests every year for different applications of AI that people across the organization. are interested in. and how they've changed over the past year. Yeah. I mean, that's a lot of requests. Yeah. We wanted to hear a little bit more about that.

That is like an insane fire hose to drink from.

It is, but there's so much demand for using AI across the entire enterprise to some degree. I think I'm almost glad that we have a lot versus not. ⁓ But let me give you some background in what it means. I work in our central AI and data team. It's our hub. It's where a lot of our very dedicated talent on the topic is focused on.

Ha ha!

One of the things that we always have to do is kind of balance that tension between providing things that the entirety of the company needs without kind of influencing or particularly impacting things that are very specific to a particular business. And so as part of that, one of the things we set up was we just need a way to capture all these ideas. It's been really interesting because it started over time with like a little couple of things and then as... people's understanding of AI grew as they understood where it could be applicable, as models improved. We started getting more and more to the point that now we're at the clip rates. It's approximately almost a thousand a year. And it's given us great insight into different things. The piece that I always like to think about a lot is, you know, a few years ago when we first kicked off this approach, a little over half of the requests or ideas that people around the company were having were chatbots. And if I look at today, a similar percent is agents. And so it's also reflecting in the way, like, how are people thinking about improving the way they work? How do they are thinking about, you know, deploying AI? And also it's actually a reflection back to what I was saying earlier in how things are improving and how those patterns are changing. Because now we've gotten to a point, right? Where you can expect AI to take action. You can embed it into a particular process. And it's good to see that.

Hmm. Ahem.

change in mindset over time as well.

Yeah, I use a Fletch reference for agents now. There's like a scene in Fletch where the, ⁓ where he plays airplane mechanic. What's the guy, what's the, he pretends to,

He's impersonating an airplane mechanic and he takes the name Gordon Liddy who who is I think someone who went down in the Watergate scandal, so

Gordon Liddy, yeah, so he walks up, that's right, he walks up to these guys who are actual airplane mechanics, right? And he's got his uniform on and they're like, hey, what do you think? Like, you know, and explain some problem he doesn't understand. And he's like, ball bearings. It's all ball bearings these days. And I feel like that's agents. Like people are like, hey, so I want some AI, what should I do? ⁓ agents. to all agents these days, they don't, there's two things that's interesting about that. Like, first of all, you go buy from Salesforce or somebody like a warehouse full of ball bearings, right? Like agents and the mechanics are like, what? I don't know what to do this. Like, we don't actually need ball bearings. And the other thing that's most interesting about the analogy is that if you, even to an engineer, if you took two ball bearings and just on site alone, had them tell the difference. So call this like an agent demo, right? Like someone shows you a demo of an agent doing something or a vendor showing an agent demo. It's even for somebody sophisticated, you can't really tell the difference. But it has to be perfectly machined, perfectly balanced. the material has to be exactly right, polished, et cetera. And the difference is if you put a bad ball bearing that looks exactly the same into an airplane, that engine blows up. And it's very similar. These demos are not... viable ways to discern the difference between point solutions that will work and point solutions that won't, which is so tough because if an engineer can't tell the difference by looking at two, then demos aren't going to cut it as far as trying to figure out what solutions to use. I don't know what your thoughts are on that.

I think it's a very fair way to put it. And maybe I would break it down into almost like two principles that we've been trying to focus on. The first one is actually on training and fluency. I think it's a very fair point, right? Everybody's asking for agents, so they fully understand what an agent is. What I think is important is regardless of whether folks are calling it chat bots or agents, what I care about from my standpoint on an enterprise side is do all of our employees understand what capabilities we feel are ready for use, in what way, in a way that's responsible, in a way that's safe, and that they have an understanding of how to embed that in their day-to-day work or in the products that they build. I think that's the important piece. So almost the point is they might call it ball bearings or not, but if they understand that it has to be machined correctly, it has to have the right materials, it has to the right precision when you manufacture it, I think it's important. The second piece then becomes into that precision piece, It's about trust, it's about responsibility, it's about doing so thoughtfully. And a lot of what we spend our time in from a process standpoint is embedding some of those, some of that intuition into the way we develop either internal products or market facing products, right? Is how have I made sure that a particular AI solution has a human in the loop and very critical points that could cause a failure that we don't want happening or that...

You know, you've gone through and checked for things like bias and tone of voice. If you're using generative AI, I think breaking away from the name and saying, like, what are the core capabilities I need to make sure? And by the way, which ones do I, do we think for very clear reasons are ready for enterprise use or we're not, I think has been a layer, a lot of what we focused on. And so to me, going back to the first piece is, it's very flattering to see everybody kind of ask for agents underneath the scenes is if what people understanding is,

a lot of the ideas have the right tools or the right pieces of workflows where we do have confidence then we're doing something right from a training and enablement statement for our people.

Yeah. It's, just, have a tough time getting our head around how fast things compound. Right. ⁓ and so when we think about some of this technology, we think, okay, agents that can help me do this task versus agents that build agents. And you're like, well, agents that build agents is compounding, right? Agents that do tasks is point solution. And so you start to think like, if you're going to attack the AI problem in an organization, you can first principle and say like, well, let's attack the things that can compound in this. How do we make AI compound in this organization? Which would be like, how about starting with AI that educates the organization on AI? Right? Because now you're kind of hitting first principles. You're like, AI will make itself. It's the same thing as like robots that can make robots. If you have a robot that can make itself, then you have this compounding effect. And the other thing I think, if you're struggling with compounding, then you're really going to struggle with the idea that these things can make a better version of themselves. Because now you're not just talking about the sheer compounding effect of an agent making another agent, but an agent making another agent that's better than the original. which really gets into like human evolution, et cetera, right? when we talked earlier, I really grabbed onto that as you were talking, like this idea of first principle teaching, like you can't take a thousand meetings a week where people have questions on how they could use AI to do their job better. There's just no way that you can scale. If you scale you, right, versus scale them, then they could scale them. And I think this is a fascinating idea for anyone who heads AI to focus on AI for educating. that to me is a fascinating and... and evolutionary approach.

I know, think, look, I think it's a very fair point. I'll pick up on one thing, which is the first principles piece very early on when we established our enterprise AI and data team. One of the first things we did is we said, what, what, what's just the grounding principle from a strategic standpoint we want to take, right? How do we basically give the rest of the organization some framing about when and where to think about using AI? And we landed on a framework that's quite simple. We like to think it's catchy, but it's actually really about focusing, right? So we very actively talk about using AI at MasterGuard for making commerce more secure, for making it smarter, for making it more personal, and for making MasterGuard stronger, right? And it's kind of very related to our core components. We think about powering commerce. How do we drive commerce with our payments networks, with our value-added solutions? And what that's allowed us to do is to say, This is going to keep evolving very fast. It's going to keep getting better. The way we use it will expand, but focus on deploying it in just these three ways or to make the company and the way we operate stronger. Because there's so many things we could do, but then go back to first principles. Just focus on these three things because it's a really clear way to think about when to apply it and where are just our more strategic priorities are today. then when you say, do you prioritize where you invest? How do you prioritize what you move faster versus slower? That very simple grounding becomes almost common language amongst everybody to say, well, this is about making commerce more secure. How so? OK, well, let's take that forward. And if it doesn't fit in that framework, could still be valuable. But at least you have a common way of approaching the problem of where do you deploy it as it continues to get better?

Thank Yeah. if instead of like looking bottom up, like how do we automate this stuff? You look at like, let's create a DNA for AI and MasterCard, right? Let's create a DNA strand that will be replicated across the whole organization. What are the core components of that are shared in terms of AI. What are the core values of the company? How do we embed those in? And then with that DNA, then that kind of spawns each individual kind of thinking about it and doing it. And some people would call that guardrails, but that's like way too narrow to think about it, right? Because it's both guardrails, it's creativity, it's understanding, it's learning, right? It's not just guardrails. And this idea of creating a core DNA first and using AI to help you replicate that DNA internally is, I think, a really cool and unique concept.

Yeah, I think that's right. And that helps you drive scale almost to put it in in a different way. The best people to think about where and how to deploy AI are the people working on different business problems. We have many of those people working to support the ecosystem to support our clients on that. And so part of the idea is how do we empower them? And then from a more enterprise standpoint or from looking at the enterprise lens,

How do I and the rest of our team focus on the very common problem, like the least common denominators that then help drive that scale?

Yeah, yeah, it's the like, everybody wants to do something with AI. But really in the beginning, it's, it's, it's like start at the top, which is know something. You need to know something before you do something. If you do something without knowing something, you're going to do it wrong, which is what we see constantly. As people say, agents fail. What does that mean? Agents, agents start to do tasks before they are trained in the knowledge.

to safely do tasks. So because people go to doing, ⁓ you think of like, we use Patagonia as a good example, like if you hired somebody to work in a store, ⁓ you know, and you're like, okay, great, you're gonna sell wetsuits. Like what's the first thing you're gonna do? Are you gonna teach them how to, you know, how to open the front door and how to greet a guest? Or are you gonna teach them the knowledge of wetsuits and like what? what you need to know about wetsuits first, and then you put them on the floor to actually sell wetsuits. And we just know this intrinsically with people, but with AI, we tend to like think of it in the opposite. Like, no, no, I just want to get them out there on the floor selling wetsuits, but they don't know anything about it. Like they just sold a five millimeter wetsuit to this guy and we don't have a five millimeter wetsuit. Like we don't make one.

Yeah, I think that's right. The piece of context of deploying it with purpose of embedding it where it makes sense. And honestly, having the human in the loop in the parts of the process, they're still too sensitive. think, I think it's pretty critical.

Yeah, yeah.

Yeah. And I think having that, that knowledge in place was really important too, as we start to move into this, ⁓ this era where agents are going to have wallets. I don't think we're seeing a ton of it yet, but it's, think it's just around the corner. Like we're going to have agents buying things on behalf of people, people sending agents out to make purchases and really maybe even not paying that close of attention to those purchases. that really points to, think.

Another one?

the key value prop for MasterCard, right? trust. that's the most durable asset that you have. And so make sure that anything agentic happening in the organization is strengthening trust and keeping trust going in a world that's like suddenly very shifty and confusing,

It is, it is and. we know there's interest in using agents to drive payments, right? And a growing amount of online traffic with respect to product discovery is happening more and more via LLMs. And so we know that the consumer demand is there. We know that one of the key pieces that was missing was closing the loop on that, right? You discover a product and then you just want to say, okay, I've decided on X of the recommendations, go ahead and execute that payment or that transaction for me. We knew that was coming. It was coming rapidly. And so the first thing that we, as a, as a network and a provider of many commerce solutions have to do is we have to make sure it's safe. It's equally secure and it's seamless to have an agent execute a transaction on your behalf, much like it would be tapping your card or your phone or whatnot device on a terminal. It's really, really important.

Yeah Yeah.

And so when you think about that, then you say, what are the core components of that is I have to verify who an agent is and that they have the delegated authority to do so, that the credential is right. ⁓ I have to make sure that every party in a transaction is able to understand that this is an agent and what safeguards need to be put in place as part of that. And so at the very core, we've been focusing Much on that, like how do we drive that into our operability? We have things like, you know, an acceptance framework that talks about what are the type of credentials that get exchanged as part of this? If you're a merchant, what is the low lift way to start accepting, you know, the type of tokens that are used by agents? ⁓ You know, the core adjacent capabilities, like having the ability to do agent sign-ups so that we only allow registered agents to transact on our network.

supporting many of the integration patterns as you've seen in the many protocols. That type of behind the scenes work is actually quite critical because if we from our standpoint can enable that very well early on, then that allows you to scale into moving to much more sophisticated transactions. Today it's you search for a product, you discover the one that you want, agent executes that transaction, but

Understandably so many people jump ahead to well, could it do multi-card checkout for me? Could it actually research a trip itinerary and book all the adjacent pieces and make decisions behind, know, connecting the dots on this like we're all really excited about that, but that only gets there if the ecosystem moves through what I would call was the base case just being able to discover a product execute that transaction doing so safely and doing so in a way that you feel and you see

Mm-hmm. Mm-hmm.

is equally secure, is equally seamless, is equally painless as using any of your other credentials today is. Like that is really, really important.

Yeah, the idea that like at a logical algorithmic level, the transaction, you know, the moment the money transacts and all things are handled, like race conditions, etc., like the money is now irrevocably, irreversibly done. I know that's taking it too far. It's not quite irreversible, but But generally that's like the moment that whatever was happening before, ⁓ culminated to that moment, right? Now, now we exchange money and there's a big responsibility on that moment because that's the moment that, that says, okay, are we sure all the ducks are in a row? Are we sure that, that that's the actual person that this transaction, these are the right parties? Is this the right amount? have all of the elements that are the prerequisites to this transaction have at the album met, right? ⁓ Okay, like we'll make the transaction happen now. So it's kind of like in the chain of whether it's AI agents or not, this is like the most important moment, right? And I think as things get more complicated, because you mentioned it, right? I always like to think about buying, you know, airline seats, because if I said like, how much is an airline seat? You'd be like, what? To where? And then I'd be like, how much is an airline seat from San Francisco to New York? You're like, what time? What day? Okay, this time, this day. Okay.

How early did you book it? there is no such thing anymore as a price for an airline. Like, this is algorithmic and it constantly depends and it's complex. There's no static price now. And we're going to see this across the board on merchandise that's typically static priced. We're going to see the... All of this variable pricing being built in based on demand, supply, who you are, what you can afford, how bad you want it. As we see this in travel, we're going to start to see this in buying groceries, right? And that does complicate things.

I'm not sure. I'm not sure yet. I think, I think it's going to get very interesting because while having buyer agents and selling agents does open the capability for a lot more price dynamics. As a consumer, I still have an expectation of what a number of things cost, right? And that usually drives some of my behavior. And so where we're more focused on is talking about complexity.

you Mm-hmm.

Think about the different patterns that could emerge from just a transaction standpoint. There's going to be agents now doing transactions. Maybe they'll do a number of transactions in sequence much more rapidly than you as a consumer would. So the way that the patterns look in our network are going to start changing. And again, part of what we focus on is how do I first help everybody in the ecosystem verify that the agent is doing a transaction that you gave it authority to do, that it's actually

Ha ha!

exchanging the right information with the merchant to ensure that you get the pair of shoes that you wanted or the plane ticket you wanted or whatnot. And then once that happens, live while the transaction is being routed, things like our decisions intelligence capabilities, how do we in real time understand the number of variables to say, this is right. This doesn't fit a fraudulent pattern or it doesn't fit a set of relationships that point us to be concerned. And then the last piece, which again,

things that don't get talked about a lot is as we manage these networks, there are rules. There are rules for when a transaction can be settled and what that looks like and what happens if something was incorrect or services or goods never tendered. And that helps protect everybody in the ecosystem. And extending those rules and making sure those frameworks are in place is actually quite important because back to your point, the world is going to get more complex and having that base understanding of How do we all agree to operate with each other to make sure that at the end of the day, when you tell your agent to execute a transaction, that it's going to mean that you eventually get what you asked for and that the party that's doing selling is going to get paid for it. It's really important.

Right. Yeah. it's a point because I think on the surface, people might think this middleman handling the transaction becomes less important in AI when in fact it becomes like 10X more important. It's like the inverse.

That's right.

Yeah, and I think it connects with other things too. Like we had a conversation with Joshua Gans recently who's an economist and a professor at University of Toronto. And he sees AI as just cheaper prediction basically. And so on his model, like at some point Amazon gets so good at predicting things that they just start sending you stuff. I think he called it ship then shop. Although it seems like the shop part kind of falls away, but like Amazon just knows you well enough that it just starts sending you stuff. And so in that scenario,

You know, if MasterCard's in that ecosystem too, then the trust is like this shared passable thing that almost has to be agreed on by different entities within an ecosystem.

Yeah, think, look, trust for us is the currency of innovation. ⁓ If you think about MasterCard, you think about, we safeguard a lot of trust. That is core to what we do and everything we do. And as the world gets more complex, as some of these patterns get more complex, we find it to be part of our role to make sure that the ecosystem can operate in a trusted way. If you think about that example and you take it logically,

Again, I try to use myself as a consumer, as a grounding mechanism very often. I don't know if I'm always going to be comfortable with something like ship then shop, right? And so for some things like paper towels and toilet paper, if I could have an agent that learns how frequently I need it or I have some more automated way to do it without having to tell it, send it to me every couple of weeks because I behave differently every day, that would be great. I would be very comfortable with that.

in a minute.

But I would only be comfortable if I can trust it to have a number of controls so I can delegate some authority to the agent. I can say, I'm willing for you to more autonomously send me paper towels and toilet paper, but don't do it more than X frequently or don't spend more than this much money and only do it via these merchants. And actually those controls is very important. And I'll want to make sure that

my agent not just has cardrails, that can only transact with parties that I think of, you know, that are going to be verified, that are trustworthy, that kind of operate again within that same kind of rules. Like I as a consumer want that because even for somebody like me that's spending every day, you know, at work thinking about how do we push the way we work forward? How do we innovate in what, you know, millions and billions of people use every day with AI?

I still think about like, what am I at a core truth, very comfortable with and where does my trust lie versus what am I just not ready for or what needs be true for me to be ready for it.

Right. And if you think about it with trust, and this kind of gets back to our earlier point, trust is about the knowledge that the system has, like everything you just said. Like, let's say it has the knowledge of your smart thermostat, right? Let's say that's part of its knowledge. Now you might think, well, what does that have to do with toilet paper? Right. But if the thermostat's been at 50 degrees for a week, you're not home. And if you're not home, you're not using toilet paper. So now that the system knows you're not home, now before it goes and buys the toilet paper, it's going to confirm with you Hey, I don't think you're home. You want me to skip the toilet paper work this week. Now what just happened? Was it guardrails? No. Was it trust? No. It was knowledge. It needed to know and have full context. Without that, you're going to have it ordering toilet paper every week. And then you're going to have a closet.

Let's keep it.

Well, I'll tell you, which might be fine if there's another epidemic,

I don't know if I have enough space for that right now. Yeah, fire hazard, space.

also be a fire hazard though you know yeah

And I think this is important because these, what I would call dumb algorithms, right, these like basic algorithms that so many companies have tried, like automated. I remember they had like Amazon had, you know, these buttons that you would, they would put and then they would auto order and then like my kids, when they became toddlers, found the button. And I had like, yeah, and I had, I had put it under the sink. I thought I was really smart. You know, I put it, you know, where, where you would notice that you're out of ⁓ dish soap, so that you just press the button when you see the dish soap getting low. And yeah, I had like 100 cases of dish soap show up at my front ⁓ door. So again, like not to say my kids. don't have knowledge but they did that too. And I'm likening agents today as two-year-olds, right? Like yes, agents make sense. A two-year-old agent that's in charge of ⁓ B2B helping you shop a B2B software solution for your company. doesn't make sense. What makes it a two-year-old agent? The knowledge it has. How basic is the algorithm it's operating on? And so Josh, I think that ties into this whole knowledge management is also something that people address on a point solution basis. We think of it as a document repository versus how do you actually manage knowledge in an organization? in an AI first sort of way, ⁓ in a way that's not about how we used to do it. ⁓

I think it's pretty foundational. And I remember when we originally were chatting, I shared of the most common patterns of development we had early on when we really, when generative AI really started permeating across the enterprise was that was start with getting a knowledge management solution in the hands of a team or employees.

Because that then helps you start figuring out where is the AI tools you have at your disposal. Where do they behave like a two year old and where are they very impactful? So let's take the example you said, would you take a standard AI agent today and have it make a decision on a B2B software service solution? No. But you know what happens today? For a number of material decisions on that, you do an RFP or an RFI. That requires a lot of information and typically each party on the issuing side of an RFP is looking for different things. I will be looking for different answers than my security counterpart, than my technology counterpart. And so actually I would use an agent to help me answer those questions. I would not today use it to make a decision, but I would definitely streamline the way that I do the evaluation because I have.

you Mm-hmm.

something incredibly powerful at synthesizing that knowledge, but where I actually want critical thinking, like critical human thinking today for the decision making process. ⁓

Yeah. Yeah, and 150 % on that. 150 % because what we're talking about is, again, back to learning. Like, first principle is if we teach how to select a solution first, then you don't need an agent to select the solution. We need an agent to help teach people how to select a solution. That's where you begin. And if you teach an agent to select a solution without teaching people or the agent itself the knowledge on how to, then you're gonna get the wrong solution. That's just, it's so obvious when you think about it, like we never do this with people.

That's right. take it, think about the way you develop products today. Think about your average product manager. They spend a lot of time at the very early stage of the funnel on knowledge management, right? Talking to users, observing behavior and synthesizing into requirements. Well, what I really want to get out of a very talented product manager is not the procedural or mechanical step of consolidating feedback into stories that then become requirements. I want them to think about

what do you do about those requirements? What do you do about, how do you actually solve the need? Which continues to be an extremely hard problem. And that's where you really see very talented product managers shine versus the normal ones, right? And so it's all about shifting the way you consume knowledge or frankly, how you deploy the tools. Because again, when you are now accelerating the way you make requirements into things like, okay, making a decision.

Right. Yeah. Yeah

Can I augment that product manager by coming up with more ideas or more rapidly prototyping something so they can test their ideas, right? I get very excited about this topic, but it's so powerful when you can take that step from just knowledge management and say, ⁓ because I have much easier access to my knowledge, I will now think about step X differently so that I can, I, the human can spend more time.

Yes. Me too. Most people get bored and fall asleep.

critical thinking or valuing something which you know, that influencing others, communicating a story very clearly are things that continue to require, like, I think are requiring more and more of a premium on people than they used to.

100%. And I think this idea of things being boring, know, if things are boring, they're good in AI, if things are exciting, you're probably doing things wrong. And if knowledge management is boring, then that's what you should lean into.

we're in this paradigm of knowledge with these systems like what I would call the chat bot era, which is you ask it the questions and then it gives you the answers. So you need to know what questions to ask, right? we took our best-selling book, stuck it into an LLM, this is like years ago, right? And we're like, aren't we smart? Now you can ask it questions. And then people would sit there with like, why do have to ask it? Like, ⁓ good point, because that's not how we consume knowledge.

I always liken it to the classroom like, okay, you're now a teacher at MIT. It's your first day. You arrive in the beginning of the class. There's a bunch of students just eager to learn and you go, okay, any questions? Anyone? Like I'm a chat bot. Just ask me what you want to know. And they're like, what is this, right? These things need to be proactive, not reactive. And

And this AI first approach to knowledge management is about not just like it having the knowledge, but it proactively sharing the knowledge within your organization and curating it and like driving knowledge down instead of like waiting for knowledge to be requested. And Josh, I think we have some like kind of demos.

Yeah, I can kind of show you what it looks like for us right now.

So this, what you're looking at right now is actually a knowledge model of pretty much every episode of the last season of this podcast. And it's broken down into tags, which are topics that were discussed areas of interest. And then you have these notes, which correlate to specific conversations or moments of conversation that occurred in different episodes. can see they're all different colors.

Yeah, these are like canonical ideas.

Very

And so when

cool.

you look at them together, ⁓ like when I first loaded the information into this model, it kind of had more of an orb shape and like a lot of the ideas were sort of disconnected. But then what you actually do is you kind of go through and verify each one. And as you're doing that, you're also deciding like what other tags it connects to. like, is the knowledge in here correct? And what other pieces of information in this knowledge model are related? to that knowledge. And so what you yeah, like what you're doing is kind of going through and you're saying that this knowledge is essentially ready for consumption. And Rob, you wanted to talk a little bit about the composition of this too, right?

B-B-B-B-

Yeah, idea is, Patagonia again, Imagine you have ⁓ a knowledge model for every product, right? So wetsuit, a specific wetsuit would be a knowledge model. And then within that you have these categories like weather, materials, sustainability, and then under those you have ideas that are canonically like the... the thickness of the wetsuit, where you're not going to have that replicated so you don't have an overfitting problem. It's it's a that's canonically correct. So this is why it's not your document repository that's a mess from a canonical standpoint. There's no truth. This is now a source of truth of knowledge, right, with a history in each one of how that knowledge changed over time. Now with

Right. No.

is really cool is you look at it like a map and I don't know if you're familiar with the traveling salesman problem but it's like this idea that says okay well this is a traveling salesman problem now we're going to teach you everything that's to become an expert you need to learn everything that's in this map right now if you want to be a full-on expert so what are we going to do the first thing we're going to do is we need what you know so we go in we make a digital twin a learning twin of all this right? And we go, okay, this is you. Now this now represents what you know. Then we do an assessment. We go through and assess what you know and what you don't know and we light up all the things you know. And now we have a gap analysis of what we need to teach you. And then we do traveling salesman. We go... What do need to learn next? We don't try to map out a curriculum for everyone. This is your curriculum for what you need to learn next. We consume that. You learn it. We get feedback. And then the next thing you need to learn. And so you're traveling through this, learning what you need to learn. And what it accommodates is A, where you start. It's not the curriculum that's for everyone starting from the beginning. It's a custom curriculum for you based on what you know. And then the second piece that it's doing is as this stuff is dynamic and the information is changing, you're picking it up as you go because it's not creating a curriculum that two years later you're teaching the same curriculum even though everything's changed. So you have accounted for the dynamicism of the knowledge that's now picking it up as you go. And So now you have this GPS for learning, which says, here's where you are, point A. Here's where you want to go, point B. And turn by turn, we're going to get you there. But at each turn, you could go a different way, and we'll recalculate the route. And so this is all just happening. And it's canonically correct, so we're not going to teach you the same thing three times, which is ultimately the objective. And this is AI first learning versus the opposite, which is somebody goes ask a bunch of questions about what you know, puts it into an LLM and says, based on this, create a curriculum, and it just generates crap, right?

Okay. The piece that also jumps out at me from this, and it's something I spend a lot of time kind of wrapping my head around is you're not just talking about the capability, you're talking actually about a, almost like a cultural way of working shift. And I think that's one of the things that is quite underappreciated about this whole transformation. Like it requires teaching people to change how they work, to think differently. And that, that is at an enterprise level for us, like that's a cultural.

That's like a big change, right? It's cultural shift. It requires people to think differently, to think about where in your day to day are you going to adopt a tool like this that all of the sudden, right, will help you learn better, will help you improve in a number of things that you do, but it requires you to engage differently. You might not think about training the same way or you might not approach it the same way. And that to me, on top of the fact that it's really creative about tweaking the knowledge based on all of this.

is you're also changing the way somebody's going to approach the whole concept of training or readiness as a whole.

is just one of many front ends you can now put on that knowledge. The whole idea is now that I have this knowledge on the wetsuit, I can deploy agents that can do things, like handle supply chain. ⁓ So now you take that knowledge in the center and then you orbit the agents around that knowledge. But without that, you've got these agents with no core, with no brain.

from that.

aimlessly trying to accomplish stuff out of context. And so this is now like, okay, now we'll take that and create an agent to teach.

Yeah, and this agent is ready to teach us about everything basically that was discussed last season. And like Rob described, first you can go through and read through the different topics that were discussed and kind of mark yourself. Like, am I an expert, a novice? How much do I know about all this stuff? So then the system kind of knows where to begin. Yeah.

That's where you are.

That's where you are, that's point A, yeah.

And then here, like, what's your objective? Like, am I curious? I think for a lot of people in enterprise, it's going to be upskilling or mastery. then, you know, you can put in a specific goal. then it creates a curriculum ⁓ that's tailored to what you're trying to do and what you already know. ⁓ And the idea there is that

Like, you we were kind of thinking in terms of our book, right? It's like this thing that you read from the beginning to the end for most people. But, know, there's a world where like what I really want to read about are design patterns because I do a lot of design. So I want to start there. And so if you kind of like lead people to the information they're most interested in, then you can start to build out that. Yeah. The, knowledge twin, essentially.

to expand.

Yeah. And I think that's really important. Like once you start to understand how much everything is learning, you realize like, want to buy a web suit. Which one should I get? That's learning. What do you know about web suits? What don't you know? So now the sales process is learning, is a class, right? If all of your folks are looking at AI products, each one of those, like each vendor should be providing this, right? Like, what do you know? What do you want to do? I'm going to teach you the parts of this product and how you would do it with this product. Now we're talking about educating people on how to make choices on which AI products to buy versus asking you to help them pick one.

I know. Yeah, yeah, and you do over time, right? As your catalog changes everything, you'll just continue pushing on that front. You'll continue teaching people about the next thing and makes a lot of sense.

Yeah. And you can imagine an organization where you could have a knowledge model like this that's designed specifically just to get people up to speed on a gentic AI and figure out like how they'll make use of it. But then throughout an organization, you might have all sorts of organizational aspects that you really wish everyone knew. Right? here is, here is how we do this specific thing. Everyone needs to know this, but everyone can kind of learn it in the way that will work best for them

Yeah, and I think the most important thing in that what you just saw is it's not building the ten steps or the hundred steps to be an expert. It's just looking at the next one and then recursively taking what you learned so that if you go learn something somewhere else, right, ⁓ it can accommodate that. You don't have to like also learn it here. can say, okay, you already learned that? Okay, we'll skip that and go over here. And then the other thing I really like about this and how it changes the paradigm of learning is we're so used to teachers being the delivery mechanism for the teaching, right? Like they'll communicate the learning. They'll be the vehicle of the ideas. But if we think about teachers as a facilitator, not necessarily the deliverer of the ideas. And then we look at this as being a facilitator of the idea. Then it can see an idea that you need to learn next, but it can go find a good YouTube video on it to teach it. So it doesn't have to facilitate the teaching itself. AI first is, isn't about technology problem. It's about rethinking. You said it earlier. Yeah. You said it earlier. It's like rethinking how you do business, how you think.

It's rotating in process. Yeah. Yeah, that's, that's the most exciting part of all of this transformation is how do you rethink something to get to more value or to really, you know, focus somebody's time on the pieces that are going to be most impactful. Or for example, back to, back to training, where do you focus hands on learning where you have to do versus, you know, asynchronously capture ideas, et cetera. It's I think AI has a lot of promise about. Rethinking some of the ways we've done things ⁓ But again, maybe back to the beginning of our conversation Like what is the point of teaching as well? I want somebody to learn these concepts so they can be effective at doing this or they can be great citizens or whatnot therefore What do I need to make sure the person is getting out? I think it kind of goes straight back to the beginning which is But what's that end state that you were looking to get? And then how do you deploy slightly different tools, some very advanced now, to get to that outcome?

Yep. Yep.

Yeah. And I think too, just this idea that, with trust being such a, such an important feature in, in anything really AI related, especially with commerce, you realize that knowledge is, is almost, ⁓ inextricably tied to that, right? Like, like agents can't operate without knowledge. Humans have a hard time operating effectively without knowledge.

Yes, it is. And one thing...

That's right.

And one thing we didn't show was that that system, that expert had the ability to go out and learn every day. as knowledge is changing, it's proactively meta-learning, right? And that could be talking to individuals in a company that could be searching the web, that could be going through the latest documents. But each day...

It's. Yeah, additional files.

listening to the podcast, right? ⁓

Listening to the transcript. Yeah, I actually saw a good site I don't remember the name of it, but they aggregate all the transcripts of the top like 5,000 podcasts I'm like, okay, I'm gonna make an expert out of that. I'm gonna Consume those by different ⁓ Categories like AI and I'm gonna and I'm just gonna have it every time a podcast comes out. I'm gonna have it consume

that knowledge and what's cool about this idea of compartmentalizing the models like this is if the subject matter or the transcript has nothing to do with wetsuits, for example, nothing gets added to the model. So we enter this world where like, don't give me the summary of a meeting that AI summarize. Give me the transcript. I'm gonna feed it to my experts.

Tell me the pieces I need to know.

Each one's going to pull out different pieces that are relevant to me. I don't need your two-year-old agent summarizing this. I need the ones that learn. So if we talk about wetsuits, that will get added to my wetsuit expert. The other expert might be healthcare related. It's going to pull those. And when we were talking about my vacation, that's not data that's going to get consumed at all. It just makes so much sense.

And you're going to want human experts, like to the human in the loop piece. think you're going to want human experts up and down this stuff, making sure that what's being taught is accurate because transcripts of all the most popular podcasts doesn't necessarily mean true things, right?

That's right. incorrect. That's right.

Yeah, good point. Fact checking. Which is like another thing that people call it. They'll call it fact checking or guardrails. But in the end, it's just knowledge management. It all rolls up to, is this true? You know, is this the right person? did they mean to buy this? is this the actual price they agreed on? It really all rolls up to knowledge.

Exactly. That's right. It's not management, it's context engineering, it's framing the right pieces and then developing algorithms, processes, approaches that kind of drive that little trust. That in a non-deterministic set of tools drives a consistent outcome.

Yeah. Exactly. I'll have my like MasterCard agent that knows all of my context for what I buy and if some weird transaction happens it'll be like, text me, wait a second, this doesn't seem right. Who buys...

There you go.

Amazon's trying to order toilet paper again. I don't know if you're ready.

15th

packet of paper towels on its way.

Yes. Well, cool. This was great. It's a great chat.

Yeah, super fascinating. Thanks for spending time with us today.

No, please. It's lovely to have the conversation and I it was super engaging.

Love what you guys are doing. It's awesome.

Absolutely.

well, thank you.