Nuclear Fusion, No Power Lines

It's a great menagerie of plants.

Connecting.

Yes, I remember the plants.

I remember the plants, yeah. It's nice to see them all again.

Yeah, I love going through my old podcast for the past three years because I always get like, you know, I can watch the plants grow.

There's such a metaphor for like pruning data, pruning plants. There's some shaping and thing going on.

God.

You it's almost like the podcast we did is still advanced in terms of the subject matter, you know? And now, which is on our list to talk about, which I want to talk about just for this exact reason, we should be talking about context window size and how much it matters and does a million tokens. Does that make a difference? Like this idea that people have that, I don't need to train anything anymore. Like it's just going to know everything. don't have to manage documents I can just dump everything in there and it's just going to give all the right answers. Help us out with that one.

I'm going to give you probably the same answer I gave you last time. ⁓ when, know, there was also a giant context window we had released at that time, 65 K context. was huge. All you'll ever need. but the, the bottom line there is you got to test it. You got to go through the progressions and try the stuff because every use case is so different. And the way that an LLM behaves is going to be different depending on the document and the way you structure it and all sorts of other things that aren't within your control or are within your control. like.

Yeah, yeah.

Will people, you know, no longer need to train? I think some people are never going to need to train again. ⁓ is everybody going to be done training? Nah, I've got lots of customers who are fine tuning or running DPO or what have you. And it's crucial. ⁓ should, can you just dump all your documents in? Sometimes it works great. Sometimes you actually see performance get worse as you retrieve more documents and add them to the context window. Cause you have more distractors and the LLM is still not perfect at figuring out what's relevant and what's not relevant.

you Right?

⁓ so the answer that I give everybody is, we're scientists. everybody who works with AI is going to have to be a scientist and you're going to have to sit down and figure out how do you measure what a success look like. And then like climb the ladder of techniques But I think the true place where the long context windows matter. And the reason why everybody's doing it is it's about multimodal. Like an image is a lot of tokens.

Yeah.

Yeah Mm-hmm. Right.

Video is a hell of a lot of tokens and to even do basic things with these models on images and video, you need these massive context windows. So I wouldn't, I wouldn't overlook that part.

Yeah. Mm-hmm. Yeah, so it's, it's use case oriented. And I think going back to our last conversation, the whole, you know, where we kind of came up with the mixology it doesn't change garbage in garbage out, like whether you're putting it in the context window, whether you're training it in the model, the data you put in sifting through that, making sure that that data is right, dictating what the output is, it doesn't change. that that's the hard part, not how easy it is to get it into the model as much as how easy is it to sift through and figure out what should go in.

Yeah, I think that's exactly right. But you know, what's changed in some sense is like, you know, first of all, a lot more people just kind of understand how to be scientists about this stuff. You know, a lot of people have gotten their, their feet wet, whether it's, you know, explicitly thinking about it as a scientist, or whether it's just working with chat GPT and trying to get something to happen. You know, I think we just, we've created a lot more scientists over the past couple of years and it's great. Like when I go to customers,

That's true.

You know, there's someone who I would consider a scientist at least, you know, I'm not a big credentials guy. There's someone who is doing science in a formal way that I would, who I respect as a peer, as a scientist at pretty much every company I go to at this point.

Mm-hmm. since you brought it up, I want to hear your opinion on this. There was a recruiter for AI talent and they basically said that if you didn't work for one of the hyperscalers, the big companies that funded AI research at a major level, she just wouldn't even consider you as an expert period. that obviously never sits well with me because that makes no sense. But, I wonder what you think of that comment,

I mean, you know, I'll be direct. That's BS. That is such BS. And, ⁓ you know, like taking a step back in the talent situation, there are a couple of different aspects to it. One is like, what kinds of people can be successful at this? And the answer is this is still a new field and there's nobody who knows what they're doing. There's not a single person on the planet who understands this stuff. This is like, you know, going to ancient Greece and looking for someone who could do quantum physics.

Mm-hmm. Exactly

Like the stuff hasn't been invented yet.

Right.

and I think also just a fundamental belief I have about the world talent is very broadly distributed. There are brilliant people everywhere. ⁓ and there are brilliant people. This technology has been so democratized if you told me you needed to hire someone who's going to build a new world-class LLM from scratch and lead that project or be one of the key people in the project. Yeah. I'd probably want to hire someone from a lab where they had done that before, because you don't want to relearn every lesson the hard way. But as our friends at deep seek showed us, you don't need that. it's a question of time versus money. If you spend a lot of money.

Mm-hmm. Mm-hmm. Mm-mm.

hire a lot of people who have done this before and give them a huge amount of GPUs. You can move faster, but if you give good scientists some good resources and let them take their time and be rigorous, which is what the deep seek people did. If you read their papers, they just took their time and did the science carefully. And I don't know if, you know, folks at other places have had the luxury of time. Like I imagine for the folks at Meta they've been under huge pressure to ship and ship and ship.

Mm-hmm. Right. Mm-hmm.

You get a hundred thousand H 100s, not because you're training one model that needs all those GPUs, but because it gives you more at bats and you know, more chances to hit something because you're, spending money to buy time, which is, it's kind of cool that you can spend money to buy time, but talent is widely distributed. Good scientists are all over the place. And, I've seen incredible people from all walks of life from like, you know, self-taught former middle school teacher who's, know,

Right? Yeah. Mm-hmm.

One of my team leads and what is the one of the most valuable people in the team to like PhDs from Stanford. Like you can, you can come from anywhere and do this successfully. It's just a matter of, you know, getting the time and space to do that. But there, I mean, there are other aspects of the talent thing, It's the good people are really good. Like there is a, there is something to that. ⁓ now that doesn't mean by the good people, I don't mean the people who worked at deep mind. I mean.

Yeah. Right, right.

The people who just seem to really have a knack for this, who seem to really get the science, who seem to have kind of the ideas that seem to work repeatedly, who just, their brain seems to be a good fit for this. And who are like really, they can get shit done in an environment where like, it's really hard to get anything to work in AI. Just it's complicated. You have to know computing, you have to know science, you have to know AI, like just getting all the pieces together. Those people are really rare and they're very highly compensated and very highly sought after.

Right? huh. Hmm.

but they're not exclusively at these amazing institutions. And there are plenty of people who don't have that pedigree, who do have that ability, who are just getting completely overlooked right now by people who, you know, they'll only hire you if you already have a job at one of the fancy places.

like in science, I always, you know, I always tell junior students, like your ability to communicate and connect with people is probably more important than your scientific ability. because at the end of the day, lots of people have great ideas, but being able to disseminate those ideas and get other people on board and build alliances, not build rivalries, like all the little things that help you do that, or figuring out how to speak to the audience that, you know, controls the levers of power. How do you speak to VCs and speak to, the, the open community and convince them your ideas have merit. And not every good idea wins every time.

Yeah. Yeah.

You know, there are plenty of great startups that were ahead of their time or, know, we're just kind of almost there, but didn't quite win. need a lot of luck. Like I will tell you took a hell of a lot of luck for Mosaic ML to be successful. I, know, plenty of skill too, but a lot of luck. so, you know, there is a bit of an insider outsider dynamic, but it's the, it's the marketplace of ideas and sometimes, you know, good ideas don't win immediately, even if, you know, something similar will win in the end.

Yeah. Mm-mm. Yeah. I was thinking about this from your perspective because there's one way to look at Mosaic's path as unselfish and the sense that for builders, it's more fun to just work on your thing and getting it out to the world can be seen as a distraction from just working on your thing. Right. But the impact that the thing you've built has on the world is completely predicated on your ability to get it out there. And by selling in a way, you're almost choosing to forfeit the ability to focus on just building, right? And saying, look, I'm gonna balance this. I'm gonna go to a company that can help me get it out there and actually get people using it and actually make a difference versus just keep your grinding away. I wondered if... what your thoughts are on that trade off for yourself as you were like, man, I just want to be in the back room building.

I think you're giving us far too much credit. You know, at the end of the day, was a $1.3 billion acquisition. you know, if you want to call that one unselfish, you know, but, but, know, but, but yeah, no, I look at it a bit differently, which is like, I don't know, Navin, you know, mosaics founder, CEO, the, advice he always gives me just on everything is just to maximize your impact on the world and everything else will follow.

I'm just saying you could see it that way, you could see it that way. Right.

That's just, he tells me that all the time and the good times and the bad times and it's helped me focus in the good times and it's like helped me stay grounded in the bad times, but it's always just maximize your impact on the world. getting to join forces with a company like Databricks, which has already had monumental impact on the world and, know, be able to do our small part to, know, to even increase that, that impact on the world, you know, lot more than we could do with mosaic, but. You know, for me building, I don't know for some scientists building and doing science is its own reward. And I love doing that. Like building's fun. Science is fun. But, you know, if it doesn't matter in the world, kind of what's the point? Like, you know, who did it help? Why did it, you know, what am I going to leave behind when I'm gone? for me, those things matter a lot. And there are lots of different kinds of founders and lots of different reasons to found a company.

Yes. Yes. Mm-hmm. ⁓

But when I look back and like think about the different categories, I definitely put myself in that Naveen bucket of like, the goal is to have impact on the world and startups and venture are just this extraordinary lever to change the world. And, you know, if you told me this was a less interesting lever, but going into government was a more powerful lever, you know, maybe I'd be in government right now. Or, you know, if you told me like being a scientist in academia was the biggest lever, maybe I'd be doing that, but I want to make the world better. you know,

Mm-hmm. You

make it a more interesting place and bringing technology forward the way I can do that and venture and startup and now being at Databricks are a way I can do that. And, you know, for me, that's kind of, that's what I want to look back on and be proud of. I think I may have said this last time, like I'm really well known for this paper on the lottery ticket hypothesis. That was my dissertation, like this whole interesting set of ideas about what it takes for a neural network to learn. ⁓

Mm-hmm. Mm-hmm.

And, you know, it's got like 4,000 citations. And when people see me at conferences, they're like, you're the lottery ticket guy. And I'm never going to like get past this. But when people used to ask me even before the mosaic days, like, what was the most impactful thing I had done? My answer was like this report I worked on before my PhD on police use of facial recognition in the U S that actually like. Mattered it changed laws. changed people's perspectives on an issue.

Mm-hmm.

people in their day-to-day lives heard about this and it made them think carefully about security versus privacy in their lives and what trade-off we wanted as a society. The lottery ticket paper made a bunch of grad students excited. Who the hell cares? ⁓ So that's kind of, for me personally, those are my values. And Mosaic and then Databricks especially now is just like, it's such a huge lever to try to help create the world I want to see.

Hmm. Yeah. So where do you think we're going? and I mean more technically speaking,

I'll tell you what's been on my mind lately. know, far be it for me to predict the future. ⁓ you know, I've only ever been wrong. But I'll give you a metaphor that I've been thinking about a bit. Like.

Mm-hmm.

You know, with these models, we've built like nuclear fusion. We've built this incredible, unlimited source of energy, just like we built this factory for intelligence. It's not perfect. There's a lot we need to do to improve it, but it's really powerful. we forgot to build power lines and, put electricity in people's homes and figure out what an outlet should look like. And, you know, build some electrical appliances and some blenders and, you know, TVs and other stuff like.

you Mm-hmm.

We've, we've built this extraordinary thing and we need to figure out how to connect it to people and build applications with it. You know, the other metaphor being like, you know, we're in the Fortran days and we really need to invent Java. mean, I'd settle for inventing Pascal or like, you know, I, you know, C it doesn't have to be C plus plus I'll take C right now, but I'm thinking a lot about like, you know, my customers, cause I'm spending a ton of time with them now. Like it's cool being at Databricks where we have 12,000 customers.

Hmm. Yeah. Mmm. Right. Mm-mm. Right.

And I can just pick one out of a hat and go call them up and learn a ton. But how do they, what do they do with this stuff? How do they actually shape this into something that solves their problem in the same way that like inventing the internet was great. We needed someone to come along and invent the worldwide web and HTML. And then suddenly anyone could build a website and you had an explosion of creativity and people could say crazy shit. Like, you know, I'm going to build an encyclopedia that anyone can edit, which sounds stupid.

Mm. ⁓ Mm-hmm. Right.

and ends up changing the world. But it's like, how do we get from the internet to the web or from nuclear fusion to like TVs in people's homes? And so I think a lot concretely from a technical perspective. Like I think a lot about specification, by which I mean, you want an AI system that solves a problem. How do you even describe what you want? Like what it would mean to solve the problem and what it would mean for the AI system to do what you want.

Mm-hmm ⁓ Right? Hmm.

How do you like turn that intention into specification? And it's more than just like building an eval set. That's a cop out, like build a benchmark, total cop out. It's really hard to build a benchmark. There are very few good ones in the world. And there are all these automated tools to try to help you. And I don't think any of them work that well, including, know, you know, I've taken a couple of cracks at it I want to do better. So how do we help people specify what they even want? Cause you can't get something that meets your needs until you can even describe what your needs are in a pretty precise way.

Right. Mm-hmm. ⁓ Yes. Yeah.

And then how do you build that? And that could be, how do you give people the right tools and frameworks and ideas and concepts to build it? Or it could be, how do do that automatically once you have a specification? But I'm thinking a ton about those two problems. Like what does specification look like? Is it thumbs up, thumbs down? Is it just someone writing down in a couple of sentences? I want an AI system that does this. Is it your set of prompts that you've collected, even if you don't have any good responses? there are a lot of ways to say what you want, but it's like software engineering.

Mm-hmm. Hmm.

You need a, you need a way to build unit tests and a way to build integration tests and regression. Like, you know, testing and software is so rigorous and there's disciplines and frameworks and ideas for how to do this. And none of that exists yet in AI. ⁓ but also creating a test for an AI system that is not just a deterministic program is really hard. Programs are in some sense, self-documenting and self-specifying, cause you can kind of look at a well-written program and know what it does. You know, there are going to be bugs, but like.

Mm-hmm. Hmm Yeah. ⁓ Mm-hmm. Mm-hmm.

It's at least going to kind of do the thing you want. And you can tell just by looking at the code, there's nothing like that for AI. So it's much harder problem. So it sounds kind of abstract. I'm doing a bunch of concrete stuff there that I can tell you about, but those are the power lines and the substations and the transformers and the, like the outlets and the electrical system in your home and the blender. Like that's the, you know, and the end thing that the user comes and wants is I just want to make a smoothie, but.

Mm-hmm. Yeah. Mm-hmm. Yeah.

Mm-hmm.

For you to sit down and say, I'm going to make a smoothie, having nuclear fusion, there's a lot between A and B.

Right. Yeah, I guess that's true. It's easy to just overlook that there's as much engineering in this blender as there is in getting the electricity to the blender.

Yeah, it's, there's kind of like, and I think it's like, there's an enormous amount of science to do and figuring out how to get the electricity to turn into a smoothie. Like this technology is far more powerful than we've been able to actually utilize properly. I think that if you froze progress in this field for the next five to 10 years.

Yeah.

you would continue to see an explosion of innovation just as we figure out what to do with this. The same way that like TikTok didn't come out in 2001, you know, who thought that like weird videos like played to music was going to be a popular thing, but it turns out, you know, that also changed the world.

100%. Yeah. Marshall McLuhan has this like concept of our old media gets enveloped into our new media. it's that each time we sort of evolve, we shrink wrap our old media inside of it. ⁓ So television shows over the internet streaming, know, and television shows are, you know, we're like variety shows that were plays on stage and everything just envelops our old media. And it's probably some concept of just... the fact that familiar matters and we only want to deviate so far. Um, but I was thinking about that the other day while I was building like my fourth graphical UI, that's entire purpose is to construct a prompt that the output of the graphical UI is a written prompt. And I'm like, this is, this is like a whole set of apps. Now we're going to create that are that are graphical UIs to create natural language prompts.

So you can look at that as a good thing and a bad thing. Like bad thing is this is really weird. Like the same way that like, you know, I don't know. Now you have all these people who are like, I need to compose a formal email. Let me just have chat, GPT generate it. They send me this long email and I'm like, what's the first thing I'm to do? I'm going to have Gemini go and distill it down into bullet points again. And so what did we accomplish? And you know, now, but, that changes culture because now where it used to be that writing a big formal email that looked nice was a sign of effort and a sign of care.

Yes. Yeah.

Now it's actually a sign of lack of effort and lack of care and sending someone three very clearly thought through bullets is going to be the new equivalent of a formal email, which I think it's, it'll be interesting to see that. But the flip side is like.

Mm-hmm. You

You Thanks. and spelling errors now become like, you wrote it yourself whereas before it was like too sloppy to proofread

Yeah. Yeah, yeah, yeah. Exactly. this is, you know, culture changes. It's the same way that like, you know, what is it in the, you know, or at least the apocryphal story goes that in the middle ages, like, you know, being fat was considered being attractive because it meant you had enough food. Like, you know, things, things change with context and it'll be interesting to see where that goes. But I think the other piece, the scientific piece to this is like, you know, let's suppose that you're just using Claude for all of your AI work. You're not going to fine tune a model. don't have access to the model weights.

Right.

Mm-hmm.

You're still in some sense, I would say you're still training a model. It's just that the thing you're training, your parameters are no longer weights and biases. Your parameters are the words you put in, but it's still a training process. You've just got natural English parameters. Yeah.

Right, right. It's still an algorithm. Yeah, it's still an algorithm and you're still adding values to that algorithm. It's just that they're being added in different ways. And it's still weighting because the more times you repeat something, like there's ways to weight it in the prompt. There's ways to weight it directly. It's all, yeah.

Yeah. And what if, what if I told you that like your prompt wasn't going to be words anymore? I was just going to give you a prompt that was all emoji, just like random emoji. But if I gave it to you, it would be so optimized for your task that your accuracy would go up by like 50%. You know, that's kind of, that's basically the same thing as fine tuning a model.

Yes! ⁓ Yes. Yes. Yeah, I've been playing around with using algebra, like physics formulas and boiling down my ideas, my prompts into like physics formulas and then using those formulas as new prompts, right? To see how well and how much more accurate it is than sort of writing it out. And I'm surprised at how effective it is. because it does sort of clarify. It takes the gray out of it, And yeah, is this language of prompting gonna end up being code? And we're back to like creating UIs that create code that tell LLMs what to do.

I see it's. It's computing. It's all the same. As far as I've learned about the history of computing, it's just the same set of patterns over and over again. We're creating new kinds of programs, but all programs are the same. You need to have a specification for how you want it to behave. You need to have some human comprehensible way of editing it, where when you want to make an edit, there's a predictable way of like, I want to make a change and I know how to make that change and the outcome is what I predicted. And you need to have a way of verifying the program, the specification match. it's, you know, this is This is computing since the beginning of computing and probably mathematics before that. It's just a matter of what forms do they take and how do we actually figure out the easy ways for humans to do this?

Yeah, exactly. Yeah. Right. Yeah. ⁓

That's interesting because I've been using ⁓ one of the products on the platform that Robb and his team have spent years building and I'm just using natural language to create skills and things But there are many times where I'm instead of like wanting to ask it to change a setting. I want to go in the back and like see where it is and delete the thing that I accidentally put in there. Like I almost feel like I need that visual check to see like the old interface. So I have this new medium where I can tell it what I want it to do, but there's part of me that wants to go back into the old medium and like make sure that it happened the right way, even though it's all kind of an abstraction, right?

Yes.

Yeah. And this is where, you know, you know, I feel like I'm getting old and getting left behind a little bit by the new generation of people who grew up, you know, came through high school or college, you know, with chat GPT, even like the generation of startups that are being founded now, like the whole user interface is just you talk to an agent. There's no GUI or there's no like, you know, there's no buttons and knobs. It's just tell the AI what you want. And the backend is figuring out how to do that. And whether that's a good idea or a bad idea.

Mm. Yes.

It's, it's clearly something that a generation of people are now very comfortable with in a way that I don't know if I'll ever be comfortable with. And I I'm talking like my parents now.

Yeah. I wanted to bring this up to you in this episode. the data you put out on the internet today isn't just for consumption by humans now, it's for consumption by people training one way or another, whether it's like some sort of search and prompt ingestion or it's actually being ingested, you know. and weighted and et cetera. But as companies think about like what they put out on the internet and saying like it's not now it's not all for the purpose of being consumed by a Google spider for searchability or human. But now, hey, this our future LLMs are going to be consuming this and and it will be trained. seeing as like that's your origins right like the the data mixology Do you see a whole like world and industry around? What? Text they produce and how they produce text and just put it on the internet for the sheer purpose of being ingested into an LLM

I hope so. There, so I, you know, I have, I have two big thoughts on this. One is, know, the world always moves a little faster than you think. And the world always moves a little slower than you think. Like, you know, we've had web crawlers forever. ⁓ we've had PDF parsers forever. These are just making up for the fact that we've locked information in really inefficient forms, like websites and PDF documents. We could have put them into structured JSON. We could have put them in behind APIs. And yet a lot of valuable data, data that people actually really do want to share is still locked behind this. know, Wikipedia, until recently people were scraping Wikipedia ⁓ for their LLMs. Like why can't there just be, you know, and there are dumps of Wikipedia that you can just now go and download.

Perfect. Mm-hmm.

that just have all of Wikipedia so that people leave the poor Wikipedia servers alone and let humans grasp them. So on the one hand, yeah, I think we're going to have new forms of making information available. Maybe behind MCP, will be kind of, you know, there will be knowledge sources that you can share with the model. Maybe you can charge for that or something like that. But on the flip side, I don't know, old things seem to stick around. ⁓ You know, I think we're going to be dealing with PDF documents for the next couple hundred years, and we're still not going to be able to parse them properly. You know.

I hope that's something we solve at Databricks, but it seems to be one of those impossible problems. We're still locking information in bad forms. And even if we stop today, the sum total of the world's knowledge is probably locked behind a bunch of PDFs and websites right now. And we still need to figure out how the hell to parse them, not locked behind some nice clean API where we can just query it, except maybe for the folks at Google who have spent a lot of time parsing this for their search index. So it'll move fast and it'll move slow.

Yeah. Yeah. Is there like a format though that you would think of like, build out a solid static set of FAQ pages, that are not dynamic, that are not in PDFs that would just be like, hey, put this out there because the next, you know, the next pass at scraping, we'll have a better shot at ingesting this. And what information do you put in it and not? Because obviously if it's tentative information, information that changes, you know, do not put it out there to reduce hallucination? Like is there any, should anyone think about that? Or is that just?

Oh yeah, people should think about that. I bet there's going to be a cottage industry of people thinking about this soon. And the same way that like search engine optimization, like Google didn't intend for there to be a whole industry of search engine optimization. They didn't put out a protocol for it. The only form of search engine optimization was pay Google money and show up as an ad. but you know, nonetheless, there's still a huge industry for this. There are still tricks whenever the algorithm changes, you know, people need to update things.

No, right?

You know, I don't know, I bet on decentralization, but I bet on in the same way, you know, I bet on a neural net that you train on enough data to do gradient descent in the right direction and eventually turn into something good. I bet on, you know, putting out cool new technology and capabilities in the world and humanity to, you know, try to make a buck. And in doing so, optimize for the technology as much as possible. It's kind of amazing how capitalism somehow just like, it's not perfect by any stretch, but it gets.

Yeah. Yeah. Hahaha

a lot of the incentives to move in the right direction for technological development and for things to run efficiently. It's kind of interesting.

Yeah, that's pretty true. Yeah. Yeah, I agree with that. think it's interesting concept. I think there's like still this idea of source of truth that companies will be responsible for, which says like, yeah, you may get it from other places, but if it matters and you want the truth, then companies will still be responsible for being that source of truth. But it sort of brings up one of those things we we've been talking about this whole time, which is incremental change. If we stay in that incremental mindset, we're going to say, cool. I don't need to talk to my customers because they're going to go to chat GPT or something else to get the answer to their questions. Not realizing that they spend an enormous amount of money just trying to start conversations with their customers in marketing and saying like, now, now you don't get to talk to them. Like you might regret that. Yeah.

I think there are going to be a lot of dumb things that happen that we all learn from I don't think those lessons are going to be catastrophic. Like the AI goes and enslaves all of us, but I think there's going to be kind of a nice long list of like, remember when that really dumb thing happened? ⁓ and in hindsight, 10 years later, we're all going to look back and go, wow, how could anybody have thought that was a good idea? But they were just early adopters who were, you know,

Yeah.

trying to do something innovative and we all got to learn from their mistake.

Yeah. Yeah. It's funny. You, I was having this conversation. have a buddy over at Amazon and the gen AI side. ⁓ and we were talking about what you said, the unlocking of like PDFs, right? and the fact that this stuff's all locked up and there's like a flip side there's a benefit to it being locked up, which means like when somebody writes something that's completely false and puts it in a PDF and puts it in a document repository. At least you know it's like very limited on who's gonna see it just because of how hard it is to see, you know? It's almost like a benefit because only a few people will even find it. And now all of your mistakes that you've made in all of those PDFs, if like ingested and... become the new source of truth for your organization because you've decided not to go through all of that data and not to properly curate it before you dumped it into the system, now becomes like wrong information that you're producing at scale. And if that information was locked into PDF somewhere and it just so happened that out of the 100 or 200 PDFs, this person that gave the wrong information randomly picked some PDF no one ever reads, It's okay. Now that you dump all of that bad information into a system and make it useful, you also make it available. ⁓ and I guess there's just a big danger in underestimating the curation yeah, unlock it, but curate it

Yeah. I I think you're making a bigger argument about just digital technology and the internet age. Like, I mean, you could say the same thing about self-driving cars, like overall fewer mistakes are being made, but if there were some like really nasty bug that made all the self-driving cars from a particular company suddenly accelerate in certain situations. Like you send out that software update and you know, you could have a lot of people hurt in a day. Whereas, you know, one driver who takes, you know,

Yeah.

who doesn't get enough sleep or drinks too much, they can cause immense damage, but at least it's localized. And I think it's true with data breaches where like, yeah, we aggregated all of your social security numbers into this one big database. ⁓ It's very efficient until it's very bad.

Yeah.

You it feels like too, that brands have shrinking opportunities to maintain their reputation in a world where like, like we were talking, like I might ask an LLM for some questions that might direct me towards a brand. And then maybe I'll go to the website cause I'm, old school, but you know, there's a growing number of people who are just going to start getting more and more information. through this feed where the brand has less control over how they're being represented because the answer's coming like maybe like you said Rob, from old PDFs or just whatever has entered into the knowledge base.

I don't know. think it's going to be a big issue. Like I think just in general, at least with Google search, for example, you know, Google can really curate what information is and isn't available and, know, deprecate old content or, you know, they're even in Europe, the right to be forgotten. can have old irrelevant content about you removed. And, you know, these are now more powerful search engines where we don't have granular control over whatever data is built in.

Yeah. Mm-hmm.

In an ideal world, we'd love to separate knowledge from reasoning. Unfortunately, those two can't really be cleanly separated, like just philosophically, but the idea that the LLM is a giant reasoner, but you just hook it up to some kind of knowledge base. And it's a very faithful to that knowledge base. Like you give it access to Google search and that's a way of, you know, that's a way of curating what data or knowledge it has available.

Yeah. Right.

You, you know, if you want it to stick to your brand guidelines, you only give it access to the latest version of your brand guidelines. And suddenly you can just plug in the new knowledge and it behaves the right way. I think we're still pretty far off from that separation. ⁓ I'm sure it's something that a lot of people are thinking about at the big labs and certainly something I think about a lot in my science. But, you know, maybe that's a good place to wrap up is just, think in the future, we're going to have to have a clean separation there. ⁓ or, know, we're going to.

and Yeah. Mm-hmm. Yeah.

be subject to this just old irrelevant or out of date or miscombined information that we have no way to really control because it's just somewhere in those trillion parameters of weights.

Yeah. Well, cool. Thank you. Another good one.

Yeah, it's been really

Thank you so much for having me as always. This is always just such a treat.

fascinating. Yeah, thanks, Jonathan.