Learn how to use AI at its exponential with Anthropic's Head of Platform Engineering

0:00 Katelyn Lesse: Claude is actually one of the best at taking action. It's really good at choosing to call a tool and get something done as compared to just saying words at you and giving you an answer. Kind of obsession with, I'm feeling the magic, I'm accomplishing something, I'm getting something done is actually kind of fueled in part by just what Claude is actually good at. And yeah, I mean, people should get some sleep. People should definitely get some sleep.

0:23 Georgie Healy: For the founders out there, what is your framework for making products? Any way that you would suggest approaching.

0:30 Katelyn Lesse: If you're building systems, applications that are powered by AI, the first thing we say to everybody is having evals really, really important. And then what I would say is you should actually probably be building on kind of the edge of stuff. The models may be like, just barely good at or maybe like not even really that good at yet. And if you're running into problems where you're like, oh, the model's like not quite good enough to solve this thing yet, like probably a sweet spot, because you can almost guarantee that the next set of models will probably be really good at that thing and you won't have built something that is not super relevant.

1:03 Georgie Healy: Quickly, a weird fact that most people wouldn't know about you, maybe even your friends.

1:07 Katelyn Lesse: A really weird fact about me is that the bone in one of my fingers is made out of coral because when I was in high school, I broke it and as part of putting it back together, they like filled in a good chunk of the bone with coral and it just works like a normal finger.

1:28 Georgie Healy: Hello and welcome to In the Blink of AI I'm Georgie Healy and I'm thrilled because we have the one and only Anthropic on the show today. We're joined by head of platform engineering, Caitlin Les. This is a huge deal, guys. This is a huge, huge deal. Anthropic launched their Sydney office and this was their first interview they did in Sydney. What a pleasure. What a delight. If you saw my laptop covered in Anthropic stickers, you would know how big a deal this is. We talked about everything. We talked about comprehensive agentic harness. Three words I'd never heard in a sentence. And even for non technical people, you're going to find out why that's important.

2:09 Georgie Healy: Caitlin really cares about removing friction when it comes to AI agents while still keeping them safe and secure. We had spicy questions around competition at the Frontier Labs. What people get wrong about Anthropic? And is this fever pitch obsession with Claude even safe? Is it healthy? What's her take on maximum dose per day. And of course we finish with rapid fire questions. Guys, this is such a delight. I'm still in a high after this conversation. I know you're going to get so much out of it. Tell me what you think. Let's dive right in.

2:51 Speaker C: Founderscale faster on deal. Set up payroll for any country in minutes. Hire anyone anywhere, get visas, handled fast and get back to building. Visit deel.com day one. That's-e l.com day one.

3:12 Georgie Healy: I want to start us straight up with your AI hack of the week, please.

3:16 Katelyn Lesse: Okay, so my hack, every time I go to a restaurant, first thing I do, take a photo of the wine menu, send it to Claude. Claude's got memory and knows exactly what I want. It'll usually land me on a chilled red or a pinot. It knows. And so yeah, highly.

3:34 Georgie Healy: I can't believe you're talking about wine. I am just coming off the back of a three day wine country experience in orange, which is like a rural New South Wales town. Are you into the Aussie wines? Have you tried any?

3:47 Katelyn Lesse: I have not.

3:48 Georgie Healy: I have not. We, we do a good Pinot gris, which is. And probably a good chilled red as well, if that's your style. I'm going to ask Claude what a good Aussie wine is based on your preferences.

4:02 Katelyn Lesse: Yes. And you'll have to redo your trip and bring Claude along next week.

4:05 Georgie Healy: I definitely will. Look, we are so excited to have you, especially in Australia. Anthropic is just so well loved as a company here. But before we get into that, a little bit about your background so people get to know you. Princeton. Even I've heard of Princeton. What's the vibe there? And why did you choose to study computer science there?

4:29 Katelyn Lesse: Yeah, Princeton. So my very honest answer is Princeton was pretty close to home. My mom's number one requirement when choosing a college was no planes. It has to be a drive away. So I was limited in selection in kind of the tri state area in the US but went to Princeton. It's a beautiful place, a beautiful campus. People are very intellectual, which is fun. I think computer science can be very application focused and application heavy. So it was cool to study it in a setting where people are a little bit more kind of, you know, galaxy brained.

5:05 Georgie Healy: Can I also point out that I've got a few American friends in Sydney and we always make fun of them that they'll be like the tri state area and like no one knows what that means. Caitlin, what is the Tri State area? I'll explain.

5:18 Katelyn Lesse: I'll explain. I'm in safe territory here. Probably people won't recognize my accent, but I'm very much from New York.

5:25 Georgie Healy: Okay, great.

5:26 Katelyn Lesse: In the US you can hear it. Here, you probably can't, but the Tri State area is New York, New Jersey, and Connecticut. And the reason why it's called the Tri State area is because they're the three states that you could reasonably live in and commute to New York City.

5:39 Georgie Healy: Amazing. Okay, well, we've already unpacked some. Some topics that I feel like needed to be unpacked. And you, you were the head of engineering for Core Connect at Stripe, and then you joined Anthropic last year. What inspired you to make the move?

5:55 Katelyn Lesse: Yeah, the really interesting, I guess, position that I was in in Stripe was I was working with an engineering team, and everything about the way that my team was getting work done was just starting to change as AI tools were coming about, proliferating. And it's not even just in work, in life. I'm taking pictures of the wine menu and sending them to Claude. And so there was clearly just a big wave of how technology was about to change that I was fascinated by, but kind of also a little bit terrified by. I think AI is. It could be something that is excellent for the world and for humanity, and it's also something that could cause harm if it's not used in the right ways.

6:41 Katelyn Lesse: And so I was pretty fascinated by Anthropic, specifically as a company that had the exact mindset of, we want to build this technology, we want to put it in people's hands, but we want to do so in a way that will be really safe and that will only kind of aim for those really great outcomes and be really careful about avoiding the negative outcomes.

7:00 Georgie Healy: Yeah, fascinating. You say that, and you've obviously got a very technical role at Anthropic, maybe quickly share what that is, and that values alignment in technology. And technically, it doesn't seem to be mutually exclusive that you're in a. Yeah. In that kind of safe space as well as a very technical space.

7:22 Katelyn Lesse: Yeah. So my role, I lead the platform engineering team at Anthropic. Platform is primarily our API. So anyone who's building anything, whether it be a product they're exposing to their users or automation they're building internal to their company or even personal projects. Anything that gets built on top of Claude and is powered by Claude is built on my team's APIs. We also own a good amount of the Kind of product infrastructure that powers all of our products. And so the way that I think about it is products like Claude Code or Cowork are just some of my best customers. I guess in building on the platform,

8:01 Georgie Healy: I know those customers. How amazing. Incredible. We're going to get a little bit deeper into that. I asked Claude about you and it said that in Claude's words, you keep a low profile. And so I was like, okay, challenge accepted. I will stalk every social media. And it was really hard too. So you in the hot seat and now I get to pick your brain a little bit. You're holding a coffee. What's your go to coffee order?

8:30 Katelyn Lesse: So in the us, my go to coffee order is a flat white. In Australia, I've learned about a piccolo, which is the most perfect ratio of espresso to milk that I've ever experienced.

8:41 Georgie Healy: Which is what?

8:42 Katelyn Lesse: Oh, I don't know. I don't know.

8:43 Georgie Healy: I'm sorry.

8:44 Katelyn Lesse: Well, this is a tiny cup. That's all I got for you. It's a tiny cup, so not a lot of milk and more espresso. So it's great.

8:51 Georgie Healy: We love that. It's also breaking the third rule, the afternoon. So. Yeah, the times difference is still. Still working its magic on you.

8:59 Katelyn Lesse: Right. Went to a long lunch. It's time for a piccolo.

9:04 Georgie Healy: Okay. Cats or dogs?

9:06 Katelyn Lesse: Dogs. I grew up with dogs. Love dogs.

9:09 Georgie Healy: Yes. And average screen time per day.

9:13 Speaker C: A lot.

9:14 Georgie Healy: A lot, a lot. I jailbreak my own phone now. Like, I've got all these timers. I don't know if you do this as well, but these tech timers. And it's like, that's enough. And the screen goes black and white after 9pm but then I just jailbreak anyway. I don't know why I'm.

9:30 Katelyn Lesse: Right. So I'm in the really silly category where actually the only app that I have it set up for is Instagram.

9:35 Georgie Healy: Yeah.

9:36 Katelyn Lesse: Because I. I've convinced myself that scrolling on X is like, important to my job.

9:41 Georgie Healy: That's works.

9:42 Katelyn Lesse: Yeah, it's work to say on top of the community. So my phone doesn't yell at me when I spend too much time scrolling on X, but it does yell at me when I spend too much time scrolling on Instagram because that's kind of like a little mindless, but every day. Hit the ignore for today button.

9:56 Georgie Healy: Oh, that's a good one.

9:57 Katelyn Lesse: Yeah.

9:58 Georgie Healy: Favorite form of touching grass. Then too much screen time. What are you doing?

10:03 Katelyn Lesse: Yeah. So it's actually cool. I live in San Francisco now, which is fun. I'm From New York. And the cool thing about San Francisco is it's a lot easier to touch grass, literally, than it is in New York. And so it's been cool exploring the Bay Area, hiking, going up north to Marin and other places like that. So love to get on.

10:22 Georgie Healy: It's so funny. Spoke to Zach from Anthropic recently and he is a big camper as well, but he's from Australia. There must be something in the air in San Francisco where hiking's big.

10:33 Katelyn Lesse: Yeah, I think also just, it's perfect weather most of the year.

10:37 Georgie Healy: How lovely. And last one, a weird fact that most people wouldn't know about you, maybe even your friends.

10:44 Katelyn Lesse: Okay. A really weird fact about me is that the bone in one of my fingers is made out of coral because when I was in high school, I broke it. And as part of putting it back together, they like filled in a good chunk of the bone with coral and it just works like a normal finger.

11:02 Georgie Healy: I did not know that was a thing.

11:05 Katelyn Lesse: Apparently.

11:05 Georgie Healy: Did you know that was a thing?

11:06 Katelyn Lesse: No, not until it happened to me.

11:08 Georgie Healy: Have you met anyone else with a coral bone?

11:10 Katelyn Lesse: I have not.

11:11 Georgie Healy: That's really special.

11:12 Katelyn Lesse: Crazy times.

11:13 Speaker C: Wow.

11:14 Georgie Healy: Yeah, I'm thinking of like bright orange coral too. Like, I guess it doesn't matter what color it is.

11:20 Katelyn Lesse: I don't know. That's a good question. You know, I've never even thought of that. I have no idea what color it is.

11:23 Georgie Healy: Oh my gosh. Okay. Obsessed. I think I could do that for another hour. But I won't, I won't. I will stay on topic. So.

11:38 Speaker C: Found a scale faster on Deel. Set up payroll for any country in minutes. Hire anyone, anywhere and get visas handled fast. So you stay focused on scaling. Deal takes care of onboarding, hr, it eor benefits and compliance so your team can grow without borders. It's why more than 40,000 fast growing companies trust Deel to move fast. Visit deel.com day one. That's D E L.com day one.

12:12 Georgie Healy: I did read that you wrote this and I loved it. You said people are building products on the exponential. What do you mean by that? For the listeners?

12:21 Katelyn Lesse: I guess the way that we think about the capabilities of models. So how good that Claude and other LLMs are getting, we call it. It's an exponential. Right? That's how quickly the models are improving in their capabilities. So when you think about building on the exponential, we're essentially talking about people who've snapped to that curve and the things that they're building or even just the tools they're using. All these things are keeping up with the capabilities of the models.

12:47 Georgie Healy: Yeah, which is fast, right? So fast. It's incredible to watch. And you're in Sydney this week launching the Anthropic Office. You're in Singapore last week. I know you haven't been here long, but have you noticed any patterns in the Australian startup ecosystem that might be different from the US or similar?

13:05 Katelyn Lesse: Yeah, a few things. So different is actually the usage of AI here is really high. There are stats on just like per capita usage in the world and Australia is up there. I think it's like number four or something like that in the world, which is really cool. And so everyone that I have met so far is very AI enabled. They use the tools a lot, which is great. And then I think that a lot of what's being built is, I think, disproportionately high in terms of you're building stuff that has national importance. So a lot of people are building in regulated industries like health or finance. Less of the I'm going to build a little tool over here and a little tool over there.

13:50 Katelyn Lesse: Right. Like the ways that people are building, whether it be startups or large enterprises, just have really high consequence, which definitely factors into how people need to think about the technology and how they're using it and how they're governing it and things like that.

14:04 Georgie Healy: It's so exciting. Like when I have an AI podcast and an AI substack and I'm clearly AI evangelist, but whenever I speak to people that are maybe a little fearful or not happy about it even they agree that the health, the health space and curing diseases and things like that is so exciting.

14:23 Katelyn Lesse: Right.

14:25 Georgie Healy: Quite consequential to your words. And there's genuine fever picture around Claude and the obsession with Claude. We talked about Instagram before. I keep getting these reels about people just like locking in and doing these all night marathons on Claude and they're going to, you know, make no mistakes and all these memes and it's huge and look guilty. I've got stickers all over my laptop like, like I'm Claude pilled for sure. What are your thoughts? Is it too far? Is it too, too evangelical? Like, what do you think?

15:00 Katelyn Lesse: Yeah, I think so. Clawd is. It's. It's been trained to be good at things that make people love to use it, I guess is maybe what I would say. Like a really good example is Claude is actually of the LLMs, one of the best at taking action. It's really good at choosing to call a tool and get something done as compared to just saying words at you and giving you an answer. And so I think the kind of obsession with, I'm like, I'm feeling the magic, I'm accomplishing something, I'm getting something done, is actually kind of fueled in part by just what Claude is actually good at. And yeah, I mean, people should get some sleep. People should definitely get some sleep.

15:42 Katelyn Lesse: And I also really appreciate having, you know, we birthed a meme culture, which, like, you know, that's, that's, you know, an accomplishment to be part of. It's always fun to be part of meme culture. And so all of that is really cool and really great. But I think the reality actually is people are just building really cool things and getting a lot of really cool things done. And so it's, it can be a little bit addicting. And I think it makes sense.

16:05 Georgie Healy: I, you know, side note, are you in one of the slack channels, just sharing memes within the company? It's like, that's a good one. Like, I love that one. I would be high fiving.

16:16 Katelyn Lesse: Yeah, exactly, exactly. Gotta have some fun.

16:19 Georgie Healy: Okay, so for the founders out there, what is your framework for making products, you know, that, that maybe are not too heavily reliant on a previous model or something like that? Is there, is there any, any way that you would suggest approaching.

16:36 Katelyn Lesse: If you're building systems, applications that are powered by AI, the first thing we say to everybody is you've got to have evals. And so evals is really, it's tests. You have tests for how your system works and new models are coming out, or you're changing the harness around your agent, whatever it might be. And you need to know that you're going to get the outcomes that you want at the quality bar that you've gotten previously or better. Right. And you continue to improve. So having evals really, really important. And then what I would say is you should actually probably be building on the edge of stuff that the model is maybe just barely good at or maybe not even really that good at yet.

17:19 Katelyn Lesse: And if you're running into problems where you're like, the model's not quite good enough to solve this thing yet, probably a sweet spot, because you can almost guarantee that the next set of models will probably be really good at that thing and you won't have built something that. That is not super relevant quickly.

17:36 Georgie Healy: I've never heard that take before. That's really cool. So, like, I'm trying to cast my mind back, even in this room. I was talking about oh, it always falls over at the API key stage. Like it'll suggest getting an API key and then it'll always be wrong and like it'll, you know, no matter which one I select, it doesn't connect. Any tips on what might be at the edge now or anything that you're noticing people are trying to do and it's not quite there yet?

18:02 Katelyn Lesse: Yeah, I think it tends. Maybe the broad category that I could say is tasks that are pretty autonomous and pretty long running. That's where we're trying to get the models really, really great at being able to work for longer and work on harder tasks. And so if something you could fire off an agent to go get something done for you and maybe it'll kind of break early. Right. The context window is too full and it can't handle clearing it out correctly and start to hallucinate a little bit. These are the sorts of things that happen when you're really on the boundaries of what the model is capable of. If you're kind of noticing that you're probably actually onto a category of problem, that will be a really interesting and important problem to solve with the coming models that will probably actually be great at those things.

18:51 Georgie Healy: So you can kind of pat yourself on the back if you're getting frustrated a little bit. That's so good. Okay, so let's talk about leading the developer platform, which is what you do, encompassing APIs, SDKs, documentation, console experiences. Do you sleep at night? For starters. And then what's the non technical explanation for what you do and care about most right now?

19:16 Katelyn Lesse: Yeah, I sleep a little bit. A little bit. But yeah. So I guess I would say the thing that I care about is actually going back to that exponential that we talked about. So model capabilities are on an exponential. And what my team's job is is basically anyone in the world who's trying to get tokens in and out of CLAUDE that runs through our systems. And what we need to actually do is not just give you tokens out of CLAUDE through an API or otherwise. We actually need to give you the tools to be able to ride that same exponential of model capabilities. Without really powerful tools. Many businesses people are on what we kind of describe as like a linear, it's like somewhere below the exponential of model capabilities. Fancy.

20:04 Georgie Healy: But yes, most people are.

20:07 Katelyn Lesse: The world is messy. It's actually really interesting. People talk a lot about the models are getting better and better, how is the world changing? And I actually think there's already just a huge gap between where we could all be versus where we actually are in, even given how good the models already are today. Because there's maybe. I mean, the world is messy and things are hard and nobody's perfect.

20:29 Speaker C: Right.

20:29 Katelyn Lesse: But I think there's a lot of work that everyone kind of has to do to piece together the right infrastructure, to piece together the right tools in order to actually build something that unleashes all of the power of the model. And that's what we think about as our mission on the platform is giving people the tools to not just get tokens in and out of Claude, but to really ride that same exponential of model capabilities. Because we've given you the tools and infrastructure to do it.

20:54 Georgie Healy: That's so exciting. And is it because of the. It's a new skill set that people need to adapt to, or is it just like the technology hasn't been there before? Like, what is the gap? What do you think the reason the gap's there?

21:09 Katelyn Lesse: Yeah. So it's a bit of both. There's two things I would say. The first is a lot. You might hear a lot of people talking about harness engineering.

21:16 Georgie Healy: Yes.

21:16 Katelyn Lesse: And like context management and prompt caching and memory and all these things that. How many months would you have to go back for you to be able to say. I've never heard any of those terms. And now you hear them all the time.

21:29 Georgie Healy: First time I heard Harness, I was thinking, like, German, like, nightclub. Like, I was so confused with agentic Harness. And then I was like, no, right, exactly. But now I don't even.

21:39 Katelyn Lesse: But it was recent. Yeah, yeah, yeah. And now you hear it every day. So the expertise that actually exists in the world of people who can build a really, really excellent. And what a harness actually is, just like the code that sits around the model that turns it into an agent, makes it able to bring in context from external systems or call tools, so it can take action, these sorts of things. And so the expertise is already limited because it's just such new concepts. So that's, I would say, one part of it. The other part of it is, especially as agents get more autonomous, more long running, it takes a lot of, I guess, just specialized infrastructure to be able to support a system that's running in that way.

22:22 Katelyn Lesse: You've got to kind of have containers and servers that can spin up when the model chooses to do work or spin back down when it's not doing work, or else you're paying for a server to be running when you're not using it. You need to have an environment where, if CLAUDE chooses to Write code, it can go execute that code. And it's not terrifying because that environment is like a safe sandbox where it doesn't have access to a whole bunch of things. You need to be able to store transcripts so you can pick sessions back up with the model. There's a whole bunch of infrastructure problems there, and those are the problems that are still very hard to solve and the teams that we work with that are trying to solve those problems.

22:59 Katelyn Lesse: You can demo a prototype on your laptop using cloud code or whatever tool. But the gap between that and I'm running high scale Agentix systems in production that are reliable and high quality. That requires solving a whole bunch of infrastructure problems before you can actually get there.

23:17 Georgie Healy: And it requires solving problems. But do any of them require a Nobel Prize in physics or is it just time and compute and what is it that will get you there?

23:29 Katelyn Lesse: It's time, it's expertise. And I think again, just going back to the platform and the value of the platform, we just don't think that especially if you're maybe you're a startup, right? You're like a couple people, you've got a great idea, you're trying to build a product, you've got a whole bunch of taste for how to solve a certain problem, you're out building relationships with people and doing all the things you need to do to make a company successful. And now on the other side of that, you're like, I have to stand up all this infrastructure and I have to get prompt caching right, right? Like those sorts of things. That's again, just where we see the value of the platform to be like, take all those problems off people's hands so that they can focus on the more important problems that they should be focused on in order to build something that's going to be successful.

24:12 Georgie Healy: So exciting. And you know, it does feel a little stitched together at the moment. You know, people that aren't super technical, it does fall down like along the way. If I spoke to you in a year's time, do you think things would have progressed a lot? Like, is it hard to say at the moment? Like it feels very on the coal face of innovation right now.

24:33 Katelyn Lesse: Yeah, it'll progress a lot. I think it'll progress a lot. And I think one of the things that we try to say to people and we try to catch is if you're spending a lot of time on some of those types of problems that I think twofold, one stuff like our platform can kind of provide for you more and more Anyway, and we've got new things that we're launching every day that are helping people do those things. But also, as the models get better, you're actually going to need a little bit less of that harness engineering and scaffolding around the model in order to actually get really great outcomes. And so we try to catch people who are spending a little bit too much time at that layer because it's, again, just not the most important place to spend time.

25:16 Katelyn Lesse: And I do think once we get a year out, there will be a whole bunch of people who won't have spent any time on those problems, but they won't have had to because they're getting solved. And so I think people will kind of get back to building and focusing at the layers that matter a little bit more for creating a product that's. That's unique and successful.

25:37 Georgie Healy: Yeah, it's almost like they're over engineering now to solve a problem that could be solved. That's so cool. Removing friction. Amazing. Love that I'm lazy. Security, though, if you're removing all our friction. I saw someone created three buttons and it was like accept all, maybe even two buttons, accept all, accept most, or something like that. But then how do you ensure that you're comfortable with a frictionless harness or whatever we'd call it?

26:13 Katelyn Lesse: Yeah. So I think a couple things. I think the infrastructure itself that you're using needs to be built for this stuff. So I mentioned a sandbox environment earlier. I think a lot of people are doing work to make sure that if, again, the model's going to write code, it's going to execute that code. Do so in an environment that is limited damage potential. I've never heard a more perfectly named technical term than a sandbox. It's actually exactly what a sandbox is meant for. There's something in that where people just have to spend a little bit of time getting to know and understand the point of those types of solutions and infrastructure to be able to do things safely.

26:54 Katelyn Lesse: But some of this too is actually just like model steering and intentionality and how you do that. I think Claude does a pretty good job listening when you tell Claude, don't ever touch an API key or whatever it might be. And I think when people first start building, they're kind of caught up in the magic of I'm creating something and it's awesome. I think, again, taking a tiny bit of time to step back and say, what are the guardrails that I want to put in place or that I want to set for the tools that I'm using the agents that are running so that, you know, I get safe, secure outcomes is worth doing something Claude

27:34 Georgie Healy: does so well, is that multi choice question, like, are you sure about this or is this what you wanted? And I really love that specifically. And do you see that there would be a little bit more friction if there's like a payment or something like that? Like, I guess there's, there's, you know. Yes, you can draft up an email response, but I click send. Yeah, agent that away for me. Like amazing. I don't need to see it. But if it's like multi step and then a payment at the end. Yeah, like I'm just thinking of my mum who's listening. What we're talking about when we say like creating that sandbox and where it matters.

28:09 Katelyn Lesse: Yeah, yeah. So I think in products that people are working with, everyone's asking the same questions, right? Like what is the story for human in the loop? Like when should there be a human in the loop? And we ask that about our own products, like, like cloud AI cowork, cloud code. We talk to all of our customers who are building and they're asking similar questions like how much can you delegate to an agent that people will trust and be happy with? I think there will be a set of problems for which you're always going to want to have a human in the loop. And I think some of what we're thinking about is what are the right frameworks to make that successful, whether it be human in the loop or even just really excellent authentication or authorization on behalf of a human.

28:57 Katelyn Lesse: One of the APIs that we launched recently is called vaults and a vault can have credentials in it. The idea being an agent gets access to a vault and if that agent is going to interact with any external systems, maybe it wants to go shoot off a message from Slack, or maybe it wants to read your email. You can actually store keys within the vault so that the agent can act on behalf of yourself and the vault will kind of manage, like I'll go refresh those credentials when I need to and things like that. But just a very clean abstraction for how do we do that sort of stuff. Those are some of the problems that we're thinking about. So people can do these things and feel secure about them.

29:38 Georgie Healy: I would use that tomorrow, actually, even clearly you pass the test because I have an agent that will read guest pitch opportunities and say whether it's aligned to the show or not, which is amazing. And I don't know how I did without it before, but My imagination isn't very imaginative. What else should I be thinking about? What are you thinking about for agents? What should people be being a bit more creative?

30:08 Katelyn Lesse: Yeah. So one of the things that we launched recently is called CLAUDE Managed Agents and the idea is today or previous to this, our API, the Messages API, the very simple bare bones primitive like get tokens in or out of the model. We put some tools around it to make it more powerful but you had to kind of piece those things together yourself. We recently launched CLAUDE Managed Agents which is I guess I would say like a higher order abstraction set of APIs where you can like define an agent. Maybe you have an agent that I don't know, it's a data analyst agent and that agent configuration has MCP servers that it can interact with for external context.

30:43 Katelyn Lesse: It has, has skills that are the expertise it needs to do its job well, a system prompt, things like this, then you can just start sessions with that agent, you can schedule sessions with that agent and we're actually managing the harness and we're managing the infrastructure that spins up and down as needed to actually execute work and have the agent actually be able to do its job. As part of launching cloud managed agents agents, we've seen this really cool explosion of people being able to and products being able to incorporate agents so much easier than they could before and in really powerful ways. One of my favorite examples we worked with notion and notion built just like within their platform you can fire off agents to go get work done on your behalf.

31:32 Katelyn Lesse: A similar one is Sentry. Sentry is the bug tracking software and they built a bug comes into Sentry and they have a product called Seed and Seer sees the bugs and you can fire off a cloud agent to go and fix the bug that occurred straight from within the century product. And so there's stuff like that. But I guess I would also just say it's fun to encourage people to just think big, right? Like think outside of the box of what you normally would think about. And so some of the coolest customers that we're working with are working on problems in again the health space or space related, you know, like a whole bunch of different industries where again they're just taking their mind off the I'm building a harness and I'm building the infrastructure and doing all these things and actually I can kind of think a layer up from a product perspective and think about what problems am I solving and you know what is like the, the taste that I need to bring to doing that.

32:29 Georgie Healy: Well, I love how you Said like a data analyst. It's almost like, like if I was to hire someone who could do this better than I can do it, they have a multitude of skills. It's not just one task that they would do. So great. And by the way, the email was through notion. The one that I created is so funny.

32:46 Katelyn Lesse: Amazing.

32:48 Georgie Healy: Okay, so opinion on open source. MCP's open Claude is not discuss, please.

32:55 Katelyn Lesse: Yeah, so MCP. My team works on MCP. The core maintainers of MCP are part of my team. And mcp, from day one, we knew that this was going to be something that had to be open and had to be a tool that the whole community would adopt. And actually the whole point being, if you're going to build agents on top of our models or agents on top of any other models in the community, all of those agents are going to have to solve the same problem of safe and secure access to external systems. And so we should find a way to standardize that and a lot of good could come of that. And so we worked actually really closely with a whole bunch of people at other big tech companies and the Frontier Labs to create MCP and make it an important piece of technology in the community, which is really cool.

33:46 Katelyn Lesse: The closed models are. It's interesting because I think closed model essentially just means our code and our weights and all the things that actually power our models are things that we want to keep developing internally within anthropic, in part because so much of what we care about doing is making sure that we can run these things really safely. And this technology is just so new. Right. And we have to be really careful with everything that we're doing with it that I think over time our opinions on these things may change. I'm not sure, but I think for now at least it's something that we want to keep developing within our team and kind of in house to make sure that we can put these things out in the world in a way that will be safe.

34:31 Georgie Healy: So good to have you just to be able to unpack that as well. There's so much rhetoric around open source, closed source and the evolution. So that's fascinating. Let's talk a little bit about engineering teams. You've got an engineering team, an easy one. Should there be more competition from Frontier Labs? Do you think there should be more out there or for the builders in the audience, where do you see it heading? Less. More competition wise?

34:59 Katelyn Lesse: The way that engineering teams are working has changed so much and it's really actually cool to be a part of it. And to see how it's happening. I think in the past you needed just such a larger team to get done what you can get done with fewer people or you could take that really large team and just get even more done and get it done faster. I would say this is unlocking a ton for every team and the community. And what's been really important actually is the ability for people to be able to just choose the tool that's actually the best tool for the job and experiment with all the different tools that might be out there to get a job done. I think that's important.

35:41 Katelyn Lesse: I think for example, a lot of my customers, my biggest customers are cursor and GitHub and folks who are building tools within the space I mentioned earlier, like Century and their Seer tool. And so I think the way that engineers are going to get work done is not necessarily all going to converge around one specific tool. I think it's actually just really important that everyone experiments and figures out for different shape problems what is the best tool and how might that need to

36:08 Georgie Healy: evolve for the listeners? I will say sometimes I'm overwhelmed because I'll use different tools and I've got all these subscriptions and I can't pay for all of them. And then there's a time cost of money as well. Even just the Vibe coding tools which are so user friendly. I'm like, do I have to learn Lovable and Replit and do you have any advice there? Because yes, I want to play with all the tools. Even there's three different types of voices, voice enabled, whisper related terms out there and they're all different. I don't know which one. Help. What do you think?

36:45 Katelyn Lesse: Yeah, I don't have a helpful answer for you because my answer is just try. You gotta try it all.

36:50 Georgie Healy: Yeah, you do.

36:51 Katelyn Lesse: Especially because everybody, again, everyone's building so fast that each one of those tools is also evolving so quickly. An experience. Recently I had built an app in Lovable a while ago just to play around with it, see how it went. Similar thing with vercel and their V0 tool. And I maybe like a month or so ago I had a weekend where I was kind of like, I want to see the difference in experience with using each of these tools as compared to some number of months back, right when I, when I first tried them and both of them were like mind blowingly different. It's crazy. They're evolving so fast and you know, they're, they're all good tools and you kind of have to just like be Willing to experiment and for a given job, like, not feel too locked in on, like, there's only one thing that's going to get this thing done.

37:43 Georgie Healy: Well, yeah. I'm curious. Do you ever. It's just occurred to me now that I've been doing this and I don't know if it's smart and genius or the dumbest decision ever. I'm using Claude to kind of help me with prompting into lovable. I'm sure that's not necessary anymore, but I used to have to do that.

38:01 Katelyn Lesse: That actually there was a period where it was kind of best practice. It was to say, like, work with a different model. Like, tune your prompt. Right. Like, work together with a model, get to a good prompt, and then go and fire off that prompt in a different tool. I would actually say we mostly like, like most tools have built something like Plan mode to just, you know, take that same concept and build it directly into the product. So like in CLAUDE code, for example, we have Plan mode. You just like shift tab twice and you get into Plan mode and you go back and forth with Claude a little bit to make a really comprehensive plan of what you're going to go and try to build.

38:35 Katelyn Lesse: And then you, you get like, by far better outcomes for what you're trying because you've gone back and forth and made a really excellent plan. So I do think that these concepts are people. And again, is why it's great that there's a big community of people building with these things, because we're taking the concepts and figuring out, everyone's figuring out how do we build these things back into the tools.

38:55 Georgie Healy: Fascinating. And I didn't know Plan mode existed.

38:58 Katelyn Lesse: Give it a shot.

38:59 Speaker C: Tab.

38:59 Georgie Healy: Tab. Done. Okay, so we talked about this before. Your engineering teams, your engineers, the technical leads can do so much more than they used to do before. Do you think that will evolve even more? What are you seeing on the horizon? Are you changing? Like, I interviewed the COO of Vercel to speak about Vercel and talking about just. Even the titles of engineers are changing, like the go to market engineer and things like that. Where are we headed? What do you see? What are you excited about? Especially someone technical.

39:32 Katelyn Lesse: Yeah, well, we're all a member of Technical Stack, so speaking of titles, it's kind of the old Bell Labs thing. But yeah, I think teams are changing in a few different ways. One of which actually is just the lines are definitely blurring a bit between the different roles, between who's a user experience designer, who's a product manager, who's a technical lead, who's a tpm, who's an engineer. I think that the people who are kind of getting the best outcomes are people who are like, I just want to, like, think end to end about a problem and figure out what needs to happen in order to solve that problem. And I'll, like, push the boundaries of all the tools that I have.

40:15 Katelyn Lesse: Right. And everyone's still working together as a team and you absolutely still need the people with expertise and all of these different functions to be kind of the ultimate, I don't know, the person who says, like, here's what good, like, exactly what good really looks like. But you can get so much further when an engineer can say, you know, there's this problem that we've been thinking about for a while, and maybe in the past you would have said, that'll take forever and put it somewhere in a backlog that becomes a nice box and you never see it again. And now you could say, well, I'm going to take an hour, I'm just going to try something, or not even, I'm going to take a minute, fire off a Claude session or whatever type of session and see where you get.

40:52 Katelyn Lesse: And so I think the fact that that is happening means. And that doesn't have to be an engineer, right? That can be a product manager. We recently launched, actually, I think it was last week, week, a new kind of experience within our developer console that shows you how good of a job you're doing with our prompt caching, which is a pretty complex thing to get right. It's basically like if you're going to ask the model something and then you ask it again and the beginning of your question. Usually when you're building a product, the beginning of your prompt is the same. So you cache the first part of your prompt so that the model doesn't have to reprocess it.

41:28 Katelyn Lesse: And it's great because it saves you a lot and cost, it makes the model faster, all these things, but it's hard to get right. And so we launched a dashboard that helps you get it right. And it was actually built by a product manager on our team who had just spoken to enough customers who were like, everyone's struggling with this. Some experience to help you understand how well you're doing with it would be great. So we just kind of popped open a cloud code session and went at it. And of course, engineers on the team helped to make sure the thing ultimately got shipped in a way that was, you know, met all of our patterns and best practices. And security and otherwise.

42:05 Katelyn Lesse: But it was like built by a product manager. And so these are the sorts of things that teams are just in such a different space than they were even a few months ago, when a product manager would never be like, I'm going to try to build this whole dashboard myself.

42:19 Georgie Healy: It reminds me of the early days of like prompt engineering and everyone's like, you got to be a good prompt engineer. And that was like a title for a very brief moment in time. I did see a meme on Instagram of like, like, oh gosh, I'm gonna butcher it. But basically based on your prompts, we've kicked you out of the developer platform because they suck and there's tiers of how to do it well. But that's gonna be really good for people that keep running out of rate limits and things like that. Yeah, yeah, yeah. Brilliant. Do you. Cheeky question, but do you think non technical people need to push themselves a little bit more and try and get hold of the tools more or do you think the technical people need to start thinking more like product managers, even maybe think more about entrepreneurialism and things like that?

43:10 Katelyn Lesse: Yeah.

43:10 Georgie Healy: Is there a trend that you'd love to see more of?

43:12 Katelyn Lesse: Yeah. And I think it kind of goes back to like blurring lines and having everyone think of themselves as just like a problem solver and a generalist in some way. Right. And again, just being humble about the things that are not your. Your normal area of expertise and how you work with the team and these sorts of things. But you can get pretty far with just getting something started by thinking about yourself in a more general way. But I do think going back to non technical people using the tools, I actually think that applies for every function and everyone. There's just a huge spectrum between people who are kind of still writing code by hand versus people who are maybe not in a technical function, but they've figured out how to take all of the most mundane tasks that they do on a day to day basis.

44:00 Katelyn Lesse: Like the thing that you are doing to make sure that you can review those briefs and say, I'm going to take each of those and I'm going to think for a few minutes about how could I be using these tools. Better to take more of that off my hands or automate more of that. I think that exists across every function. I'm still doing stuff by hand, I'm still doing everything manually all the way to. I've thought through all of this and I'm like really pushing the boundaries of what the tools can actually do. Another reason we're excited about managed agents, by the way, because it should be very easy for most people to be able to say what are some things that are annoying tasks that I should just be able to go and automate and use that system to be able to do that.

44:42 Katelyn Lesse: So I think for everybody, it's just always be thinking harder and pushing the boundaries of what you could actually be getting done, and it's so worth it, right?

44:52 Georgie Healy: Like, I remember the first few times that just either an LLM or a vibe coding tool just worked and I. It feels like magic.

45:02 Katelyn Lesse: Feels like magic. Isn't it wild? It's like we actually. I was at a dinner the other night and someone asked this question. They made everyone go around and say, like, what was your. Like, aha. Magic moments with the first time, you were like, oh, my gosh. And I don't think mine is that interesting, honestly, because I can't. For some reason, I can't remember what it was. My favorite one is actually with my mom. I was visiting her at her house, and someone had sent her a bouquet of flowers. And she's looking at this bouquet of flowers, and there's this one flower in there, and she's like, actually, I'm not sure what kind of flower that is. I was like, oh, you want to see something cool?

45:34 Georgie Healy: Oh, my God.

45:34 Katelyn Lesse: Took a photo and asked Claude, like, what kind of flower is this? And it came up like, I don't remember what kind of flower it was, but showed her what kind of flower it was. And she was like, because we could

45:44 Georgie Healy: not Google that before. Like, there was no.

45:47 Katelyn Lesse: It's tiny, yellow and. Yeah, exactly.

45:50 Georgie Healy: My dad's one of those people that knows every tree. And I was always, like, super jealous of that skill set. I'm like, one day I'll learn every tree. And now I'm like, I don't need to learn every tree.

46:01 Katelyn Lesse: Exactly.

46:02 Georgie Healy: It is magic. It's genuinely magic. Okay. So I used to work at Google and there was this term called googliness, and it was like, around hiring, and it was kind of like a personality thing. And people would throw it around like, that's very googly of you. And like, it's actually quite cringe, but, you know, sweet and lovely and it was a positive thing.

46:21 Katelyn Lesse: Yeah.

46:22 Georgie Healy: Is there something like that in anthropic?

46:24 Katelyn Lesse: Yeah, so. Well, the funny thing about anthropic is internally we call ourselves ants, which we do. It's pretty interesting. There's like a whole bunch of little ant emojis for everything. There's party ants and there's all different kinds of ants. Very interesting because it feels like a missed opportunity to have been like the A's or something, you know, like. Exactly, you know, let's be.

46:48 Georgie Healy: Let's be competitive, guys.

46:49 Katelyn Lesse: Exactly. But probably goes to show the culture.

46:53 Georgie Healy: We're all working together.

46:54 Katelyn Lesse: We're working together, we're ants. But a term that we use is AGI pilled, which is, you know, just to say, like, are you kind of thinking on the frontier of like, what will be possible with AI and with the models? And so we. A lot of what we do on a day to day basis, you kind of like scrutinize with this lens of like, is that AGI pilled and you know, or like someone does something. Whoa. That's so AGI pilled.

47:21 Speaker C: Right?

47:21 Katelyn Lesse: Like, kind of similar to like googly, right? It's like AGI pilled. And so that is kind of our. How we internally describe, you know, you're. You're on the frontier.

47:30 Speaker C: Wow.

47:31 Georgie Healy: Okay. I like that a lot. Yeah, I like that a lot. I'll tell Google to think about it. Okay. To finish the interview. Rapid fire, quick answers. Are you ready to go?

47:41 Katelyn Lesse: I think so. Handmade piccolo.

47:43 Georgie Healy: Yeah, yeah, yeah. You're locked in. What's one thing people get wrong about? Anthropic or misunderstood about the company, do you reckon?

47:51 Katelyn Lesse: Yeah, I think probably how big we actually are, how big we are, how old we are. We have only been around for a few years. We're a few thousand people. We're not a massive tech company. And I think with the impact that we've had, you might think so, which has been pretty cool. But we're a startup.

48:11 Georgie Healy: I told you, my laptop's covered in anthropic stickers. Love that. Should I calm down? Am I too obsessed? Are we all a little bit too obsessed?

48:23 Katelyn Lesse: I don't think so.

48:24 Georgie Healy: Okay, good.

48:25 Katelyn Lesse: I think everyone wants the tool that is best for the job that they're doing. I said earlier, I think people particularly love Claude because Claude is really good at choosing to actually take action and call tools and get work done. Which is why it can feel like I love using Claude because I'm getting so much done. And so, I don't know, I don't think people should calm down. I think they should choose the right tool for the job. And if it's Claude, it's Claude.

48:47 Georgie Healy: I don't think my nails would survive if I tried to peel them all off. So that's good to hear. And then your recommended daily Dose, like what do you think? Should people use Claude when they have questions or should they use their brain first? What do you reckon?

49:01 Katelyn Lesse: I mean usually if you were going to use your brain first, it would just happen. Right, so the second you hit the like, oh, I wonder about, blah, blah, like you should actually pop open Claude and talk to Claude about it and ask Claude about it and have it be a thought partner. I think people are probably just like learning at a higher rate than they otherwise normally would be. And so I think, don't hesitate.

49:22 Georgie Healy: If I was to ask you this time in a year something that you're really proud of having done or worked on, what would it be?

49:29 Katelyn Lesse: Yeah, I think the. So we've talked a bit about how engineering teams are changing and the way work gets done is changing and I'm spending so much time with people who lead teams who are kind of working through this transition. I would love in a year if I could kind of look back and say I worked with a lot of people and had a really positive impact on how all of that evolved and just played a role in kind of the way that engineering changes as a function.

49:56 Georgie Healy: Yes. Okay, last one. And you can have longer than 15 seconds for this one. Closing messages to developers, technical founders, listening. We're all kind of, of living and breathing in this really fast moving and overwhelming time. Is there anything you'd like to share and, and any.

50:16 Katelyn Lesse: Just try stuff, just like push what the tools can do, right? Like if you've, if you've ever had that moment where you're like, oh, I've always wanted to like solve this thing. Right, but you like never took the time because it like felt like a time consuming thing. Just try just like fire off a Claude session, go have a piccolo, right, and, and see what you come back to. I think everyone's just gotta, you know, push on those sorts of things and then as a community we're all going to figure out a whole new way of working, of getting things done. And I think we kind of have to do that together and so everyone should just try and we'll see where we get.

50:51 Georgie Healy: This has been genuinely the fastest pod I've ever recorded, I think. Thank you so much for being so open and honest and sharing behind the scenes. We're all obsessed. Where can people find your work? Where can they follow you? Yeah.

51:07 Katelyn Lesse: So on X, because I already admitted that I scroll all the time at Caitlin underscore less is is me on X. But yeah, just like build stuff on Claude and that is, that is essentially connecting with me. I love it.

51:23 Georgie Healy: Thank you so much. Thank you so much for listening to In the Blink of AI. If you want to go deeper on anything we've spoken about today, I write a weekly substack called Attention Is All I Need. Yes, it's hilarious. It's a pun and essentially I go into AI rants, tech news, events I'm going to and more. It's bite sized and I hear it's awesome. The link is in the show notes below.