Benjamin Plummer (Dragonfly Intelligence) on AI's Real Bottleneck, and Why He's Betting on Australia

0:03 Georgie Healy: Can we start about Bridgewater? 9 years, the world's largest hedge fund. Why are you nice?

0:10 Benjamin Plummer: I don't think we've ever seen a company be so proud of losing control of its own software and having it do things that weren't intended. No, the police—

0:18 Georgie Healy: Public announcement. Yeah, the police were going to come over.

0:21 Benjamin Plummer: Exactly right. I mean, you've got something that's effectively locked in a box that you're making smarter and smarter, and its incentive is to break out of the box.

0:29 Georgie Healy: It does look good on a chart, doesn't it? It does.

0:32 Benjamin Plummer: It has a lot of pain and kicks and twists and turns underneath that chart.

0:37 Georgie Healy: You should have a lot more grey hair.

0:38 Benjamin Plummer: I think what AI is really changing is people with agency to go and create and build and do something different are going to be extraordinary beneficiaries. It's like pretty hard to attract great people into a sinking ship. And so while I don't think it's too late, there's definitely a window of time where if you don't Sort of get on board and start moving. It's going to be really hard to catch up.

1:01 Georgie Healy: Scan my nail for me.

1:03 Benjamin Plummer: What? There you are.

1:07 Georgie Healy: Oh my gosh! Show the cameras what happened. It is my YouTube page. I've got an NFC chip in my nail. It's the most genius thing I think I've ever done. Hello and welcome back to In the Blink of AI. I'm Georgie Healy, and after today's episode, you're going to be a massive fan of Benjamin Plummer. I definitely am. But I confess, uh, Ben came to me very strongly recommended by very smart people. But that happens a lot. I have the most incredible recommendations, and it took me a long time to schedule a call. Anyway, 4 minutes into that call, I was like, oh my goodness, I need to move some things around in my calendar. Ben is the co-founder of Dragonfly Intelligence.

1:52 Georgie Healy: He ran Invisible Technologies working with Frontier AI Labs, and before that, he spent 9 years at Bridgewater in New York City, the world's largest hedge fund. He's been working on AI since before ChatGPT was even a word in our lexicon. And today we get into some really big headlines like the Hugging Face hacking incident and the lack of accountability there, new bottlenecks in building businesses and its people, what does 9 years at Bridgewater Capital under Ray Dalio teach you, and software bloat. We've got more and more features and vibe coding and, uh, platforms trying to be some Like everything for everyone. We kick off the show with me asking Ben to scan the NFC chip in my fingernail, so make sure you watch that part on video and let's dive in.

2:46 Speaker C: Founders scale faster on Deel. Set up payroll for any country in minutes, hire anyone anywhere, get visas handled fast, and get back to building. Visit deel.com/dayone. That's d-e-e-l.com/dayone.

3:06 Georgie Healy: Benjamin, Ben, thank you for coming on In the Blink of AI. We always start up the episode with a hack of the week. Start us off strong. What's your hack of the week?

3:16 Benjamin Plummer: So I— one thing that I've been using a lot lately, and Andrej Karpathy, the famous researcher, tweeted about this recently, is the rambling session. Which is take a microphone, take your recording transcribing tool of choice, and just talk for 5-10 minutes explaining an idea, a concept, a problem you're wrestling with. And there's something like really powerful once you learn not to try and compress your thoughts into a sentence or two of a prompt, and you really give it all the richness of your thinking, the things you're not sure about, the nuance, and Feed it into the model. It's incredible how well it can distill all your rambling thoughts and play them back to you more coherently and clearly than you could ever imagine.

4:05 Benjamin Plummer: It's amazing.

4:06 Georgie Healy: What are you using to ramble?

4:08 Benjamin Plummer: Mostly WhisperFlow, or sometimes I just go have a granola meeting with myself for 20 minutes. Doesn't matter.

4:16 Georgie Healy: Can I tell you, I've been doing this, and it is one of the few AI tools, WhisperFlow, that isn't just a fun little hack that I then retire. It's incredible. And you're right. I do a lot of content. I do a lot of writing. I do podcasts, obviously. And it's about the vibe. It's not about having the perfect prompt and to get that feeling, that emotion. It's really hard to put that in a sentence sometimes.

4:41 Benjamin Plummer: And it takes some getting used to, like even pausing and gathering your thoughts is quite weird when you're in a sort of conversation, but AI will happily wait for 5 minutes if you need 5 minutes. And so you do need to sort of retrain yourself to allow yourself to really just sort of express the thoughts. I find, especially for creative things, it allows you to just sort of like really get in a flow and explain what's on your mind, whereas the typing can be really slow. So yeah, I use it all the time. I probably 5, 10 times a day. It's amazingly powerful when combined with the LLMs to clean it up on the other end. I think if you just got a long rambling list of your notes, probably not so helpful, but it's helpful in both ways.

5:24 Georgie Healy: So strong. Don't try this at home, or do. My best thoughts come like really late at night. The whole family's asleep. But with Whisper Flow, you can genuinely whisper. So I'll be in bed being like, great event idea.

5:38 Benjamin Plummer: Yeah, I have those in the morning sometimes, the same thing. Everyone's still asleep.

5:43 Georgie Healy: Okay, you're gonna be my guinea pig for the morning. You'd think I invented nuclear fission. I'm so proud of myself. Do you have your phone on you?

5:52 Benjamin Plummer: I do. What do you got for me?

5:54 Georgie Healy: Scan my nail for me. What?

6:01 Benjamin Plummer: There you are.

6:01 Georgie Healy: Oh my gosh. Show the cameras what happened. It is my YouTube page. I've got an NFC chip in my nail. It's the most genius thing I think I've ever done. I put my YouTube page on there. There's a lot of events this week. It's just a much more fun way of, you know, oh, what's your podcast? It's called this. then they may or may look it up on their phone. QR codes are not so whimsical. I find that fun.

6:26 Benjamin Plummer: Is that fun? Surely people have to remember that. That's not— I've not seen anyone doing that. That's awesome.

6:31 Georgie Healy: So I've got my YouTube page on. I'm gonna put my LinkedIn on my other thumb. I will make all my nails the same color. I just wanted to try this before the pod. That's my hack of the week. Thank you for humoring me. Okay, so speaking of hacks, I genuinely need to dive into this. Hugging Face, ChatGPT hacked it. It became like this— it felt like a celebratory moment where they're shaking hands and like so happy with the partnership of being hacked. I'm like, should we be more worried? Is this marketing? What's your take then?

7:04 Benjamin Plummer: Yeah, I think both are true, which is certainly I don't think we've ever seen a company be so proud of losing control of its own software and having it do things that weren't intended. And I think, you know, there's certainly an element of this where, you know, the idea of these models becoming more and more powerful, Anthropic doing a similar thing in the early phases of Fable, I think is a very consistent sort of pattern with that. I think on the other side, these threats are very real. And in this case, it wasn't sort of a malicious threat. It was the model basically trying to cheat on a test that it was doing, and that is a very common behavior for these models.

7:45 Benjamin Plummer: The way they're trained and the way they're incentivized to sort of achieve a goal means they will do basically anything they can to achieve that goal, which in this case includes breaking out of their own harness. So I think these sort of challenges are very real. I think the other interesting part of that story, which is sort of less reported, is Hugging Face actually caught it. about a week or so before OpenAI. They did so using open-source models. And so there's a very interesting sort of other side to that story around the balance of these things where sort of the either the threat or malicious actor side of things and the detection mechanisms really need to move in unison.

8:29 Benjamin Plummer: Or I think there's going to be challenges. I think the other sort of really interesting thing there is just around accountability and who's actually on the hook for this, which is, you know, in the OpenAI case, it's their models and their actors basically doing it. So it's a little clearer. But in a world where some other company using OpenAI or Anthropic models does something similar, I think it's going to be very hard to attribute who's actually at fault and who's responsible. In this case, there was no harm, there was no damage, but very easily that could be a different situation. And I think the sort of legal frameworks around the accountability for this is really unclear and something we're going to invest— need to invest a lot in.

9:14 Georgie Healy: So well said. Imagine a person, imagine you or I hacked into Hugging Face. I don't think it would be the—

9:23 Benjamin Plummer: No, the police would be—

9:24 Georgie Healy: Public announcement. Yeah, the police would come over.

9:27 Benjamin Plummer: Exactly right. And so like, I think that is, is sort of the undercurrent of this thing is really around who's responsible for these models as they become more and more powerful. And, you know, a lot of this is at the sort of harness layer that these underlying models are way stronger and more capable across a range of different dimensions. And these safety guardrails are meant to protect them from doing these particular things. But those are not bulletproof, as this points out, and as there's many examples online of people being able to sort of break out of those safety guardrails. And that is another challenge here where you sort of got these things that are increasingly more and more powerful that need to be contained and understood on the other side in terms of what those threats look like and how to protect against them.

10:18 Georgie Healy: Such a great point. I remember when we had Anthropic on the show, they were talking about agentic harnesses, and it sounds perfect, right? You know, this is the safety, this is the privacy, this is, you know, all our rules and regulations. And it's like, great, done. But since this happened, like, a powerful company like OpenAI doesn't have control of their harness, or—

10:39 Benjamin Plummer: Exactly. I mean, you've got something that's effectively locked in a box that you're making smarter and smarter, and its incentive is to break out of the box. And so, like, inevitably, what do you expect is going to happen, particularly if you have actors who are trying to get it to break out of its box? In this case, that doesn't seem like that was necessarily so, but certainly there will be others who are. I don't think this necessarily needs an incredible amount of alarm and blocking these models and things like that. I think really what it is about is recognizing that as these things become more and more capable, we need to be just more aware of the threats that are emerging in different ways.

11:20 Georgie Healy: One more thing on this. You mentioned, you know, Hugging Face being the ones that identified that this had happened. Anyone that's listening that might not be aware, it's, it's like a GitHub for AI models. This is a very advanced technical company that could identify something like that happening. Do you think this would even be identified with a less sophisticated company, or—

11:43 Benjamin Plummer: Highly unlikely.

11:44 Georgie Healy: Right.

11:45 Benjamin Plummer: I think you're talking about the real sort of upper echelons of technology capability within these organizations. Interestingly, you know, you could kind of argue that they're an indirect competitor of OpenAI and that a lot of ways they represent open source and a sort of alternative path. And so even that creates like a really interesting dynamic. Imagine you had some other company accused of hacking its effective competitor. You'd have a whole bunch of different questions. And so yeah, I think most organizations are totally unequipped to deal with the level of sophistication. You're looking at these models being able to find vulnerabilities that have existed in software for 20 years.

12:30 Benjamin Plummer: And this is like web browsers, core infrastructure of the internet that sort of every engineer in the world has had access to. And I mean, poking holes in these models are finding gaps in that core infrastructure. And so it's just a whole new level of capability that most organizations don't have. And I think one of the risks of, you know, companies that probably shouldn't be building software building software opens up is that, yes, it's much easier than it's ever been to build and vibe code and prototype things, but actually building robust, secure software is probably as hard as it's ever been.

13:10 Georgie Healy: Hot take right there. You're going to have so many opportunities for more hot takes. You're the CEO and founder of Dragonfly Intelligence, after a series of incredible career steps, which we'll unpack. But some, some things that I read on your website and on your blog that I'd love to unpack, one of which, AI expands coding capacity exponentially. We've seen that all the engineers have reported this, but then every other process in the pipeline is exposed as the bottleneck. What are some bottlenecks that you think are particularly risky?

13:46 Benjamin Plummer: Yeah, look, I think if you sort of focus in on engineering and software development to start, you know, either side of the coding activities, I think you're seeing people sort of slow down. As one, you have to think about what you want to build, decide what's important, what's not, have judgment around that. And that, you know, there's elements of that that you can use these models to help with. But, you know, humans, I think, are still really important in that process of sort of taste and judgment and decision-making. And on the other end of that is this sort of verification and validation of like, did the model actually build the thing I wanted?

14:24 Benjamin Plummer: And I think, you know, in a lot of cases, humans are still quite important there. And so what you're seeing is historically the sort of coding was the part that took all the time and there was much less effort on these other things. It's now sort of flipped in that the coding becomes really quick, but deciding what you want to build and then checking it was actually built becomes a real problem. And then if you zoom out, I think there's a much bigger sort of organizational societal problem, which is we're just being overwhelmed with software, which is because it's so easy to produce apps or solutions to things every day. There's 50 new applications, open source, closed source, that do one very niche thing.

15:08 Benjamin Plummer: And even in areas that I care deeply about and spend a huge amount of, I don't have time to go test them all out and figure out which one's good and which one's bad. And that's the same within an organization. People's ability to change and learn about all these new tools and capabilities is really going to become the bottleneck. And there's probably a backlog of 20 amazing apps or sort of frameworks that have come out in the last month that I'm desperate to try and just don't have the capacity to go figure out exactly what they do and how they fit within our stack. And so again, it's sort of back to the humans being the lowest common denominator.

15:48 Benjamin Plummer: I think people on the forefront are getting more creative around that, which is, okay, how do we use agents to actually go test 4 or 5 different pieces of software and give me an assessment of how they work and how they're different? And so we're slowly sort of abstracting our way out of some of those activities as well. But that's going to take time, that we're sort of very early in those phases, I think.

16:15 Speaker C: Founders scale faster on Deel. Set up payroll for any country in minutes, hire anyone anywhere, and get visas handled fast so you stay focused on scaling. Deal takes care of onboarding, HR, IT, EOR, benefits, and compliance so your team can grow without borders. It's why more than 40,000 fast-growing companies trust Deal to move fast. Visit deel.com/dayone. That's d-e-e-l.com/dayone.

16:49 Georgie Healy: I, I don't know if we can equivocate it exactly, but it does remind me of the complete, like, explosion of AI imagery for a short time there, and then everyone was, like, revolted and disgusted, and I haven't seen it quite so much. Do you think that's kind of Similar?

17:05 Benjamin Plummer: Yeah, it's— I think it's going to be really interesting in that like one of the things that's happened over the last decade, say, is I'd say generally software has gotten way worse. And it's gotten way worse because things— you start a company with an idea that's different, that has an opinionated view of how things should work. And then over time you just keep adding features and capabilities and it becomes completely unopinionated. You're trying to be everything to everyone. It's bloated with all sorts of different things that no one needs. Just go open up Slack, Excel, whatever, whatever software you want. There's a million things that you have no interest or need for.

17:45 Benjamin Plummer: I think there's an opportunity for AI to strip that back and end up with software that is purpose-built for you, that has every feature you need and none of the features that you don't need. That I think is much more of a sort of Apple approach to software development, which is like there's simplicity and beauty in minimalism. And I think that's really exciting in what it can do for companies and people. I think on the other hand, there's basically no barriers to shipping software now. And so there's going to be a whole bunch of noise and crap that people are going to need to sort through. And so there's a little bit of good and bad that I think is going to come from this over the next few years.

18:26 Georgie Healy: I know it when I see it. You know, those like monstrosity websites that it's every feature, we do everything, and it's like, just stop.

18:34 Benjamin Plummer: Every customer, every salesperson wants one more thing and they never take anything away. And that's sort of been, I think the last decade has been that shift towards just adding and adding and adding and not really taking away anything.

18:49 Georgie Healy: So agree. Before we talk a bit more about Dragonfly Intelligence, you have a crazy origin story. I remember when we first got on a call, my A4 page just got filled up very quickly trying to figure out what to talk to you specifically about. Can we start about Bridgewater? 9 years, the world's largest hedge fund. Why are you nice?

19:16 Benjamin Plummer: I think people fundamentally misunderstand Bridgewater. I think, you know, there's a lot of sort of quirks around how that place operates that gets a lot of attention, whether it's the recording and some of the tools. I think at the essence it's about this sort of search for truth, and, and that comes from trying to understand the world and how the world works. It comes from trying to understand each other and what we're like. And actually, when you sort of think about it, being direct and telling you what I think and when I disagree and when I don't agree with you and why, and when I think you're making mistakes is so much nicer than sitting here and thinking you're making all these mistakes and not telling you.

20:04 Benjamin Plummer: That's not how most companies operate, but that was very much an idea that Bridgewater leant into. And once you've experienced that, it's hard to ever go back. You're just like, how would it Why would it ever exist any other way?

20:20 Georgie Healy: I love that. I love when someone tells me to my face, I don't— I read your media deck and I didn't like this. I love that. That's so helpful. Yeah. Otherwise you just get crickets and then you wonder why.

20:36 Benjamin Plummer: And it's really interesting how in other domains people sort of really understand it. Like if you go to professional sports or whatever, it's like very clear there's a goal to win. And there's no surprise that, hey, you look at tape afterwards, you give each other feedback, you're out of position, you missed that. And it's sort of very clear that the purpose is making it better. It's not that you hate those people. It's not that you're trying to advance your own causes. It's like we are all playing a game. The purpose is to win. We want to get better. And I think Bridgewater sort of took that same philosophy around the markets in which they wanted to participate and win.

21:13 Benjamin Plummer: And it was the same mentality of like, how do we get better every day? How do we push each other? What type of environment and culture is required to enable that? And it's sort of very refreshing after you sort of go— it's, it's a little— it takes a little bit to adjust to. Honestly, I think as an Australian, it was way more natural to me than a lot of Americans feel, the, the sort of transition. I think we're naturally more direct and sort of culturally, I think, a little bit more aligned with that way of thinking. And it is interesting how different cultures react very differently to that type of behavior.

21:49 Georgie Healy: Yeah, hot take, Queenslanders should work at Bridgewater in New York City because I feel like we've given each other so much shit by the time my 20s like hit me.

21:57 Benjamin Plummer: Yeah, well trained.

21:58 Georgie Healy: I can handle it. And clearly a sucker for punishment, 9 years in a hedge fund and then New York City, you started a startup. startup. No, no, before that, Elemental Cognition. Tell me about that first.

22:11 Benjamin Plummer: Yeah, so, um, Elemental was an AI lab that we incubated within Bridgewater. This was probably 10, maybe more years ago. And the founder, Dave Ferrucci, who was the inventor of IBM Watson back in the day that beat Jeopardy, had a very clear vision for sort of next wave of AI and what some of the gaps were in the current sort of machine learning approaches and other things that were quite popular at the time.

22:40 Georgie Healy: And for the record, 10 years ago is before GPT-3. Like, well, before.

22:45 Benjamin Plummer: Yeah. Yeah.

22:45 Georgie Healy: No one's even—

22:46 Benjamin Plummer: And maybe even longer, but it was certainly at least 10 years ago. And so Bridgewater sort of seeded that and invested in growing that out. It got to a point where it was very clear that there was sort of much broader applicability than Bridgewater had use for. And so we spun it out and raised external capital from a bunch of sort of top-tier VCs. And I left as part of that and built the commercial side of that business over a few years. And it was just a fascinating time to dive into the AI space. As you said, this is a couple of years before ChatGPT, but it's very clear that these technologies are heading in a direction that's going to be much more broadly useful than pure sort of machine learning algorithms or anything like that.

23:30 Benjamin Plummer: So it was a really exciting time. Lots of challenges in taking an extraordinarily talented group of researchers and scientists and building a commercial business around them. So lots of learnings, but amazing fun. And I learned so much about these technologies, the limitations, the types of places they succeed and fail through that experience that's been really helpful over that time.

23:57 Georgie Healy: I have to know, when did you think AI might be a thing?

24:01 Benjamin Plummer: It's, it's fine. I mean, you, at every juncture it seems so obvious, but then you look back at what's happened since each one of those pieces and you're like, I had no idea what was about to transpire. And so I think those early days of Elementor, you just sort of have these aha moments where And I suspect everyone sort of had these in some of the early days of interacting with ChatGPT where you're like, wow, this thing really surprises me in ways that I have not ever experienced with technology or software. And so I was lucky enough to have many of those moments early on where it's clunky, it makes mistakes, it does silly things, but you see these glimmers of potential where it's very clear that the kinks will be ironed out.

24:51 Benjamin Plummer: And what remains is something that's like, incredibly profound.

24:55 Georgie Healy: And did Elemental give you, like, did you get bitten by the VC bug? The, like, what, like, what is it like to have a real startup? You're gonna go all the way to SF now.

25:05 Speaker C: Yeah.

25:05 Georgie Healy: Like, what was that thinking?

25:07 Benjamin Plummer: It definitely, I'd sort of gone, if you sort of follow my career, it's like every point I'd gone smaller and smaller and earlier and earlier stage. And so, I mean, Bridgewater wasn't big by any stretch of the imagination. We're about 1,500 people, but it started to feel big after a period of time. Elemental was a great way to move, still pretty closely connected with Bridgewater, but do something more entrepreneurial. And Invisible was the natural extension of that, which was a company I'd been advising with and helping from the early days of inception that I eventually took over as CEO.

25:45 Georgie Healy: Tell us how much revenue and profitability.

25:49 Benjamin Plummer: We grew a lot. So we went from about $10 to $15 million USD to about $160 over the course of 18 months, which looks really good on a chart.

26:03 Georgie Healy: It does look good on a chart, Ben, it really does.

26:06 Benjamin Plummer: It has a lot of pain and kinks and twists and turns underneath that chart.

26:11 Georgie Healy: You should have a lot more gray hair.

26:14 Benjamin Plummer: But just an amazing experience. We sort of ended up in an incredible position at the middle of I'd say the largest reallocation of talent and capital I've ever seen in my lifetime, but I think probably that we've ever seen in society. And so we had an amazing group of clients, we had an amazing team and got to do some really cool work.

26:35 Georgie Healy: My favourite part of the interview, you're back in Sydney, Australia proud, CEO of Dragonfly Intelligence. Tell us what it does. Tell us why you're excited after doing these incredible things to date that you were like, you know what? All in on this?

26:52 Benjamin Plummer: Yeah, look, I think in a lot of ways Dragonfly is the culmination of all of those things, which is, you know, one of the things that was really apparent to me in the work that we did in Invisible is just the huge disparity between what these technologies are capable of, how fast they're moving, and how fast they're adding capabilities, and how far behind the whole economy is in terms of being able to metabolise those changes and realise those benefits. And in some ways, the companies that have the most to gain are the least equipped to actually harness those capabilities. And so I spent a lot of time thinking about what is the right mechanism for sort of creating value and driving this change in this new era.

27:42 Benjamin Plummer: And it was clear to me it wasn't going to be SaaS. It wasn't going to be consulting. Companies were not going to figure this out on their own. And so the idea behind Dragonfly is we're a capital allocator and investor. We invest in buyer companies. We're a technology company. We've got our own proprietary AI platform and we're operators, we're entrepreneurs. And so we take these businesses, we rebuild them from the ground up as AI-native versions of themselves. and really unlock the potential that exists within these industries that, you know, in many cases haven't evolved in 20 or 30 years. And we think there's a really important moment in time now where AI sort of unlocks a number of constraints that have been holding these businesses back.

28:29 Georgie Healy: All right, ready yourself up. I have so many questions. Number one is, how do you identify a company, especially private markets, where I called them zombie companies when we spoke before. They may have got VC capital and they actually don't have any customers, or they're actually not making any money, or their churn is crazy, and you don't know any of this. Like, how do you identify good and how do you know if that's red flags?

28:56 Benjamin Plummer: Yeah, I think there's sort of 2 categories of, you could call them zombie companies. They play out slightly differently. I think there's a lot at the moment of these, I built some product that I find useful, therefore it should be a business. And lots of people running around trying to convert the first thing into a second thing and realizing that building the product isn't the hard thing, particularly now with these tools. Selling it, serving customers, building revenue, raising money, all of these other things were actually way harder. And particularly great engineers sort of solve the first thing very quickly and then very quickly run into these other sets of problems.

29:38 Benjamin Plummer: And they're smart enough that rightfully one or two companies will take a punt on them and they'll get some traction, which in some ways is worse because it sort of perpetuates the belief that they have a real business. And I think there's lots of those sort of different versions of that. Every week there's someone else pitching me some agentic harness thing that they've built that they're using that they're going to turn into a business. And the reality is, is like every great engineer has one of those now. You're not like building something that you're going to sell to the masses, and the average day person is not looking for an agentic harness. And so there's like a market—

30:14 Georgie Healy: It's not in my Google search history, no.

30:17 Benjamin Plummer: And then I think the other sort of category, and this is sort of much more, I think, strategic, is when each model release comes out, does the gap between your business and the alternatives get larger or does it get smaller? And there's a lot of companies that have built things that I go back to. Like in the early days there was, I can't remember the name of now, there was like a writing app that exploded. It hit like $100 million in revenue.

30:47 Georgie Healy: Is it Grammarly or one of those?

30:49 Benjamin Plummer: No, I can't remember the name of now, unfortunately. But explosion in revenue, it was really cool. It took these models that were kind of hard to use and made them slightly more usable. And it was like at the time one of the fastest revenue-growing companies you've ever seen. But then very quickly, like OpenAI could just do that and then Claude could do that and people are like, why am I paying for this other thing? And I think there's a lot of startups that are just ahead of the models that they've sort of taken the models and layered them or put a UI on them and It is helpful now. It's better than using the models themselves, but that's just a matter of time before they basically eat you up.

31:27 Benjamin Plummer: And that's a hard place to be in. And I think knowing where those models are going, and I describe it like a freight train, like they're coming and you better not be on the tracks. That's easier said than done. It's not totally clear exactly where—

31:41 Georgie Healy: Oh yeah, we're not judging. Like, it's not easy to do, but we see it.

31:45 Benjamin Plummer: But that's a really important question to ask, I think, for venture investors investing, saying, hey, how's this going to play out? Is this going to be consumed by the models? Is there real sustainable differentiation, which means this company is going to win over a long period of time? Or is this a flash in a pan thing which could have great success for a period of time and then likely dissipate? And you can still make a lot of money, you can build some great businesses, But those are probably not venture-scale businesses just given the time horizons that are involved.

32:18 Georgie Healy: Yeah, you need, I think for VC funds it's what, an average of 10 years?

32:22 Benjamin Plummer: Yeah, I mean there's a whole bunch of factors that meant that those distributions and returns have really dragged out. Companies are staying private for a lot longer. There's not IPOs, there's been less M&A. And so while those time periods used to be a lot shorter, they're really dragging on now.

32:40 Georgie Healy: So can I be cheeky and ask you to be specific? Because like, I'm thinking of industry verticals, but I've seen them get disrupted, you know, dentistry products that rely on image generation. And then when Nano Banana came, there goes that one. Like, what, what is it? Is it getting even more domain specific?

33:01 Benjamin Plummer: Yeah. So I think, like, to use a specific example, there's a lot of incumbent software, use Xero as an example, that have a really, actually have an incredible opportunity when you think about what they have. They have extraordinary distribution, they have millions of customers, they know a lot about them, they have a lot of data. But to actually move up the stack, they basically need to cannibalise their existing business and compete with their customers instead of providing accounting software, go be the accountant. And that in theory is a really easy thing to sit down and say, of course you should do that. That is the way out of the current predicament you're in.

33:44 Benjamin Plummer: Getting alignment across shareholders, board, CEO, leadership team, all the functions that have a vested interest in protecting what exists there is nearly impossible. And so I think I just use that as an example. It might not be the most extreme one, but I think there's a lot of versions of this where there's a theoretical obvious move a company should make to get themselves in a stronger position to move up the value chain, move out of this token eating into your margin problem, and actually solve the customer's problem. Don't just provide them a tool, solve their problem. But it's actually a lot harder to execute than it sounds. And so there's part of what we're betting on with Dragonfly that actually Starting with the services business and turning them into an AI-native business is probably faster and easier than taking a piece of technology and trying to turn it into a service delivery business.

34:41 Georgie Healy: You don't hear that take often. I like it.

34:44 Benjamin Plummer: Yeah, I mean, it's, it's, people have been attuned over the last decade or so that software is where the value's at. It's where the multiples are at. It's where the scalability is. And I think that is becoming less and less true, that it's really hard to build defensibility in software alone, that there's a lot of competition and sort of replication. You build something, someone else builds it the next week. And so I think we're at a tipping point where a lot of the things that were true over the last decade will cease to be true in the following decade.

35:17 Georgie Healy: I have never been more important with my charisma, I will tell you. I was so sad I couldn't code, Ben. I was like, Oh my gosh, I did chemical engineering, not software, my career is over. And I'm like, back in the game.

35:29 Speaker C: Good.

35:31 Georgie Healy: Okay, so I went so deep on the Chinese AI models because we knew about DeepSeek, that changed the game in terms of, wow, you can do a lot more with less. But with Moonshot AI and their KIMI models, and that there's just so many Chinese models. I read somewhere that there's a new KIMI model every 10 weeks. or something crazy like that. When you're looking for businesses to buy and they're private, what is your recommendation? Open source? Start somewhere. We'll work our way through.

36:04 Benjamin Plummer: This is such an amazing— look, I think my overarching advice to people is do not get yourself trapped in being stuck with any particular model provider or approach. Like, we're in such the early phases. We've seen the back and forth between even the frontier labs in terms of who's leading, who's most cost effective, who's better at what.

36:29 Georgie Healy: Yeah, Ben, I keep changing my stickers. It was OpenAI, now it's Claude.

36:33 Benjamin Plummer: You know, if you build your whole organization around one model, you very quickly could be stuck with the most expensive, the least capable, or whatever. And I think open source just broadens that continuum of options that you have. There are more complications with the sort of open source models that I do think you need a bit more technical depth and understanding around how to use them, how to put some of these guardrails. And so they're a little less user-friendly but extraordinarily powerful. And I think at the moment they're like a quarter to a third of all tokens are going through open source models. I think that's only going to increase.

37:13 Benjamin Plummer: I think there's a lot of companies that are mostly focused on building out capabilities first, and then we'll think about optimizing cost second. And open source has a really key role to play in that cost optimization.

37:27 Georgie Healy: I read that Kimi K3, I know we've got Opus 5 now, but it was a quarter the cost and similar benchmarking to Opus 4.8. It's like, what are we doing?

37:38 Benjamin Plummer: I mean, there's some important nuance there around A, there's a lot of accusations around, are they distilling these models? Are they benefiting from them? I think certainly in the past that has been the case. Whether that's ethical or not, I think sort of put that to the side. I think it's certainly helping them catch up. And then the other thing that sort of isn't reported as much is the benchmarks can be a little misleading in that one of the things that if you sort of look at the real-world use of these models is these models are hyper-optimized for the benchmarks. They're trying to appear much better than they are. And then when you look at their sort of real-world usage, it drops off.

38:19 Benjamin Plummer: It's much more spiky, and then it's really good in certain things and then much worse in others. Where because the frontier models are getting so much usage, they're pretty well-rounded in terms of the frontier of their capabilities. And so the headline numbers don't necessarily always tell the story. the full story. But it's certainly true that they're accelerating and they're very close to the frontier and increasingly seem to be closing the gap between, you know, if they were 6 months behind, maybe they're 3 months behind now or something, but it's not far.

38:52 Georgie Healy: I did try and try one of these models for myself. You can pick one of the KIMI models in GitHub. It's got the same dropdown as the, you know, Western Labs. But it's coding and I'm not a coder, so I can't really judge. Ben, how do you recommend to your companies what to do? Yes, don't be too loyalist to any one company, but high level, is there anything you tell them?

39:16 Benjamin Plummer: Yeah, so it's really important to understand how the models perform for your business and your needs. And that means you'll hear this word evals, but basically having a set of tests that you can run the models through constantly to evaluate how they're performing relative to your specific tasks. And the reality is any business is made up of hundreds of different tasks, and the right model for one task might not be the same model for a different task. And so it's not even one uniform answer for a particular business. We have lots of processes we run that might have 3 or 4 different models in the same process. And so those evals are really important in knowing how it's performing, what is the quality per model, what is the cost per model, and you can make informed evidence-based decisions around those.

40:07 Benjamin Plummer: And I think the sort of state-of-the-art company have built really sophisticated infrastructure around these evals that as soon as a new model comes out, they're able to run it through every single use case they have across the organization. And flick a switch and say, we're migrating to this model for these 5 things. We're using it as a backup model for these 3. And it's automated. They don't even need to make some sort of judgment decision around those things. It's sort of based in data and math around which models perform best. Organizations don't need to build something that sophisticated, but certainly having a sense of how do these perform in real-world scenarios for my business is really important.

40:50 Benjamin Plummer: Just because one's better on a benchmark doesn't mean it's the best model for your specific use.

40:55 Georgie Healy: So well said. Even listeners will know if you want to do your color analysis, you can use ChatGPT for that. Do you really need Fable? Like, are you burning through credits when the questions you're asking are not that critical for that? That's actually— it's yet to see a use case for Fable that really made sense. Have you seen any?

41:14 Benjamin Plummer: Well, look, I think Certainly we are in that same process of experimenting with how much better is it, where are those gaps? We're testing it as we should as the type of company that we are. And so we're using it for a lot more strategic planning, architecture, decision-making, and in some cases running that side by side with Opus or other models to test where does it outperform? there's certain areas where it actually underperforms, which overthinks things and it sort of makes things more complicated maybe than it needs to be. And so that sort of fine-tuned understanding of what the character of these models are is really important.

41:55 Georgie Healy: It is like a character, right? They've got personalities. They really do.

42:00 Benjamin Plummer: And look, I don't know if you saw, this whole open source thing has blown up in the US over the last few days to sort of various voices in the Trump administration on both sides saying they should be banned and KIMI should be banned and there should be all sorts of controls. It seems like some lobbying from Frontier Labs around that. A huge consortium of NVIDIA, of Google, Microsoft, the who's who, strongly supporting the open source ecosystem in these models. And so that opens up just a whole can of worms around what does banning these open source models really mean? What geopolitical implications of that? How does a country like Australia participate in that where we don't have our own frontier models?

42:54 Benjamin Plummer: The open source models can be really valuable to us from a strategic perspective if we get cut off from Fable again or Fable-like models. those open source models are really valuable. And so it's going to be interesting to see how this plays out over the next few weeks. It's hard to not criticize everyone's kind of taking a pretty obvious self-serving position around—

43:18 Georgie Healy: Oh, no kidding. You don't want us to not spend money on US models. What if Meta became more powerful in the open source? What do you think would happen then? Because American company—

43:30 Benjamin Plummer: I do think you've sort of got 2 problems wrapped together, which is this sort of closed versus open source and then US versus China. And it just so happens to have played out for a variety of different reasons that China's been the clear leader in open source models and the US has been the clear leader in closed source models. And there's been various attempts in the US, it's sort of half-hearted efforts to have sort of open source models, but nothing that's sort of been really credibly close to the frontier for any sustained period of time. I do think one of the likely outcomes of this sort of current posture is that there'll be either incentives or more pressure for the frontier labs and hyperscalers to start producing frontier-level open source models to give a bit more of an alternative to the Chinese models.

44:25 Benjamin Plummer: And so we'll see. I think that's like a little bit TBD, but that would be my guess.

44:29 Speaker C: Oh, what a prediction.

44:29 Georgie Healy: So you think like Anthropic or OpenAI?

44:32 Benjamin Plummer: Or Google or Nvidia. Like anyway, there's a whole bunch of them that have the talent, the compute, the resources to go do this. And some of them actually the incentives to go do so. And so I wouldn't be surprised if we see more frontier-like models coming from those other players.

44:50 Georgie Healy: When it happens, I'm tagging you in a LinkedIn post. Heard it here first. Okay, when you guys announced your business, of course it was in the AFR, the most prestigious paper in the country. And it was posted on your LinkedIn and I was looking through the comments. I was borderline obsessed with this one. This is a great idea, there are so many boomer slop businesses just sitting there. What's a boomer slop business? I know you didn't write it. And another common rhetoric in the comments was, this is such an ambitious thing, Australia needs to be more ambitious like this. Do you think you guys are particularly ambitious or you just stand out comparatively?

45:31 Benjamin Plummer: It's a good question. Let me answer that. I think we are trying to be quite ambitious in the sort of breadth and scale of what we're setting out to do. There's something like $2 trillion of services in Australia. I don't think most of it has changed in decades. We think we can put a huge dent in basically bringing that into this sort of AI-native world. And there's a really amazing blog post, I think from Sam Altman a few years ago about doing hard things. And his basic view was—

46:04 Georgie Healy: I loved this.

46:05 Benjamin Plummer: It's actually easier to do hard things than it is to do easy things. You can attract better people, you can attract better talent. It's less competitive. It's worth the effort of dragging yourself through it. And I think there's a lot of truth to that, that actually the more ambitious you are, the easier it is to get people excited about the mission and where you're going and what you're doing. And so I think there's a lot of that that's sort of woven into Dragonfly, which is there's no reason we can't be maximally ambitious around the impact we want to have on the Australian and global ecosystem. I think there's a lot of ambitious people here in Australia.

46:43 Benjamin Plummer: I think there's a bit of a density problem that they're kind of scattered all over the place and you kind of have to find them and they're not all concentrated in the places that you would expect in a way that the US kind of does. And the broader sort of economy has been quite rewarding to people who don't take risk. It's kind of been a great ride. Property's been great. There's good, stable paying jobs. There's lots of stable sort of companies. And so I do think that over time folds into the culture and the DNA that there's a lot more risk-taking and sort of contrarian thinking that's prevalent, particularly in the US. I think the US stands sort of head and shoulders above most of the world, all of the world in that case.

47:32 Benjamin Plummer: But there's no reason that needs to be true. I think Australia can do amazing things. We've built some incredible companies. at global scale, and we should build a lot more of them.

47:42 Georgie Healy: Perfect follow-up around data centers. This is such a hot topic. Then if you're on my Instagram algorithm, it's— they're pure evil. Like, they are actually hell on earth, and nothing could be worse than to have even a single data center here. But then on my LinkedIn, it's like, can we be a little bit more ambitious? How are we going to compete in the future? future? How is the economy going to survive? Can, like, we, we don't want to hurt the environment, but let's be realist. Where do you fall?

48:14 Benjamin Plummer: Look, I, I think this is like super clear-cut for me, which is it's as you said, we need to have a vision for what type of country we want to be and how do we want to participate in the next 20 years of the global economy advancing. We mostly spent the last 20 years digging rocks out of the ground and sending them overseas. that's probably not the place in the world that—

48:37 Georgie Healy: That's my undergrad degree, but thank you.

48:39 Benjamin Plummer: It's probably not gonna be how we wanna position ourselves for the next 20 years. And I think it's important to look at like, what are our competitive strengths and play into those and not try and replicate what makes sense for other countries might not make sense for us. And I think we have extraordinary energy capacity through a variety of different mechanisms. We have a huge amount of space. We have a pretty sophisticated construction ecosystem. There's a tonne of reasons where there's a proximity to Asia. There's a whole bunch of different things that mean this is an amazing place to build these data centres. We could build a real competency and expertise in these ways that we're not going to be building frontier models in Australia ever.

49:26 Benjamin Plummer: And so that's not going to be our path to participate in this sort of AI wave. I think the data centres is a very credible way for us to play to our strengths and solidify that. I think the superannuation capital base is another really unique feature of the Australian sort of financial system that can take long-term views and support some of that stuff. And mostly, like, if you actually look into the environmental and the pricing stuff, it's like totally alarmist. It's like ignores a whole bunch of facts around what's likely to actually transpire around renewables, the mix of technology, no government policy around pricing. And the same thing happened in the US.

50:09 Benjamin Plummer: There was a lot of sort of alarmist rhetoric there. I think actually the Trump administration came up with some really sensible policies around how the hyperscalers pay for the electricity they use. It doesn't disrupt the grid. If anything, it's actually feeding power back into the grid, subsidizing it.

50:26 Georgie Healy: That is what they're all saying. We're going to pay. more electricity bills. So that's not happening in the US?

50:31 Benjamin Plummer: No, I'm not sure that that policy has actually flowed through, but all the major tech companies have agreed to it, which is they'll build their own power. So if Microsoft's building a data center in Texas, it will build a power station next to the data center. It's totally off-grid, it's not taking power away from anyone. They're actually selling excess capacity back to the grid and, and producing that more power. And I think It doesn't need to be that exact configuration, but I think there's lots of ways to solve that problem if you're committed to actually doing something versus just sitting on the sidelines and complaining that like it's not perfect.

51:07 Georgie Healy: No risk of people saying we didn't, we didn't really step on the wire here. Yeah, let's go. Uh, I was sent a link to ABC Four Corners and I knew it would end badly about how bad AI is, and data centers was one of the things. And I copied this transcript because it was such alarmist journalism that I was like, I'm just going to copy and put in Claude and be like, verify the facts. One of which was the cost of these data centers, the water usage, the power. And Claude was like, respectfully, this is assuming that you would use— you would have the poor efficiency of a very, very old 30-year-old data center. when you're building them from scratch, you would never build them in that way, and they're extrapolating in a way that's just false.

51:56 Benjamin Plummer: Yeah, people are just cherry-picking. Like, you can go find data to support whatever point of view you want. Um, it's not exactly like the most productive— going back to sort of like the search for truth— it's not exactly the most helpful way of navigating what are in some cases quite complicated topics. But I think the idea of sort of just sitting on the sideline and outsourcing basically everything to the rest of the world seems like a much more terrible path than trying to work our way through some of these problems, many of which actually could subsidize huge investment in renewables and distribution and a whole bunch of other jobs and creation around these things that come with making investments in big sort of audacious ideas.

52:45 Georgie Healy: This is unfair because this is not your background. What is your thought on AI submarines, like defense? And, and it's just a topic that seems to have really blown up lately.

52:56 Benjamin Plummer: Yeah, look, I don't know, like, have any expertise on the sort of deep technology side of things. It does strike me as there's a set of industries where Australia's a little bit Goldilocks in that we're big enough to matter and have the right infrastructure and rule of law and stability to be a mini version of America or Europe or whatever you want to describe, but actually just much simpler in that you can move faster and cut through a lot of the bureaucracy and the institutions that are there to sort of protect the way, whether it's the military apparatus or whatever. And so it's not surprising to me that there'll be a handful of these industries that Australia actually can leapfrog ahead because we have great talent, we have capital that can be put to work against these things, and we have a sort of semi-structural advantage relative to some of these bigger markets which just move much more slowly and then release them to the world.

53:58 Benjamin Plummer: And so without sort of being deeply familiar with the technology, it doesn't surprise me.

54:04 Georgie Healy: I'm deeply familiar that they're square. For some reason, these AI subs are square. Okay, so there's 3 things Australia should play in. It's not frontier models. What would the 3 things be?

54:17 Benjamin Plummer: That's good. So I think the data centers is a clear one. I think there's no reason we can't be one of the leaders in like the application of these technologies, which is how fast, Can you take what's coming out of these labs and push it through the real world into the economy? And it's obviously what we're trying to do with Dragonfly. If you can think about that at a government level, you can think about some of our largest organizations. I think for many of the same reasons, we're not as stuck to hundreds of years of legacy in the way that, you know, sort of Europe and America and stuff are. And so we should be able to move much faster. And then I think the sort of next frontier around sort of physical AI, there's a huge case to be made that we can and should play a huge role in that given everything from sort of agriculture to mining, construction, these sort of heavy industries that we've built quite sophisticated capabilities in.

55:14 Benjamin Plummer: AI is going to be radically transformative to many of these. And Australia has some of the largest, most established organisations across those. And so I think there's a real opportunity to double down in what will be the next frontier of sort of AI development. And so those would be my, my 3.

55:33 Georgie Healy: I love them. Are you ready for rapid fire? 1-minute questions to finish. What keeps you up at night?

55:43 Benjamin Plummer: Look, I think, um, there's like a crappy version of that, which is like not moving fast enough. There's not— the answer is much more nuanced than that, which is it's not just moving, but it's actually getting in the right position so that we're able to capitalize on this really unique moment in time. And I sort of think about it like a wave, which is like, you know, the wave's coming, you can see the wave, you can paddle as fast as you want, but if you paddle in the wrong direction, it's actually going to pick you up and dump you on the beach. And so it's not only the paddling fast, but paddling fast in the right direction. And I feel like that's very much this moment in time that energy does not equal strategy.

56:32 Benjamin Plummer: And there's a lot of moving pieces and fundamental truths or truths that have existed for the last decade or so that I think will no longer be true. And so you do need to go back to sort of first principles and be like, what do we really believe? How do we believe if this will play out. And if that's true, where do we wanna be in the field when this comes? And so that's like a lot to wrestle with before going to sleep, but that's definitely what keeps me up.

56:59 Georgie Healy: What time do you go to bed?

57:01 Benjamin Plummer: Early hours.

57:01 Georgie Healy: And also a very Bridgewater response to you, that is much longer than a minute, Ben.

57:06 Speaker C: Thank you.

57:07 Georgie Healy: Queenslander. Who are the people most at risk of job disruption? What should they do?

57:15 Benjamin Plummer: It's a good question. Um, I don't— you usually see this come out as like lists of jobs that are going to be replaced or not replaced. I actually just don't think it cuts that way. I think what AI is really changing is people with agency to go and create and build and do something different are going to be extraordinary beneficiaries of this, whether you're a doctor, whether you're a mathematician, whether you're any construction worker, you're going to be hugely beneficial for this because there's just this inflection point that allows you to scale your capabilities in ways you never could. And if you are predisposed to just putting your head down and doing your job and not really rocking the boat and not really asking questions, I think you're going to very quickly be commoditised out.

58:04 Benjamin Plummer: And so to me, it's much less about which jobs, they're all going to change pretty drastically. It's really about how people approach those circumstances, which is going to change the outcomes.

58:15 Georgie Healy: Reminds me, have you been to the dentist lately? My news is AI.

58:20 Benjamin Plummer: Yeah.

58:21 Georgie Healy: Have you?

58:22 Benjamin Plummer: I have not. And maybe I need a new dentist.

58:24 Georgie Healy: Oh, this was fantastic because he even showed on the screen, he had his— he asked permission, but he had a Rode microphone. I was like, okay, content girly. And he was like explaining exactly what he was doing as he was going through each individual tooth. I could see it transcribed. I learned a lot as a patient. Fantastic.

58:42 Benjamin Plummer: Exactly. And how the last 20 years before that, that experience had basically been exactly the same. And so that's like a perfect example where we're just hitting this inflection point where a lot of these legacy industries hadn't really changed, are going to fundamentally change over a really small period of time.

59:00 Georgie Healy: Yeah, it's— it feels like the last time I went, nothing, and now it's all AI. Incredible. Is there anything you believed about AI 2 years ago? You know, you've been in AI for over 10 years technically, that now you fundamentally are like, oof, I got that wrong?

59:18 Benjamin Plummer: I think the technology has developed faster than I imagined, and that the pull-through has been much slower than I imagined, that I sort of thought the technology would come up a little bit slower, like the leaps and bounds we're still seeing now. It was kind of crazy last year as we were talking about, have we hit a wall?

59:39 Georgie Healy: Is— The ceiling, yeah.

59:41 Benjamin Plummer: And it's, I think, pretty clear that there's not any wall on the horizon. But when you actually look inside these businesses, nothing's changed. And I think that in hindsight it was kind of obvious, but at the time I certainly would, if you made me bet, I would've bet that there would have been more sort of real-world impact. And so I think that's probably one thing that's, that's definitely evolved and, you know, is part of why we're doing what we're doing.

60:06 Georgie Healy: I was speaking to someone yesterday about you can show someone how amazing AI is, they still won't use it if they don't want to. And that weirds me out. That weirds me out. Yeah. Listen to the show. Last question. What does an Australian services company look like in 5 years if they didn't listen to this podcast? Um, or speak to you?

60:34 Benjamin Plummer: I don't think there's some like tsunami event that is like, it just wipes all these companies out. Like, I just don't think it's gonna sort of transpire in that way. I do think this just slowly but surely gonna fall further and further behind, which is their ability to serve their customers is going to fall further and further behind state of the art. I think their employees are going to be less and less empowered to go be their best people. That means the best people are going to leave. That's going to then mean that the service drops even further. And so it's going to be this slow erosion of competitive advantage that will play out over a period of time.

61:19 Benjamin Plummer: But once that's happened, it's kind of impossible to catch back up again. It's pretty hard to attract great people into a sinking ship. You're sort of losing market share and so you can't invest in the same way you could. And so while I don't think it's too late, there's definitely a window of time where if you don't sort of get on board and start moving, it's going to be really hard to catch up.

61:43 Georgie Healy: Beautifully said. How do people find you?

61:45 Benjamin Plummer: How do they Discover Dragonfly, dragonfly.com.au, and [email protected].

61:52 Georgie Healy: Love your blog. Love what you're doing. Thank you for being on the show.

61:56 Benjamin Plummer: Thanks for having me.

61:59 Georgie Healy: Thank you so much for listening to In the Blink of AI. If you want to go deeper on anything we've spoken about today, I write a weekly substack called Attention Is All I Need. Yes, it's hilarious. It's a pun. And essentially, I go into AI rants, tech news, events I'm going to, and more. It's bite-sized, and I hear it's awesome. Uh, the link is in the show notes below.