▶ “It's really hard if you've only got one way of seeing the world.” — Sarah Kaur Sarah Kaur is not your typical technologist, and that’s the point. As Principal Design Strategist at Portable, she brings an artist’s eye and a community-first mindset to the world of AI and justice tech. In this episode, Sarah joins Georgie Healy to explore what it really means to build AI that works with people, not just on them. They dive into the hidden biases in design, the pitfalls of “black box” systems, and the real cost of leaving communities out of the development process. From building tools in Australia’s family law system to dreaming up a sovereign digital twin that protects your data, Sarah’s approach to ethical innovation is bold, reflective, and deeply grounded in care. Whether you’re designing for scale or simply wondering how AI can be made more equitable, this conversation will give you a new lens on what it means to build technology that respects, adapts, and listens.
Chapters
3:20 From Artist to AI Strategist: Sarah’s Unconventional Path ▶ 7:42 What Human-Centred AI Actually Means ▶ 11:57 Sitting in Ambiguity: The Black Box Camera Experiment ▶ 14:40 Using ChatGPT to Surface Personal Bias ▶ 18:20 What the Public Doesn’t Understand About AI Design ▶ 22:32 Why Co-Design Matters More Than Ever ▶ 30:32 Designing With Communities, Not Just for Them ▶ 35:40 When Explainability Matters More Than Accuracy in AI ▶ 40:00 Inside Portable’s AI Sprint Model ▶ 43:02 Ethical AI vs. Imaginative AI ▶ 47:31 Dream Tech: A Sovereign Digital Twin That Asks for Consent ▶ Resources
👩💻 Sarah Kaur on LinkedIn – https://www.linkedin.com/in/sarahkaur/
🏢 Portable – https://www.portable.com.au/
📘 Amica (Family Law Tool) – https://www.amica.gov.au/
🛠️ AI Sprint Model by Portable – https://www.portable.com.au/articles/ai-design-sprints
Transcript Synced · select any line to jump ▾ 0:02 Georgie Healy: Founders scale faster on Deel. Set up payroll for any country in minutes. Hire anyone anywhere. Get visas handled fast and get back to building. Visit deel.com/dayone. That's d-e-e-l.com/dayone. 0:21 Sarah Kaur: Me right here, like I'm so full of bias. I don't even know what I don't know. And chances are an LLM that's trained on such massive data are might be able to help illuminate some of my own blind spots. 0:35 Georgie Healy: Is there any tech that you personally would draw a line against and, and why? 0:41 Sarah Kaur: I would build a sovereign digital twin. Imagine it like a, your digital wallet that holds all of your digital data. 0:51 Georgie Healy: What blind spots do we have in Australia about AI, do you think? 0:56 Sarah Kaur: What can we do as human-centered designers to articulate that in a clear enough way that machine automation systems will be able to interpret it and act on that in a values-aligned way. 1:12 Georgie Healy: Hello and welcome to In the Blink of AI, where I talk to the brightest AI startups and innovators each week. I'm Georgie Healy, and this week I am speaking to Sarah Kaur. Sarah is the principal of human-centered design and data governance at Portable, a certified B Corp company that are Australia's leading innovation partners for public good. Portable applied tech and design to reforming justice, mental health, and other complex social systems in Australia. And Sarah's a key part of that initiative. Sarah is also the winner of Women in AI for Asia Pacific in Creative Industries. And as you'll soon find out in this episode, as per her LinkedIn headline, human-centered design and human-centered AI is her jam. On the show, we talk about creative pursuits, AI design sprints, and the importance of ethics and bias awareness while still being on the forefront of innovation. We also have our regular series on AI hacks and spicy hot takes. Huge thanks to Sarah for being on the show. You're listening to a Day One FM show. Hey Sarah, thank you so much for joining in the Blink of AI. Can we start with you telling us a little bit about your role at Portable? 2:33 Sarah Kaur: Absolutely. Hi, Georgie. It's lovely to be on In the Blink of AI. At Portable, I am our principal design strategist leading human-centered AI design research. Alongside that, a couple of pillars: data governance and strategy and cybersecurity, because I think all three pillars together make for a really responsible, holistic look at our AI ecosystems as we start to adopt them more broadly. 3:05 Georgie Healy: You're a busy woman, and I do see that you write some really compelling work online. People like myself that are prolific on LinkedIn have probably seen your face before, which is wonderful. I'm so glad that we have you in the Australian ecosystem. Were you, were you always driven to impact initiatives prior to AI initiatives, or the other way around, or did they kind of both coalesce at the same time? How did that happen? 3:32 Sarah Kaur: Yeah, I think it's a little chicken and egg. I've always been more drawn in my work to what is the impact of the work. Like ultimately, if I'm toiling away, what is the effort amounting to? And it started to feel like a few of the problems that we were encountering in Portable around social innovation and impact started to have potential solutions in emerging technologies. And so that's first how we kind of came across machine learning and AI about 8 years ago. But once I started to learn more about it, it felt so nascent in terms of the wide explosion of opportunities, not just for the application into areas where we had complex problems to solve and big datasets, but also in terms of how do we think, feel, and eventually learn to govern this technology that could be so powerful. And so we zoom forward to basically today, and I'm like, I'm working right at the intersection of those two questions. And I love, I love being here at a time when the conversation feels so important and engaging for so many of us. 4:54 Georgie Healy: Yeah, it feels to me—and I'm a millennial, just to give context—it feels like a very exciting time in tech, and the energy is very palpable. But also, I feel this undercurrent of fear and can't get the toothpaste back in the tube. And it's easy for me working in tech to just be like, "Just keep shipping products." But it's really vital to have people like you that kind of think holistically of what this means and why this matters. I'm sure you agree, but I'd love to hear if you do. 5:30 Sarah Kaur: Completely. And I don't think this is, you know, a specifically AI challenge. We've got a history of, you know, adapting and adopting so many different types of technologies, right? AI is just one current really massive tool that we're trying to wrap our heads around. But What I've been trying to encourage people to think about is if you are a designer and you've come from a human-centered design background, what does it mean to try and articulate the things that the humans in the communities or the user bases or the citizens that we're trying to serve? What can we do as human-centered designers to articulate that in a clear enough way that machine automation systems will be able to interpret it and act on that in a values-aligned way. It's a really complicated problem. It is so much more than AI engineering and data science. It's actually a really like holistic lens that we need to bring to it. And I think it's important for me to say, even if you aren't really across the technical dimensions, Please don't let that stop us from having a voice in the conversation, from asking questions, from kind of talking about the potential blind spots that we see as humans, but also as human-centered designers who have that training to be able to say, this is an area where we need to design carefully. These use cases have particular contexts when we try to use AI to automate or augment certain things. How do we make sure we understand that full context and the human impact so that we can actually translate that into good guardrails and AI engineering requirements. 7:27 Georgie Healy: Yeah, we had last week the CTO of EY on the show, and it reminds me of what she said, which is, if you think I'm not talking to you in this conversation when I'm talking about, you know, playing with the tools and upskilling and thinking about how it all works, I am talking to you. I'm talking to, you know, I'm talking to mothers that are at home, I'm talking to teachers, I'm talking to literally everyone across, you know, any kind of path at the moment that you have a right to have an opinion, you have a right to play. So thank you for reinforcing that for the listeners because we do have diverse listeners of the show. One more background question, Sarah, before we dive into the human-centered design and those questions I can't wait to ask you about. You have a Bachelor in Fine Arts and a beautiful charcoal drawing behind you that a friend of yours created. So anyone on YouTube, definitely check that out. I'm someone with zero artistic ability but obsessed with art, and I'm just curious what your kind of mode of choice was, what got you into fine arts, and if you still dabble. 8:40 Sarah Kaur: Oh, I love people who love art. I wanna celebrate you because I feel I feel like the art ecosystem in Australia just deserves so much more celebration and strengthening and resourcing, to be honest. Um, my pathway into it was actually being really involved in, uh, theater and dance. I grew up in Canberra, and it was just a really wonderful way to start to think not with our minds but with our bodies in relationship to others to bounce ideas and create an abstracted way, I guess, to think about concepts. And so that was what attracted me to it. I was like, so many ideas, so, so much abundance. Um, how can I think this through? And so part of my practice was, uh, to do community-based arts, uh, things. And I think that's actually led me to think about co-design in my later career as a bit of a bridge and stepping stone. So where I used to create like a festival, and the festival had a lot of community arts events, I'm like, actually, the skills that I learned in facilitating diverse members of a community to come together, engage with each other, actually are also the same skills that I continue to leverage today as a co-design practitioner. too. 10:17 Georgie Healy: 100%. I'm obsessed with community. Anyone that's on my LinkedIn will see to that. And I think, you know, AI and community is so key right now. Like, how are we going to possibly learn all of this on our own without a community? But I also had a dance background, clearly never as, you know, professional as you took it. So wow, we, we I knew we were very kismet from day one. 10:46 Sarah Kaur: What was your dance background? 10:48 Georgie Healy: So, uh, classically trained. I did ballet to start with. That's what— that's all I mean. I was— I have flat feet and definitely the wrong body type for it, but I really, really loved ballet. I still kind of dabble here and there, but these days you'll find me at Girly Choreo, which is kind of K-poppy jazz hip-hop hybrid. I'm going tonight, actually. Do you still dance yourself? 11:16 Sarah Kaur: I actually don't. It's really sad. And I decided not to do performing arts as my degree, but I ended up working— basically, there was a time several years ago where all dance performances seemed to have projected media. 11:33 Georgie Healy: Yes. 11:34 Sarah Kaur: And so I was around in that time, and a lot of my thing was like, hey, how can I as a video artist and an installation artist provide a canvas for performance that supports audiences to understand the, I guess, the creative output and context? Because dance is often— contemporary dance particularly can be so abstract. So how do I use video media to give a few signals or footholds for audiences to go, "Oh, like, what is happening here? What does it mean when we see this type of gestural dance performance happening at the same time as being juxtaposed with an image of a—" I don't know, it could be anything, of a coming storm, for example. So it was looking at media and performance in that kind of way. 12:30 Georgie Healy: Yeah. 12:32 Sarah Kaur: I will mention one of my favorite projects because I think it's got some kind of lead-in to AI. I used to be really interested in the idea of black box cameras. So basically a pinhole camera, it's the simplest technology. You have a box, you drill a tiny hole in one end, You typically have, um, photo paper capturing the light and it's essentially capturing a direct image much in the way that I— our eyes work to capture a photo. So very simple technology, literally, uh, you can make it out of a cardboard box. But what I started to do was to create that pinhole camera experience in a moving vehicle, in a truck. And so I don't know how safe it is. This was a really long time ago. 13:31 Georgie Healy: You're like, don't try this at home, guys. 13:34 Sarah Kaur: Don't try this at home. Um, but I found out that refrigeration trucks were pretty much lightproof, um, because they were sealed so well to keep their kind of temperatures up internally or down internally. And so I found that I could, uh, get a refrigeration truck, put people inside the truck, and drive them around. So you start in this completely black environment. It's uncomfortable, it's hot and sweaty. But eventually, as our eyes adjust to the low levels of light, this magical thing happens where you start to see like a, a, a light, like a dot of light starting to move around the truck walls. And you start to realize that's actually the sun, that the, the brightest bit of light coming in. And then details emerge. First in black and white, kind of like this. So you might start to see telephone wires and poles kind of passing you by. The effect is very much like if you've sat in a driveway and seen the headlights of a car swing past at night. And then after about half an hour or so, this beautiful thing happens where we get even more fidelity and we start to see a bit of color. And we have this experience from being in the dark, literally, to starting to be able to orient and process signals that we need to get from the inside, but they're very mediated by this one little pinhole. And there's something about like, what does it mean to sit in a black box, sit with the ambiguity and the uncomfortableness, and actually be patient enough to learn and sense and let our senses adjust enough to understand what might be happening in the black box. 15:22 Georgie Healy: No, I completely agree. I think people think that if you work in tech or you're a software engineer, everything's in that technicolor and everything all makes sense. I was listening to— even Google had their like Coachella, which is Google I/O, last week, and I heard behind the scenes that Sergey, you know, the founder, like the co-founder of Google, was like still like banging at the laptop being like, "It's not doing what I want it to do," and not understanding why. And it's just, are you okay in that darkness? And while it's evolving, just having an open mind and being okay in that kind of black and white, which is where I feel I am, and I'm sure a lot of people feel they are, and open to letting it evolve as we just slowly try and get our heads around it all. That is a beautiful and very visually compelling way to think about it. I loved that, Sarah. You can stay on for hours, please. I've got a lot of questions though, so maybe— otherwise I'd definitely pick your brain about more dance stuff, but that's just a me thing. Look, before I dive into more of the AI-specific to your role questions. We have been doing a new segment on the show called Hack of the Week, and I think we're keeping it because people seem to really like it. A new product or use case or an AI tool that you try and love and can recommend to the listeners. I've got one as well, but how about you kick us off, Sarah? 17:05 Sarah Kaur: Yeah, great. I've got a very simple one for anyone who uses an LLM chatbot like ChatGPT or Claude. I actually use it to counter fact-check my own bias and perspectives where possible. So I understand completely that LLMs are full of bias, and it's actually something where there's so much emerging research around, right? Like, how do we develop tests for this? But I'm like, actually, me right here, like, I am so full of bias. I don't even know what I don't know. And chances are an LLM that's trained on such massive data might be able to help illuminate some of my own blind spots. So the way I like to use ChatGPT or Claude sometimes is actually just to say, hey, this is what I'm thinking about, an idea. What am I missing from this? If I wasn't me, relatively privileged, living in Melbourne, kind of quite tech savvy, what are the things about this question or idea might I be missing in my exploration? What perspectives might I need to surface for myself? And what can I do to ask my chatbot partner to be a critical thought partner and prod me a little bit? So if you take anything that you're thinking about and exploring with your bestie LLM, I would just also encourage you to say, hey, ChatGPT, can you tell me what I might be missing or not thinking about that I should maybe think about factoring in before I form an opinion? 18:51 Georgie Healy: Wow. I am not proud to say I have never thought to even try this. And it kind of makes me think it's almost a muscle too, to kind of have the— this term, I'm not a fan of the term, but like the anti-glazing as well. Of like, every question you ask is awesome, and here's your response to that, uh, exactly how you've asked it. Um, I think that's brilliant. Is there anything that's really— that you're able to share that, that you uncovered unconscious bias in your questioning before? 19:28 Sarah Kaur: Actually, I might share an example that's pertinent potentially to listeners that are in organizations. Um, while I was working at Data61 with the diversity and inclusion and AI, uh, research team there, we partnered with Seek, the recruitment, um, company. 19:48 Georgie Healy: And we— The biggest one or something. I don't know the stats. Someone Google that while they're listening, but it's huge. 19:54 Sarah Kaur: It's huge. And what's wonderful is they actually have a team, um, and a role specifically around, uh, responsible AI. So what we were doing was working with their product teams that used AI in their products or were about to. And we actually, the team at Data61 took the idea of a user story that, you know, a product manager in tech might be used to. As a user of this Riverside app, I want to make sure that I'm being heard clearly so that I know that the recording when it goes out is going to be easy to listen to. That's an example of a standard user story. What we encouraged the team to do at SEEK was go through and identify user stories and then say, hey, what happens if we tried to generate this from the perspective of a 50-year-old woman who doesn't speak English as her first language and lives in regional Victoria? Mm-hmm. What kind of things, additional considerations might I actually need to do to my user story to make this more accessible? And so, it's a way that we can actually use that simple kind of concept, what am I not seeing, and apply it to a really strong engineering process that already exists, just to tweak our thinking about like, hey, for every AI use case, what might someone who does not look like me, that I do not know well, what might they think? It's a no replacement for human research, but it can be a really helpful thinking tool. 21:34 Georgie Healy: Wow, thank you so much for that example too, especially, you know, we are trying to hire in this country, I think it's 20,000 new AI-specific jobs. If you're limiting, you know, ages, genders, certain groups in even your description, you're gonna really have a very skewed workforce, I'm sure. 22:00 Sarah Kaur: We are already. It's really interesting. And because in that world of recruitment, it's really hard to actually get or use data safely that is about diversity attributes, gender, et cetera. So it's really hard both to understand how your AI may or may not be discriminating because a lot of the time employers and recruiters aren't allowed to collect that data. So, in some ways, this is one way to use AI to supplement the kind of regular data and reporting that we can get as well. But yes, all power to let's get a really diverse workforce in to support our AI productivity and growth for sure. 22:45 Georgie Healy: Oh, my favorite customer engineers at Google are like, through no particular reason, but all amazing women that I just look to and I'm like, wow, I wish when I was doing engineering as an undergrad I saw you guys more. And I'm really excited to see them kind of have a bigger platform and hopefully see more of that because, you know, it might keep, you know, inspiring future generations as well. So yeah, that's a really amazing hack and definitely not one that I had even thought of. So thank you so much. 23:20 Sarah Kaur: Helpful. 23:21 Georgie Healy: Mine is a little bit more dry, I'm gonna be honest, but hopefully helpful for productivity nerds out there. I admittedly was not a big Notion user. All my friends and colleagues seem to be massive on Notion. It's a very fast-growing product in terms of productivity and managing all the millions of jobs that we've got going on. But I kind of found the platform a little bit overwhelming, and if I'm honest, pen and paper seemed to be just a quicker, easier, dirtier way to just kind of figure out what I've got to do, as opposed to typing out everything I had to do and adding checkboxes and all of that. I'm like, this is taking me more time. But the Notion AI feature that I've just been exploring recently— I'm about to interview head of Notion for APAC and some other areas forced me to kind of play with the product a little bit more. And I will say I'm really impressed because it's got, uh, it leverages, um, LLMs in the background. So you've got a search bar, you can ask it questions and it immediately embeds it straight into the Notion workbooks. You can do deep research, you can build something on it as well, which I have not done, if you can't tell. Uh, but I do, I do genuinely find it an amazing platform in terms of you don't have to go off platform, search for something in the LLM, copy paste it into the Notion workbook. It's kind of like a one-stop shop. I don't know if you've played with this, Sarah. I'm not paid to promote it. 24:54 Sarah Kaur: No, I mean, we use Notion at Portable along with a few different other, uh, you know, knowledge databases. And I think the Antle is pretty wonderful because I guess it's almost like a prebuilt RAG solution. Uh, so If you've got enough of your workspace data in there, it can be really, really powerful. And so I'm excited. I also like the idea that it's one easy way in as a product feature for people to start to understand what AI can do in a tool that they're already familiar with. So I think that's kind of, yeah, a great on-ramp too. 25:38 Georgie Healy: And I have to dive a little deeper on that. So when you say a RAG tool, like you've already got your company's existing literature in there and you can query from that as opposed to searching the whole web. Is, is that what we're talking about? 25:53 Sarah Kaur: Sorry. 25:53 Georgie Healy: Yeah. 25:53 Sarah Kaur: Absolutely. Retrieval augmented generation is just that, that kind of idea that the LLM sits on top of your data that may or may not be also in other places or on the interweb. But it's a way of actually saying, when I ask the LLM, chatbot a question, it's specifically querying my data. And so it's got fantastic applications, and I think that's just one really easy way to start experimenting with what that can do. 26:26 Georgie Healy: Amazing. Thank you so much. And that brings us beautifully to, you know, kind of your role at Portable, a B Corporation. Definitely this podcast isn't your first rodeo because, you know, I've seen that you've got an interview spotlight series that you were on recently. And that's interviewing your team members, if I'm, if I'm correct. Why is it important to get the word out there about what you guys are doing at Portable and your team, do you think? 26:54 Sarah Kaur: Yeah, so Portable is a technology design and research, um, I don't know what to say, collective of really talented and passionate individuals. And we see ourselves as partners in public innovation, so we tend to work a lot in areas where we see we can carve out or support others to make better impact. So that might be in mental health and wellbeing, it might be in access to justice, it might be in aging and even death potentially. Yeah, yeah. 27:30 Georgie Healy: And so— Big part of our lives is, yeah. 27:33 Sarah Kaur: I have a terrible joke. Everyone's an end user, so we have to, I know it's so bad, It's okay. 27:42 Georgie Healy: Um, some— one of us had to make the first dad joke, and I'm glad, I'm glad it wasn't me this time. But, uh, buckle in, it's all yours now. Safe space. 27:54 Sarah Kaur: So essentially, uh, what we, what we found that we've been doing over the last few years is we had been, um, yeah, I think like I said at the top of the interview, we've been coming across across areas and complex problems and kind of going, hey, what's just on the tip that we could imagine technology doing that might solve or help to solve or illuminate or articulate this problem better? And so we started to experiment with ML and AI quite early. And I've realized that in the last, I don't know, year especially, we're just starting to hear so It's very hard to keep up with everything. And I wanted to share some of what we'd been doing and the nuanced ways that we were working with AI with other people in our community. So our clients, our community of like fellow designers and developers and technologists and researchers. And so the Spotlight series tries to actually go a little bit deeper with people who are leading the work to ask them, hey, behind the scenes on this project, what did you learn? Like, where were the actual snags where we might have had to take a different approach? Where did we get blocked? Because too often I noticed that we don't hear case studies in full. Like, we have the shiny story, but we don't actually often get the lessons learned in a candid way. And so the Spotlight series tries to do that and isn't only focused on AI. So we've also— I think the first two episodes, one is on AI and legal design and technology, and the second is all about co-designing with children and young people. 29:46 Georgie Healy: Oh, wow. 29:48 Sarah Kaur: Yeah, hopefully it's a little bank or resource that people can kind of use to get inspired, find out more about the ways they can use design and technology tools, and also, you know, find out a little bit more about Portable if the kind of impact that they want to make might be a good fit for the approach that we take. 30:07 Georgie Healy: And I saw it on your specific LinkedIn page, but I'm guessing the entire series is on the website or on YouTube? Not quite yet. Yet, but it will be. Start with your LinkedIn page. 30:22 Sarah Kaur: Yeah, it will be on our LinkedIn page and our website. And so once that's ready, it's been really fun actually to kind of plan it out and and kind of record it a bit on the fly. 30:33 Georgie Healy: So it looks so cool. It looks so cool. Like, even seeing the behind-the-scenes footage of like you being recorded looks awesome. 30:42 Sarah Kaur: Oh, thank you. So hopefully we kind of cut that soon and, and have that ready. 30:47 Georgie Healy: So exciting, especially with your background in like design and sets and, and all of that. That's really cool. It's full circle. Look, um, speaking of not just talking about the positive, uh, case studies that we see on every website and, and, you know, it kind of gives a very skewed vision. Has, has it ever happened where you've like worked with a client and your idea of human-centered design in this space, um, isn't like— people don't see eye to eye on it, or you have to really re-educate your clients or customers on what that looks like? Where does it get a bit sticky? 31:26 Sarah Kaur: Yeah, it's a great question. I think most of our clients have been in the public sector, sometimes not-for-profits, and also private. I've seen particularly the public sector in the last 5 years go from saying HCD is an innovation approach, we should be using it, but we don't know really what it does, how well it does it, how much budget it needs, the kinds of skills we need to be able to support to do it well. And I've seen that kind of come so far where now lots of, you know, requests for tenders or proposals that we are seeing actually have an embedded idea of human-centered design methodology being asked for in the specification. We have entire kind of communities of practice around human-centered design springing up, really actively talking with each other. And so I think some of the the challenges that we used to experience, like what's the ROI on human-centered design, or how do you really know that it works? I think we've actually transcended a lot of that, and it's become very accepted as, as a logical step. If you're designing things for humans, bring humans in really early to help to share, uh, their understanding of what you're trying to build, shape it with it feels so intuitive. That's why it's also beautiful. And so, I think the challenge now is how do we integrate human-centered design methodology and research methodology into AI tooling. And so, that's kind of like the new little path that we're trying to go on. 33:17 Georgie Healy: Yes. Okay. So, on this, you know, I run an AI accelerator and our most popular mentors are the product mentors and these human design, human-centered design product developers. And what I'm noticing from, from where I'm standing at least, is it blows these very technically brilliant AI engineers' minds of, well, why'd you put that there on the website when I actually want to click over here? And then they're like, oh, it just made sense when we were designing it, or one of the iterations that just got updated a few times. That's UI/UX. But what else, Sarah? What am I not seeing in my narrow view of human-centered design when it comes to AI product? 34:06 Sarah Kaur: I doubt you have a narrow view, but just to build on an example of UX/UI, I think I was working— it's a while ago, but I was working with— there was a team at CSIRO developing an AI chatbot tool. I think the idea was, how could we build something that allows people to understand climate change, because we've got this wide wealth of scientific literature. How does someone like you and me ask and find out more evidence-based questions about that? So I had a little look at a very early prototype, and they'd used one of the LLMs—I can't remember which one—but it had different modes or personalities. Mm-hmm. Summarization mode or creative mode, but I can't remember what it is. Do you remember? 35:00 Georgie Healy: It was vaguely this— yes, vaguely, but there was something like that. 35:07 Sarah Kaur: So in the interface, uh, people could ask the question, they get the answer back, and they could select from what kind of flavor of creativity or temperature the LLM was. 35:18 Georgie Healy: It's like, do you want to go full LSD mode, or are you like not even had a coffee yet mode. 35:26 Sarah Kaur: I was like, okay, wait, this is about understanding scientific literature, right? It's about evidence-based. So does it make sense to give people the option of like the creative interpretation? 35:39 Georgie Healy: I see what you're saying. 35:41 Sarah Kaur: So it's kind of just like those tiny, tiny little things where it's like, it could be, but if you didn't do this on purpose, maybe it's worth doing a bit of thinking or testing about that. Is that the right kind of feature or tool to actually enable users for this specific use case? So that's one example that's, I guess, building in the UX/UI kind of world, or just that idea of actually, if you look at the generic toolset, how might we curate that a little bit better for the purpose of the tool we're creating? But I think other than that, human-centered AI Actually, if we think about behind the scenes, gets really interesting. I think this is also a place where people can be a little bit scared to engage or think aloud with people who are more technical. So for example, I think that there's a lot of questions when it comes to designing an algorithm To choosing or selecting a model, to designing even the metrics that we want to measure about, let's say, model output, that have very clear data science best practice metrics. And very clear, like, hey, we've got lots of data science tools that will give you little signals, but they're hard to interpret. It. And we don't actually think that the data science best practice is always going to be the human values-aligned, uh, thing that we need to prioritize, right? So for example, um, one of the tools that we built at Portable really early on for the family court, it's called Amica. It's live. And the key kind of, uh, value proposition of that was instead of spending thousands trying to separate, going through lawyers, if you are in an amicable relationship with the person you're separating from, can you actually resolve your dispute? And can we use machine learning slash AI to suggest a fair split that a family law court would actually sign off on? So the, the use case was great, which is incredible. Yeah, I think it's like under $500 that you can do which is a great access to justice kind of impact outcome from the technology. But I think one of the things we learned was, um, we had a few learnings along the way around not being able to use historical court, uh, data because that old way of training machine, uh, learning meant that we had to go back in time. The, the court documents that we had were skewed towards towards, you know, the, the more complicated cases with high-income or high-asset, uh, splits. But also, you know, we have to go all the way back to like the '70s where the value that we put on the asset of, say, human female labor at the home, that wasn't really, uh, valued in the same way. 38:57 Georgie Healy: So yeah, to put it lightly. Yes. 39:01 Sarah Kaur: So there's all of that understanding about like, where does the data come from? Who does the data represent? And just because because it's the best practice. Does that make it necessarily appropriate? Probably not. So we created synthetic datasets. That was a wonderful thing to kind of evolve and try like 7 years ago. But what we ended up with was, hey, we've actually got really good model performance on the neural network models that seem to be better at actually predicting a fair split. However, what was important in this context was we were working with family lawyers. We were working with a law court where actually transparency and the idea that like people are gonna have to understand this and the family law court will have to understand how a decision was arrived at. That meant that it really, you know, I was like, if the neural networks are giving us better predictions. But we can't actually explain it, then it fails the use case of being appropriate for this kind of legal design context that we have. And so how do we work with a transparent or more explainable model and be able to compensate for the things that it might be getting wrong along the sides with outliers? How do we involve humans lawyers and review in the loop of that process to compensate. So I guess the moral of that story— 40:35 Georgie Healy: Yeah. 40:36 Sarah Kaur: —data science and algorithms are also throwing up questions. And those aren't just data science questions, they are also design questions. And so that's where having that human-centered lens working directly with your technical kind of teams can be really, really important. 40:54 Georgie Healy: Wow. That was incredible. It also kind of reminds me, you know, yes, the data matters and the design matters, but also if there's bias in the model, like you spoke about earlier. But anecdotally, at least, I have noticed that myself and my friends are using LLMs as a way to kind of unpack who really, like, blew up that chat and whose fault is it really? And, you know, kind of coming to a consensus. Consensus on like having an unbiased third party— I use that inverted commas— to discuss, um, who, who in the conversation or in the relationship was being unreasonable, or how you can both get to a middle point. And you've kind of taken it to the next level of a real genuine use case of like something that needs to be resolved in a positive way. So I find that really interesting and fascinating. You clearly were very ahead of the curve though, clearly ahead of my group chat. 41:53 Sarah Kaur: Oh, I think that's fascinating, having like the neutral third party observer to be able to kind of query. 42:03 Georgie Healy: Yeah. I could speak to you for another 8 million hours. We've got our rapid-fire question, but before we get to that, I'm dying to ask you, at Portable, you do these AI sprints, and I just want to hear a little bit about who's invited, how you structure them, because I want to do one. That sounds fun. Am I a nerd? I don't know, maybe. 42:24 Sarah Kaur: Love it. Well, to be completely honest, we are just starting just at the outset of starting to run them. We used to run them a long time ago, like 5 years ago, and then we kind of took a little bit of a break. It's roughly modeled on a Google Venture Sprint. So that good old lovely format. 42:44 Georgie Healy: Heard of them. Yeah. 42:45 Sarah Kaur: Yeah. It's so like evergreen, I think, in terms of a rapid prototype and innovation, proof of concept kind of outcome. But essentially we love to work with organizations and perhaps they are starting, they've got like an inkling of like, could AI be good for this? And what we want to do is actually say, come work and be embedded with our team. So our team would be someone like me, like a human-centered design researcher with AI background. It would be an AI engineer and developer, and it would be ideally someone from their team who is either an AI engineer, a product manager, an end user, the product kind of owner or stakeholder. And we're going through the process of saying, what is it that you think AI could do? Let's investigate a few ways that AI could do that. Let's eliminate some early where it's either not feasible at this kind of time, we don't know enough about the use case to feel confident that we can design it right or safely. But let's pick one idea and actually get as far as we can with your team and our team. So build it in a couple of days, test it, and kind of say, this is where we got up to, this is where we think it could go. These are a few of the kind of business questions that you might need to answer to be able to take that next step and think about scaling further development. So it's just a way within, I guess, 5 days to wrap our team around your team, come up with as much as we can as quickly as we can to support AI adoption. 44:29 Georgie Healy: Wow. Okay. And when are you starting these back and what kinds of companies are invited to apply to be part of it? 44:36 Sarah Kaur: Yeah, absolutely. So I think we've done a couple in the legal kind of design and AI context so far, and perhaps we've done a couple in the sport context as well. But essentially any, any company is welcome to come and chat to us about it. Um, I'll give you— can I give you a link to that? 44:56 Georgie Healy: Right up more, we'll have it in the show notes. 44:59 Sarah Kaur: That'd be awesome. Yeah, essentially anyone, especially if you feel like you're on that edge of like FOMO where you're like, I think we could, we're just not sure what that next step is and how to, how to take that confidently and talk about it and have safe space to experiment, I suppose. 45:17 Georgie Healy: This is me about agents 2 weeks ago. I'm like, I think I want to figure it out, but I'm still overwhelmed and had to sit next to someone who was a lot more advanced than me. And that really took down the barriers and made it fun. So I love this. 45:31 Sarah Kaur: You learn alongside each other. 45:33 Georgie Healy: It's so rewarding. So rewarding. And I think, shout out to Caitlin at Relevance AI who did this with me. It, you know, the kinds of questions I ask her kind of tell her a little bit about like where, where I fall over and where customers might fall over in the process. So really cool. Okay, we are at the end of the, uh, interview. It's not an interview, it's an episode. Um, I'm like, you're not, you're not applying for a job. Um, but it's the spicy rapid-fire questions. Um, and you know, they're spicy. So you've been such a really good sport so far, and I hope that we can stay friends after this section. All right, number 1, does AI ever get in the way of supporting your mission at Portable to reform justice, mental health, and social systems? So team player or sometimes anti those values? 46:29 Sarah Kaur: I think yet to see. One speculative spicy take is I don't think we're all aware of the way AI is being used to think or make decisions around public policies, funding, program resourcing. We don't know what data or what AI systems are being potentially used. So I can't actually see that. So I don't know how it's like friend or foe. I have so much belief in the potential though. I just would love to know more about how people are using it so that we can make it more friendly. 47:06 Georgie Healy: What blind spots do we have in Australia about AI, do you think? 47:12 Sarah Kaur: I think we're talking a lot about, um, what does it mean to not have a sovereign, um, large language model that is trained and made and maintained here. Um, I think there's a lot of us who believe we don't have the kind of capability, um, or power here, and there's a lot of us that believe, but there's every need for us to be able to own and develop and maintain our own critical infrastructure, um, in that sense. So yeah. 47:47 Georgie Healy: I would love that. Um, I didn't even know I wanted that until you mentioned it. Is there any tech that you personally would draw a line against and, and why? 47:59 Sarah Kaur: Oh my God, this is a good question. I actually don't know that I have a rapid take for you. 48:06 Georgie Healy: Yeah, that's okay. That's not a very easy, rapid answer. 48:10 Sarah Kaur: Not as well, but I kind of feel like the mindset that I have is to always try and be curious and be like, who could benefit? Who's in power that might benefit? How can we use it as an enabler? So I always try to see the possibilities instead of like dead no-go zone, but I need to get much more educated myself about that and that question. 48:33 Georgie Healy: Would you use Deepseek? 48:35 Sarah Kaur: I would for several reasons. I think it's, you are able to kind of put it into your own closed environment as an open source model, play with it, test it, see what it does. And I know there's a lot of controversy around it, but I also in general am quite heartened to see that there might be diverse makers of AI models. So without getting into the politics of it, I actually think having more potentially is a better way for us to start to understand this technology and the possibilities to be able to say, hey, what does it do well? What doesn't it do well? What do we like? What don't we like? It's really hard if you've only only got one way of seeing the world. Even having something— 49:33 Georgie Healy: And they have a complete monopoly on, on it. 49:35 Sarah Kaur: Yeah. So having something to bounce against is actually really helpful. 49:39 Georgie Healy: Wow. Thank you for that. And last question, what's a dream project you would build, you Sarah, with unlimited funding, unlimited power, zero red tape, no one can stop you? 49:54 Sarah Kaur: What would you I would build a sovereign digital twin that individuals can use and imagine it like a, uh, your digital wallet that holds all of your digital data. So it sits with you. Um, you are the owner, you understand it, but there's also an interface, kind of like a little digital bodyguard that's floating around the interwebs. And instead of the model we have now, which is you sign up for everything, you give your data away, it's centralized. You hold onto your data and the interface says, hey, before I give you my data, could you like do a little test for me? What are you going to use it for? Okay. Yeah, I think that's fairly accurate as a representation of me. I consent because it's my data. You've told me what I can expect from giving it to you and you are allowed to use it for that purpose. For this kind of limited time. 50:52 Georgie Healy: Thank you. 50:53 Sarah Kaur: That's what I do. 50:54 Georgie Healy: You're just gonna have to steal Jonny Ive off OpenAI now, and I think you two would make a great partnership, right? Did I say that right? I was like, is it John or Jonny? 51:04 Sarah Kaur: I don't know. 51:06 Georgie Healy: Jonny Ive. 51:08 Sarah Kaur: Jonny Ive. 51:09 Georgie Healy: Okay, yes, Jonny Ive, but it's got one N. That's what, that's what confused me. So he helped build some of Apple's best products. But yeah, with that hardware, I think you, you with your brain and his hardware experience, it's a match made in heaven. 51:24 Sarah Kaur: Endless resourcing. 51:26 Georgie Healy: Amazing. Yeah, right? Oh my gosh, Sarah, you have been the most incredible guest. I'm so grateful for you coming on the show. Before I let you leave, is there anything you'd like to shout out to the listeners? 51:39 Sarah Kaur: Um, just to keep being curious, keep playing. Um, I think like we heard at the top of the episode, everyone, um, deserves to develop their own opinions and voice them. And the more people we have coming forward with an opinion, the more we get to have robust discussions about how we use AI, how we want to design it, how we want to work with it in the future, because it will absolutely touch all of our lives and probably sooner than we think. 52:08 Georgie Healy: Beautifully said. Come visit me in Sydney. Coffee's on me for sure. Thank you so much for being on Love it. 52:17 Sarah Kaur: Thank you, Georgie. 52:21 Georgie Healy: Thank you for listening to In the Blink of AI. You can check out the show notes for anything discussed in this week's episode, and we will be back next week. This podcast was produced by Day One with music by Dan Hansen and visual artwork by Sophie Tyrell. If you loved the episode, please tell your mates. And I love AI. AI News. Please share your thoughts and suggestions to [email protected] . 
In The Blink Of AI with Georgie Healy
Presented by Deel & Stripe 
▶ 
▶ 
▶ 
▶ 
▶ 
▶ 
▶
▶
▶