Harry Wetherald: [00:00:00] You come across a lot of vulnerabilities. Ah, vulnerabilities is just killing us 40% year in year growth in number of vulnerabilities. Don't we already have CSPs? Yeah, CNAPPs. 90% of the findings that they give you are false positives, right? You end up with a resorted list of false positives, basically.

It's just like a logical error in what the scanner is trying to tell you. What's you red teaming them in a way. Yeah, exactly. Imagine that team of two people, three people you've built, what if it was 5,000 people or 10,000 people? If

Ashish Rajan: AI is so good, why would you just not build your own cybersecurity solution?

These just AI being the hottest topic. Everyone wants to build an AI agent MCP server, but no one's talking about the reality of what does it take to build and maintain an AI agent that is going to look after the security of your environment. Like I had the pleasure of talking to Harry Weatherall from Maze.

He spoke about what does it actually take to build and maintain an AI agent in your organization? Harry has been working in the AI ML space for a lot longer before Chad GPD became popular. So he had some experience to talk about what does it [00:01:00] take to build an AI agent, and what challenges you would face if you're trying to build and maintain an AI agent yourself.

We also spoke about some of the different ways you should be approaching doing security in an AI world. Which is really fascinating for me, coming from a bias 14 plus years of experience cybersecurity, the way I was asking question to the AI agent versus. How someone who works in this space of AI would ask a question.

So overall, it was a very fascinating conversation. If you're looking at AI and what were the possibility of an AI agent could be in solving challenges and whether you should build one in cloud or maybe pick another cybersecurity vertical, this is definitely a great conversation. If you know someone who's trying to have that tussle between should I buy or should I just build an AI agent to solve all my cybersecurity problem?

This is definitely a great episode for them. As always, if you are enjoying episodes do share this with them as well. And as always, if you are listening or watching a cloud security podcast episode for the second or third time, I would really appreciate if you can take a second to drop us a follow or [00:02:00] subscribe if you are watching this on YouTube, LinkedIn.

But if you're listening to this on Apple or Spotify, give us a follow. Subscribe there as well. I really appreciate you taking that second to support the work that we do here. Enjoy this conversation with Harry and I'll talk to you soon. Hello and welcome to another episode of Cloud Security Podcast for Hang with me.

Thank you for coming to the show, man. Thanks for having me. Maybe to start off with, if you could give a brief intro about yourself and where you've been, what your professional background is.

Harry Wetherald: Yeah I'm now co-founder and CEO of a company called Maze, which pretty much no one will have heard of because for the last 12 months we've been totally building in kind of stealth mode or just quietly.

We, I'll come onto kinda what we're up to at Maze in a minute. But on my side before that, I kinda worked in security pretty much my whole career. Always worked around machine learning style products has been main theme of my career. And so before doing this, I led the product management team at a company called Tessian, which was doing training machine learning models to spot security incidents over email. So I led the product management team there from when they were a tiny seed stage company right through to right through to 2023 when they got acquired by Proofpoint. And so since then been started working on this new company and yeah, and here we are

Ashish Rajan: and and to be fair.

This is machine learning before GenAI exactly. Yeah.

Harry Wetherald: [00:03:00] Before the whole world decided they were gonna work on it.

Ashish Rajan: Or everyone has a GenAI. I was like, oh, I have GenAI startup as well. Yeah. Maybe to double click on the whole we, before we started recording, we were talking about the challenge people have is vulnerability these days is there's a I guess some people find quite overwhelming when you start looking at it at any given problem. Yeah. At any given security solution, you come across a lot of vulnerabilities. And I think you had an interesting take on, we love to hear in all the research and investigation you were doing in this particular space. What did you find?

Harry Wetherald: Yeah, so the story kind of in three parts for us. Like we, so me and my co-founders, I let. Product management at Tession. The others led product and engineering teams at places like Amazon and Elastic. And so we'd always been involved in leading big engineering organizations in one way or another.

And we always saw this kind of never ending debate happening between security and engineering about, call it like security debt or vulnerabilities, or whatever you wanna call it. And I always felt like both sides were wrong. You know what I mean? So like security, were always pushing obviously harder to fix all the stuff and are the backlogs getting too big and we need to down tools and fix this [00:04:00] stuff.

But then also engineering, were right to question how much volume they were getting pushed towards 'em. 'cause their job is really to ship useful features. Not to just constantly fixing vulnerabilities, but also weren't always the most proactive in actually getting the fixes done. And so that kind of was like rigging around all our heads as we were leaving our old jobs and thinking about starting a new company.

And it just felt like a problem that wasn't quite right. But before we actually started the company, we spent months like, honestly like 3, 4, 5 months just doing cold, like outreach to CISOs, security leaders, these kind of people. And just getting on the phone, just ask, just being like, Hey, what's going on in your world?

Yeah. What are you struggling with? And it became almost a joke at some point. How often Like the answer on the other side of the line, they would just kinda sit there, take a big inhale of breath and be like, Ugh, vulnerability's just killing us. And so that happened time and time again.

And then the real last moment for us where we really thought, okay, this is something that we need to like desperately go address, was when looked at actually the underlying data around vulnerabilities, which has now become quite like well known in the last six months in particular.

But if you look at it, it's 40% year in year growth in number of vulnerabilities. So much so that, we've heard [00:05:00] about NVIDIA and others kinda like basically falling over from the volume they've tried to deal with. But if you look at some of the other stats around it as well, what you've seen is like there's a massive decline in the time it takes an attacker to exploit your vulnerability.

So you're now seeing new vulnerabilities get exploited in 3, 4, 5 days on average rather than the 30 plus that it used to take them a couple of years ago.

Ashish Rajan: Yeah.

Harry Wetherald: So you kinda got this like exponential increase in number of vulnerabilities. Yeah, exponential decrease in the time you have available to fix it.

But then all the security teams have the same number of people available to fix them, which is the SREs engineers, IT staff, et cetera. So that to us felt like a problem that was getting worse and worse at the moment. And if you also look through the lens of what's happening on the attack side of things.

It's well known that now we're seeing automated pentesting tools, here at come, come to the fore. And so attackers, of course, are using those, building their own, making them good, moving much faster than the defense side is probably. And so that's only gonna exacerbate the problem where suddenly you're gonna see what used to be quite hard attempts at exploiting vulnerabilities become a lot easier, and therefore become a lot more widespread.

So we saw it as this problem that was like we'd seen ourselves, we [00:06:00] had heard so many times about how it was top of mind for security leaders and if you, and then looking at where it was gonna go, it only looks like it's gonna get significantly harder in the next couple of years.

Yeah. So kinda took us to the point where we put our heads together and thought, okay, what can we do combining our backgrounds with machine learning and working with big data systems and then all the changes that have happened recently around generative AI and how do we put all that together into something that can actually solve this for people?

Ashish Rajan: Vulnerabilities could be a different types to your point, cvs, it could be cloud vulnerabilities. Yeah. And I know cloud vulnerabilities is a quote unquote misconfiguration. Yeah. But there's an obstacle. So what vulnerability specifically do you zero in on?

Harry Wetherald: Yeah, we see, we see ultimately the more coming back to the same core problem, which is as a security team, one of my fundamental needs is I need to figure out where I'm likely to get breach from tomorrow.

And stop it happening, right? That's right. So everything just falls back into that problem. And that's an easy way to remove some of the complexity around it. Like the vendors in particular like to add as much complexity to this kind of problem as they can. So what we try to do is just focus on that.

That is our remit. We just wanna be an amazing way to figure out what risks you're facing, how bad they really [00:07:00] are, and how you fix them. But obviously as a startup, like we need to do that in kind of phases. So where we've started, where we saw the most pain amongst the people we spoke to was basically.

Cloud based vulnerabilities and the CVEs that they basically have in the cloud. Yeah. Some of the customers we've deployed with so far, they might have. I don't know, 5,000, 6,000 employees, but 2 million vulnerabilities. And so what, what are they gonna do with that?

And that's where we see where we're helping them at the moment. And then obviously kinda over time expanding it out into more and more different types of risks.

Ashish Rajan: And when you say a cloud vulnerability, what would be an example of what we used to be before and what was making it overwhelming?

Because, I asked this because. A lot of people may hear this and go don't we already have CSPMs? Yeah. And CNAPPs in all the world.

Harry Wetherald: Yeah. What why not use one of those? We do. Most of the people we work with use a CNAPP, but mostly the conversation goes something like, Hey, what, what do you use for dealing with cloud vulnerabilities?

And they say, I use, CNAPP X and I say, how's it going? And they say, ah, yeah, it's tough. And it's not to say that those tools have necessarily done anything wrong. But they haven't they basically managed to scan the environment, find the vulnerabilities, do some [00:08:00] lightweight prioritization on them, but they still leave you with an enormous amount of work to do.

Yeah. In terms of when, based on our analysis, something like 90% of the findings that they give you are false positives, right? So they actually cannot be exploited in any way, shape, or form in your environment. So that's a whole, bracket of noise that you need to deal with in the first place.

And then even with the remainder, you need to then figure out which ones, which are the very small number that really are gonna hurt you, and then how you actually fix them and need to push those fixes out to the team and get them to actually do it. So there's a huge amount of work that comes with owning a CN app today.

And that's basically the problems that customers tend to come to us with.

Ashish Rajan: So you, I think you're afraid of reachability and the attack bot.

Harry Wetherald: Yes. Yeah, I mean there's, so the reach reachability as as some of the vendors have defined it so far is quite a narrow definition of kind of whether the code is specifically reachable for for a vulnerability.

But when you look at the broader kind of basket of. CVEs that a company can be faced with. There are tons of other reasons why you can find something to be a false positive, not just purely through what people have seen as a code reachability and what they've done with that. If you look at it more like a human would and triage it more that way round. [00:09:00] They're not always just gonna be purely looking at the code. They're gonna be looking at all sorts of other factors to try and understand whether this thing is a false positive or almost just so low risk that we should never even care about it. And it can be really advanced things like some of the stuff are kinda AI agents have found are like amazingly advanced blows away like week on week. Some of it is like maddeningly dumb, yeah. My, my favorite one that we always come back to was, it was a very large company. We're working with a Fortune 50 organization and they had a large number of CVEs of all one type.

And our AI agents went through and analyzed them all and we were all there like ready to see this long, great, big investigation with all this smart stuff in it. And instead what it said was it's a critical vulnerability that was like screaming at them and it said we've reviewed this. All the research about the vulnerability says it only applies to Linux.

All these win, all these machines are windows. Oh. So you get a mixture like that. Like that is something that is by definition, not exploitable, but it's not reachability or something like that. It's just like a logical error in what the scanner has tried to tell you. So there's much, much more to it than just purely the narrow definition of reachability that people have thought in the [00:10:00] past.

Ashish Rajan: And so I Would you say then the. I love this question because the way you explained it, it made sense in terms of why there are so many false positives, but it also makes some sense of why the joke of the wall of red that people talk about. Exactly.

Harry Wetherald: Yeah. Yeah. The, I've heard this so many, like the Christmas tree lighting up Yeah. So you get this really quick deployment with the CNAPPse. Yeah. And it feels very satisfying and suddenly you have all this visibility of all this stuff. But yeah, the common thing that comes back is honey, like my Christmas tree lit up and now I have to deal with all these, problems I didn't know about before.

Ashish Rajan: So how does it work in an, with an AI agent versus like the, at least the way CNAPPs have worked, traditionally, they'll have an IAM role and they're making API calls. It is funny because to your point now, everyone knows a lot about AI these days. They just throw the word.

Yeah. So just to double click on that a bit more, how does an AI agent interact? And what's happening at that level?

Harry Wetherald: Yeah. So one of the most frustrating things for us right now is how over hyped the word AI agent has become in the last few months. It was like in January, someone in the security marketing sphere [00:11:00] decided this is our word now.

Ashish Rajan: Yep.

Harry Wetherald: Everyone is gonna use it. And what it means is that basically still most people don't understand what it means, both the vendors and the security folks. 'Cause just so new. And it's also has a fairly vague meaning in the first place. Best way I'd describe it would be if you think of like using an LLM on its own, like a ChatGPT or something as running a script or running a single piece of code.

And using an agent as running a whole program.

Ashish Rajan: Yeah.

Harry Wetherald: And that's like the easiest way to understand the difference between the two. What that means is that the fact that you use agents doesn't mean it's gonna be a great product. Just like using programs or applications doesn't mean you built great software.

Ashish Rajan: Yeah.

Harry Wetherald: The problem with what I see a lot of right now is that this there's massive, widespread, like screaming about how everyone's using agents, but actually the use cases are fairly shallow if there're at all. Yeah. And so what we've tried to focus on is building the whole company around the idea of what an agent can do if you make that your absolute sole focus and your kind of I don't know the thing that you as a company are absolutely best at.

So that's all we've been doing for the last 12 months is just building up these agents to do more and more advanced things in more and more reliable ways. And specifically what that means is that we [00:12:00] basically give the agents access to what your best security engineer would've access to.

Yeah, that's the analogy we've tried to build the product around. So what would your best security engineer do if you gave them one CVE and you gave them access to the data they needed to understand it, and you let them take the whole day to understand it in as much detail as you want?

That's basically the agent that we've built or the series of agents that we've built.

What that means for us at the moment in terms of the kind of things that we protect is they need access to lots of intelligence about vulnerabilities, right? So they need access to the world knows a lot of things about every CVE. Yeah. But a lot of it's stored as text, right? Which is not ideal. But for AI agents, that is ideal.

They can go crawl through that and understand it in detail. And then they also need access to the cloud environment to understand what's actually happening there. And then sometimes we give them access to additional kind of piece of information. For example, like a compensating control that might give us a bit more information.

But really the analogy is just whatever a human would need to review this properly. Yeah, you give an agent access to it, they can actually understand how all that data fits together, and you don't have to write a ton of logic to, to define how it all fits together.

Ashish Rajan: So do you see yourself as like the layer, which is [00:13:00] making sense of what the wall of red is.

Harry Wetherald: Exactly. Exactly. It's a bit like, imagine if you were to outsource triaging all your vulnerabilities, right? Yeah. And you were to pay some kind of managed service and they were gonna take every single vulnerability and they could have really expert people spend a day on each of them, right?

Which obviously is just financially impossible for everyone, given the scale that we're dealing with here. But imagine you could do that and at 9:00 PM right, you click send. And they go off to this team of a million people. Yeah. And that team of a million people all spend a whole day on each of them.

And then in the morning they come back and you've got this perfect investigation of each, you've got this kind of categorization of which ones matter and which ones don't, and then you've got the fixes ready to go for all the ones that you need to fix. That's basically the analogy that we've built it around.

Ashish Rajan: And so what do you find as, I guess one challenge to call what you called out this overwhelming sense of wall of red. Yeah. Is there complexity in terms of the kind of compute as well? Or does it, does it become easier or most organizations, these just are container first these days.

Yeah. Container workload is very common. Kubernetes is very common. Yeah. What are you seeing in terms [00:14:00] of, how AI agents are working with complex compute. Is that any different or is it better or,

Harry Wetherald: it is a little bit different. A lot of the customers we work with use have very modern cloud native architectures.

Not all of them, but some of them do. And so that does give you additional things in some cases that you can go and look into. Yeah, connectivity between pods and things like this. Yeah. But it doesn't necessarily, again, it's if a human can do it. An AI agent should be able to do it if you give it enough time and examples and stuff like that.

So it really doesn't matter too much between the two. Yeah, it would just have slightly different data that we look at in each case, basically.

Ashish Rajan: Okay. But to your point for people who are trying to think of applying this kind of logic Yeah. Is it for everyone though, or, 'cause I feel like CNAPP was one of those ones, or CSPM, CNAPP was one of those ones, which is not for everyone.

Like I think if you were like just care compliance, you go for CSPM. Yeah. If you want complex workload, you go for CNAPP. Where does this I guess this AI world that you're moving towards with doing, I guess a additional layer on top of CNAPP, what kind of people should be looking at it? And I guess is that people, everyone should start with it, all size organizations or,

Harry Wetherald: yeah.

For, [00:15:00] to begin with, we've been more focused on large companies in general. We do have a few companies we work with who are in the hundreds of employees sort of thing. But typically it's the larger companies. They have a lot of scale, a lot of data. They have, backlogs in the millions they have huge groups of engineers and SREs that are kinda responsible for fixing them.

And so the problem of understanding all of that and then operationalizing it all is really challenging at that scale. But at the same time, there is use cases, even in very small companies that even maybe don't have as stringent security requirements. So for example you can, like you, in almost every compliance framework you'll have to fix.

Vulnerabilities by an SLA. Yeah, but those frameworks also don't specify how you need to do the triage. And so one of the things you can do with a tool like this is basically say, cool, like in our policies we have AI review our vulnerabilities. We have it proved to us the ones that aren't exploitable, we have it leave us with just the ones that are truly exploitable. And then we go off and fix those. And for the auditors, we give them a report showing them why we didn't do the other ones. And so that, that is true, even if you're a 10 person company, you need to do that kind of stuff if you are, if you're under compliance requirements.

So [00:16:00] yeah, I think there's a use case like up and down the size of companies, but for now we've tended to work with the slightly larger ones.

Ashish Rajan: And to I was gonna so throw another acronym that's been very popular, MCP. Yes. And I think a lot of people would assume, and at least the internet seems to believe that.

Most security teams can build their own AI agents with MCP. Yeah, and I guess to, because you're in this space and you've been in this space much longer before GenAI came in, I'm curious, what's the reality of, I, I hear this, I'm like, yep, Harry's great, but you know what, I think I can do this myself. Yeah.

So what is the reality you're doing someone, to your point, security engineer, picking this up. I can do this myself with. MCP and whatever agent. Yeah.

Harry Wetherald: So MCP is another one of these funny ones that in January everyone decided. Cool. We're talking about MCP now, which we have our own thoughts on that.

I'm going to now, but more generally about people building it themselves. I've actually seen across this year in particular, I've had more and more conversations with people that are like, oh, that sounds cool. We're doing, we're doing a version of that internally. And I actually love those conversations 'cause it's really interesting to learn like how they approached it and what they've done and stuff like that.

But the thing that. I think a lot of people don't [00:17:00] understand about working with AI is like the large language model providers give you all these tools, right? And they let you just easily call an API and get an answer back, and that feels very easy to begin with. But the complexity involved at the moment, at least with working at scale.

Working cost efficiently. And then getting accuracy up to a point where it's really reliable requires huge amounts of engineering work.

Ashish Rajan: Okay.

Harry Wetherald: And so the thing I'd always say to people if they're embarking on doing this internally, if you have lots of engineering resources and you can commit those resources to the project for 1, 2, 3, 4 years, you'll probably get to something decent, right?

There's no reason why if you invest a lot of, and you have the right people. Yeah. No reason why you can't do it in-house at all.

Ashish Rajan: Yeah.

Harry Wetherald: But it's one of the most misleading things and that you can get from zero to something quite cool very quickly.

Ashish Rajan: Okay.

Harry Wetherald: But getting from that quite cool thing to something that you can rely on for a enterprise security team is like maddingly long.

And that's basically what we've been for the last 12 months. We, in the first few months of building it, we had a amazing prototype that worked great, but it was like if we tried to run it at scale, it would basically [00:18:00] burn our whole balance sheet in one day. Ah, it would it would go off in strange directions sometimes.

So all our time is spent basically grinding out all those mistakes, making it super reliable, making it really cost efficient, making it scalable. And those are the bits that are hard if you're building it internally to get 'cause it takes a lot of engineering work

Ashish Rajan: because to your point at the moment, if I ask the same question twice, I get two different answers.

Harry Wetherald: Yeah. They're

Ashish Rajan: Related, but they're two different answers. Exactly.

Harry Wetherald: So that's a solvable problem, but it takes a lot of time. I think a lot of people, when they see LLMs and they see that happen, they're like, cool. That just invalidates the use of N LM for this use case. Yeah. But it's not the case at all.

Like you can, there are lots and lots of different techniques you can use to remove those kind of hallucinations or wrong like assumptions and stuff like that. And that's what we spend almost all of our time on. But that's the kind of stuff that just takes, it is just, it's not like sexy, glamorous work, right?

It just takes a lot of time, a lot of testing, a lot of identifying small things, a lot of using a different technique here and there. And then eventually you start to get to the point where it gets really reliable. And I think that's what people are gonna see now across 2025, 2026 is more and more AI products come out that have done that kind of unsexy work, [00:19:00] right?

And figured this stuff out. But it just takes a lot of time. It's not unsolvable, but it is a hard problem basically.

Ashish Rajan: And also to add to what you're saying, because the models are improving as well. Engineer who decides to just embark on the journey themselves. Whatever they do today to improve on that, then they have to start again with the model updates.

Harry Wetherald: Sometimes. Not always, actually. Okay. Like you can, the smart way to build these kind of products is to build them in such a way that you can actually swap in and out models. Have a very easy, quick understanding of how it's changed the product. 'cause what you want to be doing, like you want to the best place to be as an AI product is kinda on the edge if possible.

At the moment because there is, the world has never seen so much investment in new technology. It's, nothing has ever improved as fast as this is.

Ashish Rajan: Yeah. Yeah.

Harry Wetherald: So the smartest place to be is like right on the edge of what's possible. And then as that next model comes out. Oh cool. This is now like working that bit better now.

Yeah, so it's actually, I would, for anyone building it in house, I would a hundred percent recommend thinking about how, when the next model comes out, how you use that model. But that requires you to think a little bit differently about how you build the product and you rely more on kind of building some kind of [00:20:00] scaffolding around it that you can reuse as you use new models.

Ashish Rajan: Alright, so you'll probably work on building the scaffolding first, and then you decide, okay, I'm gonna start from prompting or whatever. Yeah. As Oh, okay. But. That, that scaffolding is what your source of truth is that you always keep coming back to.

Harry Wetherald: Yeah, there's like a whole system that kind of can fit around it.

Prompts are definitely part of it as well. But yeah, I would basically say, think of it less as like you're trying to make the, you're trying to make the system better independent of the model that you're using basically. And then obviously better and better models can help make the product better and better too.

Ashish Rajan: Oh okay. 'cause you're trying to focus on the problem you're solving rather than the what model am I using? Yeah, exactly. Exactly. Oh, that's to your point. 'cause most people would go, I would use the latest Yeah. Claude Sonar. Yeah. Yeah.

Harry Wetherald: Which right now, like the latest Claude model is probably the best thing to use for this.

Yeah. For most cases. So but it just, like a successful product should not fall over the moment a new good model comes out, A successful product should be like, cool new model is out. Let's bring it in, we're good to go.

Ashish Rajan: But just optimize what changes need to happen. Yeah, exactly. Because the scaffolding hasn't changed.

Exactly. Exactly. Ah, interesting. So I guess it's just an interesting point as well because I think a [00:21:00] lot of people obviously go on that path as well. Where does MCP fit into all of this as well then?

Harry Wetherald: For us, nowhere at the moment. I think it's, there's, it has interesting use cases, don't get me wrong.

I think the thing that shocked us in a sense was how many of the big security vendors were like, cool, we now have MCP. Yep. And the difference between where that is in terms of usability for the average security team and how hard it's been pushed seems like it's in a slightly, strange place.

So maybe that will change over the next few months. I don't know. But like we were slightly surprised to see so much emphasis on MCP as an answer to some of these problems because still reasonably nascent and still requires quite a lot of skill and know how to use it properly.

So yeah maybe over the next six months it'll evolve, become more normal part of how we use these kind of things. But for us it's not. It's not, it doesn't really help us right now, basically

Ashish Rajan: because I guess to your point, would it pay for people who are building themselves? And I think where I'm coming from, this is, it is an interesting one where I see the AI market as almost like split in between one.

I had this conversation the other day about build versus buy. Yeah. Now people are asking the same question today with AI being so easily accessible everywhere, every [00:22:00] organization has it. Yeah. A lot of people are making choices between if I have a problem to what we were talking about. I have a wall of red.

Yeah. And I stare at every morning and get frustrated. Now I potentially have an AI system that I can give it access to and hopefully be better at it. But to what you're saying is I. You, you should definitely try that. But at the same time, once you realize the complexity behind Yeah. Ongoing management

Harry Wetherald: I, anyone that talks to me about it, I always say the same thing, which is just go and do it this weekend.

Yeah. Like it's so quick to get it up and running. You can get a sense of what it can do really quickly, which is exciting and not, and unusual versus what we used to have to do. I always say to everyone, go and give it a try. And some, if they keep the use case like narrow and constrained enough, they should be able to get value out of it in certain ways.

I'd just say to them, if you really want to rely on it doing end-to-end automation of like quite complex tasks, just be ready for some pain basically. And be ready to commit the level of engineering resources needed to make it successful. And if you have that, then you know, maybe you can make it work fair.

Ashish Rajan: And I guess bringing it back to the vulnerability and reachability, then I guess. That's the next layer of [00:23:00] complexity to these challenges. AI agent has basically helped me narrow down and I find the vulnerabilities. I've gone down this path of getting excited about the possibility that I can reduce my criticals to hopefully zero.

What does that normally look like in terms of what it used to be before AI agent was a thing, and what is it now if people were to build an AI agent for it?

Harry Wetherald: Yeah. So there's so many products that try to help you with vulnerability management, as yeah. And the easiest way of distilling most of them down is that the typical approach has been either, we used to rely on just CVSS calls.

Yeah, and just, we had a big list sorted by CVSS and most people familiar with the problems there, which is that does nothing to put it in the context of your company or how risky it might be. So we came up with, over the years, all these new approaches, EPSS or Kev. And really what we ended up with was these kinda like scores.

And the scores tried to capture some combination of CVSS plus or minus a few features, right? So the most common things are like we take A-C-V-S-S score and we say it's on a public IP and it has threat tell. So now it's a nine, right? It goes from a seven to a nine because of these two [00:24:00] factors we found that make it seem more risky, the.

The benefit of that is it helps you get beyond just the simple list of CVSS scores. But if, like any, ask any person that does this day to day and just be like, how many of the vulnerabilities, false positives? And they'll be like, they'll give you a level anywhere between nine 90 or like 99.99%, which someone did say to me the other day.

Yeah. So if that's true, then those kinda scoring models that take your sevens and turn them into fives and nines. Yeah. They're reshuffling the wrong thing. If if 90% plus of our vulnerabilities are false positives, then you end up with a re-sorted list of false positives, basically.

Ashish Rajan: Yeah.

Harry Wetherald: So that's kinda the difference between like where we were before, I think where what people end up doing is they, there has been a few more modern techniques people use, like runtime agents, stuff like that to try and break down the list again, but they end up being quite constrained to a single technique they can use.

Yeah. And a single they don't always have good visibility into all sorts of different environments and stuff like that, so they end up being able to chop a bit more of the tree down. But not really get to the full human level view of what, what's going on. So people end up.

[00:25:00] Often either they just push all the vulnerabilities the best they can out to the people that need to fix them, can I hope for the best, and then frustrate the hell out of those people because they have to deal with all these false positives. Or they try and hire people in house that can go through and manually triage everything, but the volume is so high that just gets so hard, so quickly.

And so that's typically when we tend to come to people, which is they've tried a few of these different things. They've maybe built their own scoring model in-house. They've tried some products, they've tried building a team, and they're just like, there's just no way of managing all this volume to the level of detail that we need.

And so that's then when, in my opinion, agents can come in and say basically, okay, imagine that team of two people, three people you've built, what if it was 5,000 people or 10,000 people? Yeah. What outcomes would you be able to generate? And that's basically what the AI agents can go in and generate for you.

Ashish Rajan: Yeah. That's awesome. I think that's all it's funny, I think if you would've had this conversation before GenAI was a thing come on, man. It sounds ridiculous. Yeah. It sounds totally ridiculous. Exactly. I'm like, what are you what are we doing over here? But I think I love the idea.

I also love the fact that you are able to get an AI agent to a point [00:26:00] where you can have reliable re responses, which is what you need in a security context, that higher trust level as well so when you do go back to your development team, you have a lot more certainty for this is not a false positive.

Harry Wetherald: Yeah. And then that builds the trust as well, right? Yes. Because if you go, if I like an SRE and I go from receiving 10 vulnerabilities in a certain time period, and I know deep down a lot of them are nothing.

Ashish Rajan: Yeah.

Harry Wetherald: And then I maybe go and investigate 'em myself and prove it, or I maybe just put my head down and fix them, or I maybe just ignore them, which may you quite covered.

Yeah. If I go from that. And with all of those vulnerabilities coming to me with quite low context, if I go from that to like I get one vulnerability in that same time period. And it comes to me with a big, like clear proof on why this thing is exploitable and what would happen to our business if it's exploited and why that's bad to us.

And then I also get some kind of here's how you fix it. Here's a button you press to fix it.

Ashish Rajan: Yeah.

Harry Wetherald: The experiences as an SRE is so like maddeningly different in that scenario. That it not only helps them just move quicker, but it also builds a lot more trust with the security team. Yeah. Because then the next time they receive something, they're like the last one I got was good.

Yeah. So I'm gonna look at this. And then that trust builds and [00:27:00] builds. And then we get out of that initial spot that we used to be in, where everyone was disagreeing with each other and trying to trying to figure out who was wrong and who was wrong.

Ashish Rajan: Yeah. So you don't dig yourself deeper into the laboratory.

Exactly. You created, so I guess maybe adding another layer to this as well then, if people are trying to build this. You obviously went down the path of using cloud as a first option. Yeah. 'cause there could be multiple kinds of vulnerabilities. Yeah. And I imagine it's the same in the AppSec space.

Yep. 99% of I, I'm not saying 90, like just say over 90% are always false positive. Yeah. In most scenarios, why did you take the cloud path? And I guess because I'm thinking also people who will listen to this Yeah. Are already on that build versus buy fence. Yeah. And they're already in that, Hey, I can build this.

But you obviously made the conscious choice to go towards, after hearing all those conversations, you went on the cloud path. But why cloud not AppSec and other things we just

Harry Wetherald: Yeah. Pulled by web where people were pushing us to go in terms of what they needed solving for them, basically.

So that was a more priority at that point from Exactly. Yeah. Yeah. Yeah. And I think where people like I had a good conversation with someone about this. They were the like one of the security leaders in a big kinda public tech company, and they were talking to me about this exact topic and [00:28:00] they said that.

If I want to, if I get one thing it's what's in production, basically.

Ashish Rajan: Yeah.

Harry Wetherald: So if that's the thing I wanna make sure I'm getting right and then everything else comes second. And that person actually had been massively reducing the number of typical application security tools they used.

They felt like Shift left in a sense, had failed them a little bit. They felt the developers were just very frustrated with what was happening. So they'd massively reduced their reliance on typical application security. Yeah. And they were ramping up their reliance on. What was actually happening in production and what was happening at Runtime.

And so I do think there's a good case for using this approach in both worlds. But that was

basically where we started, where we did just what we heard from the people that we spoke to.

Ashish Rajan: And I guess even to start with building AI agents in a cloud context. What's a, are there like, I don't even know what's a good use case?

Should you focus on services, should you focus on products like

Harry Wetherald: this?

So in terms of what data you try and look at? Yeah. Like where do you start? That's, there's so much, that's the brilliant thing about agents is that you don't need to, it breaks so many of the paradigms of how we think about building products.

Because you don't need to [00:29:00] define so much upfront. You define like the inputs and the goals and you let them go work it out. Oh, and obviously you guide them along the way Of course, yeah. Of what they're trying to do, but,

Ashish Rajan: and give them context as well. Yeah, exactly.

Harry Wetherald: But much if you bring a new security engineer into your team and they're, they know what they're doing and they've done some things before, you don't say, great to do this, you need to do here's a playbook of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 things. And here's the data to go look at and here's the exact API to call and here's the exact query to call with it. You say, Hey, we've got a vulnerability here. It's a, it's something to do with the Linux kernel and it involves these machines and it's, here's the information on the vulnerability.

Off you go.

Ashish Rajan: Yeah.

Harry Wetherald: And so the human in that case, or the agent in our case goes off and define the own adventure which is what makes this way of building products so fascinating because. It implies basically that you can move like around much more quickly, because you're no longer, if you think in the old way of doing things, where you're always defining a rule to define how a product should behave in all cases, it means that you're always, every time you give it new data, you have to build a great big library of logic about what to do with that data and [00:30:00] how it all fits together.

Ashish Rajan: Yeah.

Harry Wetherald: In our world, it's wonder what happens if we just give it WAF policies, right? And then you give it access and you see what happens. And then obviously we need to tune that and improve it and do a bunch of things to make sure that it uses them sensibly, but. It changes a lot of the nature of what you need to go and start with.

It's more a case of think about the sources of data that, that a product might want to, an agent might wanna have access to.

Ashish Rajan: Yeah.

Harry Wetherald: Think about what might be needed to achieve your goal, whatever it might be, and then make sure that there's the right controls in place of having read only access and limited access to stuff like that.

And then let it go and see what it goes and accesses and the amount of times that we've sat there and we've read through one of these investigations and we've been like, huh, it did that. Didn't expect that, but actually it was smart, right? Yeah. Yeah. And so like you actually see this like a slightly emergent behavior where it starts going and looking into things on its own accord rather than you driving it to something

Ashish Rajan: in a funny way, you wanted to hallucinate a bit

Harry Wetherald: in a sense.

You want it to get creative basically. Yeah. You want to start thinking about like a lot of the way we try and get the agents think is oh, what if I went and looked at that? What if, and like sometimes it might come back and be like, ah didn't tell me [00:31:00] anything.

Ashish Rajan: Yeah.

Harry Wetherald: But much again, an inquisitive security engineer might go.

I might just go look at, the VPC logs around this, because I'm just intrigued to see what I find. Yeah, you want the agents to behave slightly like that so that they unearth things that, that you wouldn't have known look for. Oh,

Ashish Rajan: So your goal could be as simple as that.

Hey it could be as simple as how many S3 buckets do I have over to the internet all the way to. Hey, what's the status of S3 bucket 1, 2, 3, 4?

Harry Wetherald: Sure. Yeah. Like those aren't things that we do today. Like we just focus more just on the risk. But like you could play around, if people are wanting to do this themselves, they can absolutely play around with giving an agent access to their cloud environment and then playing around with asking it to bring back data for them like that.

But you can also ask more subjective questions, right? And let it go and try and answer that subjective question by assembling all this data from different places. And that's means

Ashish Rajan: what's an example of a subjective,

Harry Wetherald: In, in your example there of talking about S3 buckets, rather than saying, Hey, tell me which S3 buckets are open.

You might wanna say tell me which S3 bucket that if opened would cause the biggest damage to us.

Ashish Rajan: Oh, and

Harry Wetherald: that's a much more subjective question, but a question that agents can answer, especially if you give them enough kind of [00:32:00] guidance and context. But that's where you're gonna see them go off and start looking at data that you probably

Ashish Rajan: also red teaming them in a way.

Yeah, exactly. Exactly. You're just basically asking the AI agent to red team the S3 buckets

Harry Wetherald: In a sense and slightly different to the typical like model of red team. Yeah. Yeah. There's a lot of similarities there.

Ashish Rajan: Yeah. You almost wanted to hey, think like a hacker or like a bad hacker and tell me what could go wrong here.

Yeah, exactly. And then you build controls around it.

Harry Wetherald: Exactly. And this is why, the more time I spend on this and our team spends on this, it's just so obvious that this is how everything needs to be done. Yeah, because the level of like flexibility and power you can do is just, it's not like twice as good as writing some rules and logic and whatever.

It's like hundreds of times as good because you get to just. Get it to delve into these topics and go explore the data and pull it all back and understand how it comes together. And the difference between that and trying to write a single piece of rule-based logic to do something you want it to do is is amazing.

But it's hard. It does make it easy, but like it does mean that, I think, in my opinion, most of how we do security tooling and software should start to become, using these kind of paradigms.

Ashish Rajan: And where do you see the people who you're working with [00:33:00] use AI agents? What kind of questions are they asking outta curiosity and on their AI agents?

Harry Wetherald: So asking them to do yeah. So we actually don't at the moment allow them just to ask any question. Okay. So we have basically defined upfront the questions that they're asking, and that's all about just purely here's my list of vulnerabilities. How risky are they? What should I do about them?

Ashish Rajan: Okay.

Harry Wetherald: So that's the questions that we've set it up to ask. But in theory, yeah, you could over time start using the same agents and allowing people to basically ask their own questions and explore their data. And that's gonna be a really fun thing to explore when the time is right.

But for now, we've tried to keep it reasonably constrained. I actually find there's a lot of AI products out there that are like. Hey, here's our big open empty box. Come and do what you want.

Ashish Rajan: Yeah.

Harry Wetherald: And from a design perspective, that's actually like a really bad practice 'cause okay. You aren't giving people an ability to understand what the product can do.

You're just saying, here's a big empty box, good luck. Ah, and so I think for me, product should always, like AI products should, for the most part, first, show you what you can do Right. And do a lot of work for you. And then, if they wanna layer in kind of the bigger empty box [00:34:00] kind of feel, so you can do some more customizable stuff, then great.

But the big empty box for me is it's like a product just being a bunch of toggles that you have to go and do all yourself before it does anything useful. You wanna do it, you want it to do something useful first, and then you come in and customize it later.

Ashish Rajan: Interesting. This is really fascinating for me 'cause and for people who are building programs for security in 2025, 2026. What's your recommendation for where do you feel, obviously everyone has a budget for AI now. I don't know of many people that don't have a budget for AI security. In terms of putting this in a program for security, whether it's vulnerability management or to your point specifically for, Hey tell me about this particular vulnerability.

Where do you see, if you were to just zoom out a bit. Where do you see the AI impacting most today? In a security program? In a just say a Fortune 500 company.

Harry Wetherald: Yeah, just

Ashish Rajan: in general across the whole security program. Yeah. Yeah.

Harry Wetherald: Yeah. There's obviously a few areas. The quickest moving one has been the SOC.

Okay. And so I'm sure you've met many people building products in that world. Yeah. 'cause there, there is quite a few of them. And some great products been built by some great people. That's [00:35:00] probably the fastest moving one so far in terms of, it was such an obvious thing that we had these big, rooms of people doing semi repetitive work.

Yeah. And therefore it seemed like a very obvious place to start. And so that is gonna be definitely a big one, I think over the next few years as people start to use more and more of the kinda like SOC automation tools to do threat detection response. I personally think that this area around what are my biggest risks and how do I solve them is the other big area of investment that's happening that's gonna happen around AI in the next few years, which is basically, if you think of as a security person, if you really boil down what my team is responsible for, it's find an attacker once they've got in and kick 'em out.

Ashish Rajan: Yeah.

Harry Wetherald: Which is basically SOC and SOC automation today. And then it is, figure out how they're gonna get in tomorrow and stop it from happening. Yeah. Which is basically this whole area of kind of vulnerability management and broader kind of posture management.

Yeah. We really need amazing AI products and tools that can do, both sides of that fence. And so I think those are the kind of problems we're gonna see the most the most activity around. There's obviously other things it can do, security awareness and stuff like that, but I think those two most fundamental problems of where is the attacker and how do I get them out? And then how do I stop [00:36:00] them getting in tomorrow is where we need to focus our kind of efforts from AI.

Ashish Rajan: Wow. Awesome. That's most of the technical questions. I had got three fun questions for you as well. Yeah. First one being, what do you spend most time on when you're not trying to solve the AI world problem with AI agents?

Harry Wetherald: What do I spend most time on? Probably nothing that fun, to be honest. I like to go out and play sport every now and again when I can, but just, spending time with, with family, friends reading every now and again, but yeah, nothing that, yeah, unfortunately the life of a startup founder doesn't leave you tons and tons of time for hobbies, I'm afraid.

Ashish Rajan: Fair. Second question. What is your proudest moment that is not on your social media?

Harry Wetherald: Not on my social media. Plenty of things. From a, from like a work capacity. I think it's the early like engagements we had around the product with Maze, I think seeing it actually in action and seeing the level, like the insane things that it's able to do has been like one of the most exciting things in my career and I've worked with machine learning, like my whole career, I've seen a lot of cool moments of machine learning, doing interesting things, but these have been like levels, and levels above what we've seen before. So those are definitely some of the most exciting things we've seen recently.

Ashish Rajan: Awesome. And final question.

What's your favorite cuisine or restaurant that you can share with us?

Harry Wetherald: I am, [00:37:00] I love. Like most foods. So it's a hard question, but yeah, I think I think for me it's always comes back to Italian.

Ashish Rajan: Oh, nice. Pasta or pizza? Pizza all the way. Pizza. Oh, Italian. Was it nap style? But I'm not gonna go into the types of pizza, but I think it's, where can people find you on the internet? Talk more about what you're doing at Maze and other pairs as well.

Harry Wetherald: Yeah, I talk quite broadly on LinkedIn to be able to find me on LinkedIn. I also have a substack, so I post semi weekly or biweekly on there, just about anything that's interesting me at the moment in security.

Awesome. So that'd be those would the places.

Ashish Rajan: I'll put that in the show notes as well. Thank you so much for coming.

Harry Wetherald: Awesome. Cool. Awesome. Thanks, for having me on.

Ashish Rajan: Thank you. Thanks so much for joining tuning In people to see you next time. Thank you so much for listening and watching this episode of Cloud Security Podcast. If you've been enjoying content like this, you can find more episodes like these on www dot cloudy podcast or tv.

We are also publishing these episodes on social media as well, so you can definitely find these episodes there. Oh, by the way, just in case there was interest in learning about AI cybersecurity, we also have a sister podcast called AI Cybersecurity Podcast, which may be of interest as well. I'll leave the links in description for you to check them out.

And also for our weekly newsletter, where we do an in-depth analysis of different [00:38:00] topics within cloud security, ranging from identity endpoint all the way up to what is the CNAPP or whatever, a new acronym that comes out tomorrow. Thank you so much for supporting, listening and watching. I'll see you next time.

‍