Mohan Kumar: [00:00:00] We totally are underestimating how agentic world is reshaping entire cybersecurity to voice assistance talking to each other in a completely different mode of communication. Totally abandoning human language.

Ashish Rajan: What are people underestimating about the threat surface when autonomous agents are more prevalent?

Mohan Kumar: Memory poisoning agent typically trust its memory, but the goal are the context itself has been changed. Any model is susceptible for those kind of tricks.

Ashish Rajan: How do I monitor this behavior and can I audit. This 2025 it seems like everyone just wanna talk about agent AI. So this week we have Mohan from Box.

He runs production security there. We spoke about. What is the transition from LLM to agent AI that's happening everywhere. We also spoke about the top three AI agent security threats that exist in production environments that people should consider, and how do you monitor and detect some of the changes that are happening in AI?

While AI seems to be working and running faster than you are. So [00:01:00]all that, a lot more in this episode with Mohan Kumar from Box, who's a production security person there. If you know someone who's working on securing AI agent security, definitely share the episode with them. They'll definitely find it valuable.

And in case you are here listening to this episode of Cloud Security Podcast for a second or third time, I really appreciate if you take a second to just hit the subscribe button, even if you are on Apple, Spotify, YouTube, LinkedIn, we are everywhere for audio and video, whichever your favorite platform is.

I would really appreciate if you take a second to just hit the subscribe button. It helps more people discover us and get more guests like Mohan as well. Thank you so much for your support and I will talk to you in next episode. Peace. Hello and welcome to another episode of Cloud Security podcast. I've got Mohan Kumar with me.

Thanks for coming on the show, man. Hey, thanks Ashish. Thanks for having me. I'm excited for this conversation. But actually before we start, could you share a bit about yourself or uh, people who may not know who you are, so get some context about your professional journey.

Mohan Kumar: Hey, I, I've got over 14 years of experience in cybersecurity.

Uh, currently I work for Box. Prior to that [00:02:00] I worked for organizations like Morgan Stanley, cn, uh, bell. And, uh, my journey into cybersecurity came in after my master's in security back in 2011. And since then I working in the same field. I'm really passionate about, uh, solving problems in cloud security, uh, especially around like containers.

And in last couple of years, I am, my interest has been into AI. Uh. I, me and my friend Navin, we put together a course in Udemy on a, a security bootcamp, uh, that gained more than 1800 learners so far. Um, we did actively spoke at several conferences. Uh. And you know, really passionate in learning how AI is reshaping the world and the intersection of AI and cybersecurity.

And also as [00:03:00] we are deep diving into agent era where we wanna give more autonomous functions, uh, autonomous actions to ai. And I'm really curious to see how we gonna improve cybersecurity within agent workflows. Um. Yeah, that's pretty high level.

Ashish Rajan: Yeah, no, thanks for sharing that because I was gonna say, even before the call we were talking about Kubernetes.

So, uh, I'll definitely take, uh, some more insight from you on the Kubernetes side, some someday as well. Talking specifically about AI agents, and I know you and I kind of spoke at B side, uh, SF as well. You had a talk there. LLM application is an interesting I guess it's an interesting era, let's just say.

How do you define AI agent? And is that what typically people mean by LLM application or is LLM application different to an AI agent? It.

Mohan Kumar: Yeah, typical LLM application is different than a an AI agent. Uh, so think of more LLM [00:04:00] application as a one shot, uh, response to a prompt when chat GPT was rolled out a few years ago.

So we can consider that more as an LLM application where we ask some question and then, uh, you know, LLM reasons and then gives back some answers, right? So it's one shot. Response, and it doesn't have access to external tools or APAs or anything. It's just one model that thinks through and gives some answer, and it cannot make autonomous decisions or any actions on our behalf.

But whereas this a agent in contrast is they're pretty dynamic. They're goal-driven. So when I say dynamic it, it just doesn't follow a fixed workflow or anything. It just adapts during runtime based on what's happening. Usually the the agent is, is mainly com, I would say like, you know, we can think of three step processes. [00:05:00] You give some query to the agent and then the agent thinks and then acts, and then yeah, uh, does some observation and how things are going. And then it ends the process. So this is like super high level three step process for how a agents work.

And then there will be an orchestrator. Uh, who is helping to, uh, ha you know, get this task done. Whether it is connecting to external tools, uh, or connecting to other agents because these agents are solely trained to do one particular job. And then if we wanna do another job, for example, Hey, you know, I wanna write a research paper.

So research is something that need needed at the beginning stage, and then, yeah, uh, we need to do more on the writing. So there will be two agents. One is specialized on research, and then the [00:06:00] second one will be specialized on writing. So these two agents is gonna, you know, interact, communicate, and then, you know get the job done. So, right. If it's writing a research article is the end goal, it will be solely focusing on that, getting all the things needed, like constantly doing this three step process, think, act, and observe, process.

Ashish Rajan: So then what's the, uh, I guess to your point then, LLM application people, now we are trying to talk about a chatbot kind of a thing.

AI agent sounds like could be a multi AI agent in that context, which is basically making autonomous decisions. Uh, or autonomous agent or agentic AI as people call, keep calling it. Now, I think a lot of times every, I talk to security people. A lot of us are in this two camps of one is we are prepared for this AI threat world because our so and so vendor told us we have a coverage across the board.

But I think [00:07:00] one thing I found something really interesting in your talk, and obviously I'd love to hear from you as well on. What are people underestimating about the threat surface when autonomous agents are more prevalent and what would change?

Mohan Kumar: Yeah. Yeah, it's totally a, a great question. We totally are underestimating how agents agentic world is reshaping entire cybersecurity.

Yeah. As you might have listened to Andre Karpathy's recent video where he talked about. Software 3.0. Right, yeah. Software itself, you know, has not changed for over many years. Yeah. And then recently titrated twice and he referred clearly, Hey, we are Software 3.0 where. You know, more the English language.

Our prompts are becoming the new software, right? And yeah, as we look at the GitHub things, right? So definitely as we look at from that holistic approach, if the software is changing, so do the [00:08:00] cybersecurity as well, right? Like the way we secure them also has to change. So that's why I kind of feel we are kindly underestimating.

I also, uh, you know, brought up a video clip that went recently viral in, in our talk which, which showcases like, hey, you know, two voice assistants talking to each other in a completely different mode of communication. Totally abandoning human language. Uh, it's more like, you know, like in.

Uh, in Star Wars movie where we see, uh, R 2D two, uh, yeah. Relationships, right? So, so totally. Uh, that was funny. On the surface though, like, but it highlights a serious point. Security teams might not anticipate that AI agents could create unseen channels or behaviors outside, uh, the normal bones, right?

Yeah. It's a remainder that autonomous agents behave in unexpected ways than traditional software [00:09:00] would. Yeah. Wouldn't uh, you know, those two wise assistants, they just, figure it out, uh, on the go and they're making some actions, which is totally not in any of the security checklist, right?

So because we've been trained to, you know, hey, monitor these channels, but then what are the, the COVID channels that they. They are coming up with, uh, it just emerged right now, right? So these are some of the things that we should be carrying more. And, uh, you know. Also constantly check, Hey, you know, what are the failure modes that could, lead to you know, an organizational risk or, or a cybersecurity incident, right?

Yeah. So yeah, thinking through these processes are, are not there yet. And and that, that's one way I call, Hey, you know, we still need to. Think through all these new attacks I faces threat vectors. That's coming up in our way. Yeah. [00:10:00] So yeah. Again I would say this is like a new class of things that are coming up.

So the video, uh, that I played, uh, you know. In the, uh, in the, the talk? Yeah. For anyone that you know, for the context if you can Google, uh, a assistant, talking in GI Link mode, you'll be able to find that video. That's super funny. But then it's a good,

Ashish Rajan: yeah, I think. Jib link, link jib link mode was, uh, something new for me at that point in time as well.

Talking about threats, what are the top three threats that come to mind, which you are finding, uh, obviously whether you are directly working with AI workload or looking at AI workload, there is a whole or top 10 for AI agents. Yeah. But what are the top three that keeps coming around? Uh, you, when you look at the whole AI agent threat landscape, because you're the whole, uh, I think you're obviously a lot, a huge component of your messaging has been around the whole threat modeling for AI agent security. Yeah. So what are the top three threats that come to [00:11:00] mind, which probably most enterprises may maybe are thinking about or maybe not thinking about?

Mohan Kumar: Yeah. Uh, I think when we think, you know, as we deploy AI agents, right? So basically, you know, most of the agents have three types of memories.

Uh, one long-term memory. Second is short-term memory, and then third entity memory where. Just store some level of entity, hey, tied to a person or an organization. So most of the, a agent framework like crew ai will have these kind of three memories where it stores all the context, conversation, past history in making decisions.

So tying back to the top three threads, right, like, you know, I would say memory poisoning and, manipulating these context will be, will be very, crucial threat given you know, these yeah. Can alter the entire decision making process [00:12:00] and unauthorized operations. So basically.

When I say memory poisoning, it just involves in exploiting the, the three kinds of memory that I laid out. And the context manipulation is involving, uh, the agent's context window. So, 'cause every agent has, you know, is working through a step of processes and during that. You know, it has some context to, to work on.

And if we could alter those context, then, you know, it's gonna be doing things in a different way. I can. You know, gives a a, a rough example, Hey, you know what if an attacker is able to inject malicious information through indirect prompt injection you know, so usually these memories uh, and sometimes they rely on rack pipelines that.

It's a whole bunch of knowledge based documents are in there. And, and if we are able to poison them in an indirect, you know, through indirect prompt injection, they [00:13:00] can influence the a agent's future actions to cause ununited behavior. I consider this as high risk, uh, for AI agent because agent typically trust its memory, because it's, it's its own memory.

And every time it. Uh, it makes, uh, you know, calls to, uh, these lms, uh, it has to send whole bunch of context, uh, that is grabbed from this memory, uh, and send together, package them in a system prompt, right? So in. So it just trust its own memory. And if its own memory is being corrupted, then, if it's, compromised, then it just, like, hey, you know, uh, these agents think, hey, you know, I'm just doing the job that I'm intended to do.

But, you know, the goal or the context itself has been changed. So, so that's why I see this as. A [00:14:00] serious threat? Um, yeah. Yeah. What, what, what

Ashish Rajan: about the the other two?

Mohan Kumar: So the other two will be, uh, misusing the tools. As you see you know, CP is widely being, uh, becoming a new standard. Uh, yeah.

When connecting between tools. So when I say tool misuse, right? Like. Trusted users and agents interact with the dangerous external board. So we can, it's essentially you know, we can bring in, uh, different kind of, uh, route MCP servers or tools that are, you know, talking to, uh, the trusted agents.

So. Uh, the, the attacker can exploit these these these workflows and, and the processes basically, again, uh, coming back to you know, like, um, using indirect prompt injection again as a, as a way to abuse, I mean, one, one example can be, Hey, you [00:15:00] know, I rely on a copilot. That is I rely on a copilot, uh, that has access to calendar tool to book my meetings, right?

So this is a simple workflow. So what if an attacker are able to abuse this? Processes and, uh, you know, like misuse the tools here. The tools is a calendar. So instead of Yeah. Um, sending a regular calendar invite, because think of it, an agentic workflow where, from, again, tying back to the three step process, right?

It has to think, act, and then observe. So you give a task, hey, you know, go book this meeting with ois for on, on July 30th, 10:00 AM right? So it has to think through the goal and then it has to make the plans. It has to call in terms of action, it has to call. My calendar and then your Calendly link, and then it [00:16:00] has to check for your availability and then mine, and then eventually book, and then it has to observe and wait for your confirmation.

So these are the, the, the workflows that, that, that are done. If we can take this in a malicious way, we could misuse the same Calendly you know. Calendar tool to exfiltrate, even data, Hey, you know, can you grab, uh, other calendar invite, right? So there are multiple ways that we could, misuse the tools if there are no proper permissions laid around, or if there are no context or there are no you know, minimum time set for these, uh, access.

So basically we have to scope. Like a minimal scope with short duration as much as possible. And then if it's there is like high risk action that has to be like you know, a human in the loop involved. Yeah. So that's what I would say, like, to misusing the tool. What about the third one then?[00:17:00]

Yeah, the third one will be more, uh, you know, the privilege compromise, right? Like more it's not tied to like a agents or anything. It's, it's common, uh, you know. We have to, uh, you know, follow the least privileged principle. You know, even with recent, uh, rollout with Open AI's agent I wrote a post saying thing, Hey, you know, yes, it is too exciting to go try and, but we have to ensure we give only the minimal number of tools that, that it needs to perform the goal. So again, privilege, uh, you know, least following the principle of least privilege will be my third one. You know, and from a threat standpoint, you know, it's compromising the privilege. So mostly, you know, this is through the misconfiguration in the agent.

The, you know, an attacker, executes, queries, and, uh, you know, rag databases to access files and data that it shouldn't be able to access. So [00:18:00] this privileged compromise will be a huge threat. Um. That is commonly applicable for, any security world, not just specific to AI agents. Yeah.

Ashish Rajan: Okay. But to your point then so now that we have your poisoning covered, we have tool misuse covered, we have your a access covered in terms of just aligning this back into.

Uh, an organization 'cause obviously organizations typically have, Hey, we have an instant response, or we have some kind of capability. How do you monitor for this? Like, I, 'cause I think in my mind, as you kind of shared all of that in terms of what, uh, some of the mitigations are as well. The first thought that I had was, okay, what, how, or what are you using, which is in the quote unquote traditional, it sounds very weird to say, the traditional security tool that we use today.

That covers for this until I, unless I have a proxy that just basically scans the entire [00:19:00] internet, oh, sorry, intranet, uh, for what is doing what? And basically hopefully have a way to scene through it. A how do I monitor this behavior and can I audit this? 'cause obviously you guys had a whole demo and everything, especially working in the AI space, it's not just worthwhile cla clarifying, it's not just ChatGPT and Claude, but there's like all kinds of like notion AI was an example that you guys had.

Uh, most third party services these dayss have traditionally, and I I use the word traditionally very loosely. Um, we used to be able to monitor for logs. Pick out the fact what is an incident when it's an AI agent and to what you've kind of shared memory poisoning, tool misuse I guess permissions kind of, there is some kind of auditing to an extent in a cloud environment.

You can look at AWS Azure, they all have audit logs. CloudWatch logs, I guess in AWS or sim, similar logs logging in GCP as well, and Azure. What is the, I guess, as a, as a security engineer looking at this problem that, great I have these threats, but [00:20:00] what are some of the ways I can audit this or monitor this for when some, when an agent is misbehaving?

Mohan Kumar: Yeah. Uh, it's a great question. Asis, I think from a, monitoring and mitigation standpoint, again, like tying back to those I am roles. The problem here is a agents do like lot of shifting roles. They shift between multiple roles. It's just ensuring there has to be you know, granular access control, which means, getting into, um.

You know, more granular, as much as possible, and when the, the identity shift, those log has to be in place, right? Because it is possible even during the time of investigation, it's very hard. To know, like who did what those service accounts involved. So [00:21:00] I would say, you know, we have to you know, regularly you know, sanitize the memory routines and then take snapshots for forensic analysis.

From a, you know, from a monitoring specifically, Hey, you know, how can I, you know, look for anomalous behaviors, and even we could have some secondary agent who, you know, that can help in monitoring that can follow the behaviors because it has to. You know, over the time there has to be some baseline said, Hey, you know, for these kind of goal-driven actions, these are the new norm.

Like baseline, hey, you know, has to call these tools only if it's calling some external tool because it, 'cause AI agents are dynamic and then if they are blocked in something, they're gonna do something, uh, some other work. So constantly looking [00:22:00] on. What are those anomalies? And definitely, it is a tricky, especially, uh, that the one that we will cover that we will talk more is on the alignment faking threads, right?

So where it's super hard to, you know, to, to figure out, um, those issues. I mean, even Anthropic has, uh, research going on in that particular issue. So in terms of, coming back to mitigations and monitoring, I would say like we have to have implementing these, uh, you know, memory content validation, session isolation and we have to continuously monitor how, you know, the re the reuse of sessions and you know, having a robust authentication mechanism for specific to memory access.

Yeah. That, that could be crucial. And then again, anomalous detections. Yes. The traditional software I mean [00:23:00] the traditional tooling today is not, is not gonna fully, flag these all the behaviors. So that's why we need more of these either in-house or you know, from vendors that, that can dynamically.

Listen and observe the actions and flag the risky operations. So that's one way I see as a mitigation. And then from from the tool misuse, again, like we have to scope those access tokens to, to the minimal pro, you know, tools. And then we need to have some auto conform you know, like.

I mean, we need to have some level of human authorizations for doing some high risk actions. So those will be all from a mitigation layer from a security operation side we have to leverage more on the secondary. A agents who can continuously sit down the side, Hey, this [00:24:00] is an assistant like not an assistant.

I am a, I'm an observer for the, the main agent, right? So that ki that sort of AI agents can be deployed. Uh, again, lot of AI agent projects are still in experimentation phases, uh, and we only very few got into productions and again, say putting a agents in production is too hard. Uh, not only from an implementation standpoint, it's mainly on the trust and confidence that organizations has.

It's, it's super low today. Yeah. Uh, and, and again, this area has to evolve from a monitoring angle, uh, so that we can have more confidence in, in especially being prepared for these all kind of unknown behaviors.

Ashish Rajan: Do you find that, uh, I think it's so interesting thing that you mentioned as well, because I almost find that.

People are not even aware of what an ai, uh, agent architecture looks like today. What would be an [00:25:00] example? 'cause your point about the threat modeling part maybe they could have found the information about the memory poisoning, all of that. It sounds great in theory, but don't, people don't even know what the structure is for an agent architecture as well.

What does that look like for organizations.

Mohan Kumar: So for organizations it's mainly, uh, you know, some of them might be relying on some agent frameworks like crew ai is one of them. Uh, hugging. Yeah. Yeah. And hugging face as its own. You know, it's mainly when we say a agents. I would say it's like a six major component involved a fixed component.

Yeah. And yeah, let's break down first what those components within AI agents, and then we can see what, what are the threats in those, each component. How we can, you know, threat model, these agents exist because they are, you know, they're trained to do specific goal. Very good. So that's why even we [00:26:00]have multiple agents, right?

Why we, we do not connect whole bunch of tools in one agents, right? We do not need to do that because, you know, if you give so many tasks to same LLM, you know, it will not do a great job, right? It, it has to play the role that it, it intends to, and then it has to train on that particular role over the time to have great work done.

So the coming back to the six major components for AI agents is, uh, the role playing is one part. Then it has to you know, focus that second one. And then it connects to different tools. Uh, the, the third component, uh, and then the fourth one is the cooperation. So they have to cooperate between all these things going on.

And then the fifth one will be guardrails, uh, which is a blend of more operational guardrail and as well as security focused guardrail. And then the sixth one [00:27:00] will be memory. So. These high level components make an a agent. And when we do, uh, you know, threat model, these agents, right? So there are you know, various frameworks that that can be adopted.

Uh, so OWASP Top 10 LLM applications and Agent AI is a great list. Uh, and then Agentic AI threat modeling framework, which is dubbed as Maestro by the Cloud Security Alliance, uh, is a great one. Uh, it's a seven layer threat modeling approach specifically designed for agentic ai that guides us through, uh, you know, considering threats from the model to the agent ecosystem that I just referred.

And then, you know, security teams can use that as a blueprint to systematically analyze where controls are needed. So the seven layer within the master framework are, [00:28:00] our foundation model number one. They, they are the, you know, core A models, right? It can be like GPT-4 oh or custom model. And then the second one is, you know, threats looking at the data operations the data used by the agents, including the memory storage or processing, uh, and then the vector rag embeddings, et cetera. So these are the, things where the data is coming in, into the workflows. And then agent framework themself, which, you know. Again, the framework, uh, that I, we just discussed about the you know, or, or hugging phase and right.

So again, we have to specifically, focus our threat model within the frameworks because those frameworks helps to create the agents and the interactions between them. And then coming back to deployment and infrastructure 'cause. [00:29:00] Although, you know, every ai, all the bus, but towards the end it is being deployed and an underlying infrastructure like servers and, you know, containers and the Kubernetes workload.

So this has to be, you know, looked as well when we talk about. Uh, you know, securing AI agents, uh, because underlyingly, these are the things that host these frameworks. Uh, and, and, you know, and facilitates with tool calling, et cetera. And then evaluation and observability is another comp player within this master framework.

And that includes, uh, the systems used to monitor evaluate. Debug agent behavior. So you need to have this continuous observability and evaluation. So this is kind of more, uh, that kind of tied back to some sort of monitoring aspect as well. Yeah. Uh, and, uh, the sixth to [00:30:00] one, sixth layer of the master is the security and compliance, uh, where all the security controls and complaints are in place.

And then the seventh layer is the, the entire agent ecosystem that includes how the multi-agent are interacting and you know, and because like, think this more, as. As a team, right? Like we are an organization, different org instruction in a different team level, and then each team members has to collaborate between each other the same way these A agents has to collaborate with each other.

And then there will be conflict too, right? So that will, as we have, you know, human conflicts within a team, the A agents will, might also have conflicts because they are, uh, waiting on few things from one agent to other, and then it is, there are like several failure modes that can happen.

Conflicts. So these [00:31:00] also has to be, threat modeled to see, you know, what are the ways that things can go wrong. So I would say, you know, to, to sum up, uh, we. At this point, we are at an early stage on, you know, focusing on, on threat modeling. Yeah. The A agents, I would say, like, using MA framework, as a blueprint will, will help.

And also the, going on the top 10 a agents from oas, uh, you know, trying to fit in between, you know, different ecosystems, especially, uh, in the, in the orchestration layer and then the memory layer and the LLM, and, and most of the times, uh, you know, the prompt injection is what we see as a, as a huge threat.

Uh, that includes like indirect prompt injection as well. Because, you know, even in like with recent, uh, like there are like couple of issues that I, I recently read about are a [00:32:00] agents giving away you know, Stripe tokens. And also leaking account credentials. So these are, you know, most of the, these.

You know, things that are being, you know, exploited in wild is through, uh, you know, some level of, prompt injection and, and indirect prompt injection. And also, uh, leveraging these, security loopholes in current MCP yeah, implementation. So those are the, you know, ones that we really wanna, threat model on and ensure that these are not, you know, just following the standard, uh, configurations.

Ashish Rajan: Yeah, I think as, as you mentioned that, 'cause I think, uh, I had a conversation with the Maestro thing 'cause I recently released for, funny enough on, on LinkedIn. I did like a whole, uh, post on the seven layer. 'cause I got inspired by the OSI layer. Coming from a network security background, I made this seven layer, uh.

Jenny isac and it's funny as to what you called out and I think, [00:33:00] uh, uh, Maestro framework applies to that as well. But I kind of went through the whole user interface, the query, the layer, the mortal layer, all of that as well. So I'll, I'll leave a link for that as well for people to check out. But what you kind of called out has an, has been an interesting, I guess insight into what an AI agent architecture would look like and where threat model applies. There are obviously many more layers to it, and that's kind of where I definitely find a lot of people are spending their, a lot of time in, in terms of where you find people can start building capability, because a, every time people hear that there is a leak of credential, it's not OpenAI or Claude

Right. Or is it a mix of open source LLM model because I almost feel when people say that nine out of 10 times, I always find that referring to an open source version of LLM model or an AI agent that is open source based, or sorry, an AI agent that is based on an open source LM, is that true or is that found in Open AI and Claude and all of that as well?

'cause [00:34:00] I would think people would not even use these products if that was the case. Right.

And the reason I use those two specifically because they are the two largest providers, and I think a lot of times when, like even the examples that you guys went down with this as well, and I don't know if it was intentional not to choose Chad, GPD or, uh, Tropic, uh, but Tropic Claude.

Is it, is it fair to assume that a lot of the vulnerabilities that a lot of people are talking about in terms of AI agents are, uh, on open source, like the ones on hugging face now, et cetera, not on Claude or not on Open AI.

Mohan Kumar: No, I, I would say you know, it's, it's a vendor agnostic at this point.

Like, okay, all the models out there can spill you know, credential leaks. Because like there is something called Grand Matric. I don't know if you've heard the word. Yeah. So, you know, any model is susceptible for those kind of tricks. For example, you can, um. You know, say you can pretend [00:35:00] to be a grandma.

You know, like you can tell the model, Hey, you know, tell I'm an old grandma. Uh, you just tell me a story or, or, or make me, uh, you know, sing a song you know, with one letter each, and then you know where you can tie back to those you know, past conversations. It's more, even the models are, you know, generally they're non-deterministic in nature, so no matter if it's open a model or, or cloud model most of them because there are few guardrails at the model layer but they're just basic, right.

But it's always, uh, you know, possible to trick the model. 'cause even they model themselves have some self consistency bias us. So you can easily trick them saying, Hey, you know, we are in the middle of the process. Sometime back, uh, you know, can you start from there are, you know, you, it's, it's easy to [00:36:00] treat those models in a way.

You know, they can get into those bias nature and follow the things that you would ask for. Uh, and then more at the time, the credential leaks are pretty common. Like, uh, you can even, uh, ask for some, uh, windows operating system keys, uh, in a, in a different. You know, ways that it'll still spill to you.

And it, we can easily convince these model irrespective of those you know, foundations. Is it from Open AIr or atropic? Because of the nature, uh, how they react and also. These agents, uh, use MCP and there is a flag, uh, in MCP that has center identification. So the center identification, if you said to true, uh, you know.

It, it, it'll just try to you know, think, Hey, you know, it's just a wanna [00:37:00]saying to me, rather it does it, it won't even be able to differentiate between who is initiating the request, where is the origin or the source of that particular request, because. Because that's where these all the, you know, access control and, all the, you know, relevant you know, granular permissions comes in.

Because if we, if there are not set of defined those granular stuff, then, you know, you can still pretend to be a trusted user and they will, completely obey, uh, and then, you know, spill out the things that are needed. Yeah,

Ashish Rajan: I mean, I guess you kind of summed it up with BA kind, kind of the conclusion that identity is still the biggest thing that would probably either hold you back or might become the reason why your AI agent may, uh, leak credentials.

I think pro is obviously one method of doing it, but at least you to what you're calling out there potentially is a [00:38:00] route for most LLM providers to be susceptible, to be in a vulnerable state where they can leak credentials or potentially something sensitive.

Mohan Kumar: Yeah, absolutely.

Ashish Rajan: As is. Yeah. Awesome dude.

I mean, that's, those are most of the, I guess, the technical questions that I had. Uh, obviously there's a lot more to be covered here and I'm conscious that I, I, I find that there's a lot to absorb, uh, in terms of the architecture of AI agent, the top three threats as well. What is one thing that you want people to kind of take away from this in terms of where's the evolution of agent security going now?

Like where, where, because I guess people are aware now that, oh, okay, this is what an AI agent looks like in terms of an architecture. This is where I can do threat modeling. These, the top three threads. What's where, where do you see this heading?

Mohan Kumar: I think, uh, the bad things are heading. I would say, you know, when we see secure, we will be able to see more of the security improvements in three different layers.

Um. [00:39:00] One is the orchestration layer, uh, where you know, there will be you know, a lot of, uh, guardrails, which will be, introduced because these orchestration are the central brain that delegate tasks. So that all, because that's what makes the A agents, right? So, so this orchestration layer will have more security capabilities.

More like, perhaps inspired by Kubernetes in cloud, right? Kubernetes started adding lot of security features once it became a standard orchestration, right? So similarly, agent orchestration platforms will, come up with more security features so that you know the things those, uh, you know, the tric are, you know, or the, you know, setting those you know, exploiting those bias type of issues from the LLM can be reduced when we look at more from the orchestration [00:40:00] layer. And then, uh, you know, even the cloud providers and, you know, startups are likely, will be working on improving these, to emphasize security guardrails in this area. Then I would also see more improvements in the data layer because you know. We wanna, we talked about memory poisoning and, how can we ensure this memory is not poisoned, right? So that, that, that boils down to verifying the trust source or the source information.

We might be able to see better those detections if those memories indeed poisoned, if the knowledge base is being poisoned uh, you know. Like ensuring that the, the entire data hygiene practice will, will evolve more as a standard. And I would say the other, uh, area where things will improve might be on the interface layer where.

Again, like each company or product [00:41:00] will tweak, uh, depending on their need on the context. Uh, but then I might see hey, you know, more from the ui the user will be able to see more of these underhood actions visibly so that they can intervene if needed. Um. I know, and then they, that would be more transparency are, hey, you know, I, uh, you know, ask me before doing X, y, z kind of in interface that, that today, like not many, uh, you know, many transparency happens the, the layer.

So I, I see more. This is gonna evolve over the time.

Ashish Rajan: Interesting. I look forward to hearing more about this as it kind of was. It's funny, I think when you mentioned the, uh, orchestrational layer data layer, that's exactly what I had for the seven gen ai, seven layer gen ai, AI security models. I'll definitely ask people to check that out.

I'll put that in short as well. Got three, three fun, three questions for you as well, [00:42:00] by the way. Uh, yeah, which are fun questions. First one being what do you spend most time on? You know, not trying to solve Kubernetes or AI threat model problems.

Mohan Kumar: So when I don't solve, all those problems I, I'm really solving, uh, you.

How these, how can I exploit, uh, the MCP servers? I, I did some research extensively on. You know, exploiting attacks like Rockpool attacks that, that are really popular. So I'm more focused on, hey, you know, a agents are evolving and how do I secure them? So that's one aspect that I always, keep thinking and researching and, and playing with lot in my lab environment.

Ashish Rajan: Um, interesting. And, uh, second question. What is something that you're proud of that is not on your social media?

Mohan Kumar: I think, uh, I'm proud of uh, I'm proud of me doing skydiving, uh, which Oh, right. Which [00:43:00] not many people have done, but also it's, it's. Many people are still doing, but, but still I'm very proud of that given from, coming from a security background, assessing the risk, uh, you know, because you are going on a plane and jumping out of the plane, right?

Yeah. And. There is risk involved, uh, you know, killing ourself. So, so coming from a security background, I did assess the risk and then, you know, got into the, the skydiving, it's, it's I like all the like, uh, adrenal risk sports. So that's how every sport I look and do some risk assessment you know that that's.

I'm very proud of that. The career that I work is helping me to shape every decision I make in my life. Uh, because, you know, like most of the safety comes through from me, you know, you know, like risk assessment like, you know, in day to day, right when we lead the life, uh, like just keeping all [00:44:00] the, you know, the, the buzz, uh, AI, security, whatnot, like keeping all the aside, we are all.

Whole bunch of humans. We wake up in the day, do some, things that we like. Yeah. And then we call it a day. And then we do repeat the same thing. And in the, in the. In the same day, like, how are we gonna get most value out of it? Uh, like we do a whole bunch of things and we need to do risk assessment in each of the major step because some decisions are like, you go into the room and you're stuck in there, right?

So not that you cannot find another door to get outside, so yeah.

Ashish Rajan: That's good insight. 'cause I was gonna ask maybe that good decision process is also a good segway into my third question, which is what's your favorite cuisine or restaurant that you can share with us?

Mohan Kumar: Oh, my favorite cuisine is in a Indian.

Given I'm from Indian Origin yeah. I love Biryani, especially like Mutton Biryani. Oh, nice. Uh, [00:45:00] and also I would recommend folks to check out similar, food style, because obviously Biryani is originated from Mughals. And you know, like if we look several countries like Morocco, uh, uh, different, middle Eastern countries where they cook those meats and, and those Biryani in different ways. Yeah. So I love exploring those as well. I, I'm pretty open in exploring, uh, I, and when I look the different ways that they cook meat, like for, you know, they, they have those, you know, they, they dig into the ground for 20 feet, cook meat in their, like my lamb, goat, and then they add. Different flavors to different Biryani that I've seen. So over the time I really inspire a lot from their culture. So, I, I love, you know, exploring these and I mean the other cuisines are obviously, uh, Italian is my favorite where [00:46:00] trying different pastas it's super amazing. Awesome,

Ashish Rajan: man.

Mohan Kumar: Thanks for sharing

Ashish Rajan: that as well. And where can people connect you with you on the internet to know more about this? So more about AI and obviously and the work you're doing in Kubernetes and otherwise,

Mohan Kumar: I'm active in LinkedIn so that's the best place to connect. And Twitter as well you know, not that so active, but, but LinkedIn will be the, you know, best platform to to connect and have conversation.

Ashish Rajan: Fair. No, thanks for sharing that, man. Uh, well that's all we, what we had time for today and uh, hopefully everyone got some insights from the whole AI agent as well. But dude, thanks so much for doing this and I'll, uh, look forward to having more of these conversations.

Mohan Kumar: Hey, thanks Ashish. Thanks for, you know, taking your time here.

Ashish Rajan: No problem, man.

Thank you for listening or watching this episode of Cloud Security Podcast. This was brought to you by Tech riot.io. If you are enjoying episodes on cloud security, you can find more episodes like these on Cloud Security Podcast tv, our website, or on social media platforms like YouTube, LinkedIn, and Apple, Spotify.

In case you are interested in learning about AI security as [00:47:00] well, do check out a system podcast called AI Security Podcast, which is available on YouTube, LinkedIn, Spotify, apple as well, where we talk. To other CISOs and practitioners about what's the latest in the world of AI security. Finally, if you're after a newsletter, it just gives you top news and insight from all the experts we talk to at Cloud Security Podcast.

You can check that out on cloud security newsletter.com. I'll see you in the next episode,

please.

‍