Attack Path Analysis for Better Kubernetes Security

View Show Notes and Transcript

Kubernetes security cannot just be Kubernetes but it is like security of a datacenter within another datacenter. In this episode with Tim Miller we spoke about CNAPP, how to approach kubernetes security. This episode was recorded at Kubecon + CloudNativeCon North America 2023

Questions asked:
00:00 Introduction
02:42 A bit about Tim Miller
03:35 What is CNAPP?
04:30 Traditional Kubernetes Security
05:18 Where to put a CNAPP?
06:20 CSPM vs CNAPP
09:00 Attack Path Analysis
11:05 Kubernetes Attack Path
12:43 The team you need
14:06 Resources to learn more
16:24 Fun Question

Thank you to our episode sponsor ⁠⁠Outshift by Cisco⁠ -

Tim Miller: [00:00:00] The whole point of that, of a CNAPP is to have multiple aspects of security related to your cloud native app. I really view CNAPP as being a solution approach where you're covering multiple different domains of security around your cloud native app, which, Gartner's definition of CNAPP is workload protection, identity and entitlement management, cloud security posture management, if I'm trying to remember all the C's, because there's a lot of C's in there.

So it's all about all these different domains of security that you have to be aware of.

Ashish Rajan: You probably would have heard of CNAPP. You probably would have heard of Kubernetes security, but you would not realize that everything they talk about in CNAPP and everything they think Kubernetes security has. There is a gap there and specifically in the context of how CNAPP is relevant from a Kubernetes security perspective, what it can cover, what are some of the threats you might have to look into if you are thinking about deploying a CNAPP in your Kubernetes environment, what you can expect, what's an example of an attack path that can be shared, which you might have to consider if you are thinking about CNAPP.

And does everyone really need a CNAPP? In this [00:01:00] episode, we had Tim Miller from Panoptica and he shared his perspective on CNAPP. And what does that really mean to have a CNAPP in a Kubernetes environment? What you should consider if you are thinking of looking into CNAPP as something that you want in your environment.

If you know a friend or colleague who's also working on the CNAPP space and probably looking at Kubernetes, definitely share this episode with them. If this is your second, third, or maybe even 10th, or maybe 50th episode that you're listening to of Cloud Security Podcast, or maybe watching on YouTube channel, And you have been finding us valuable.

I would really appreciate if you could take a few moments to drop us a review or rating on your popular podcast platform like iTunes or Spotify. That is if you're listening to this, if you are watching this on YouTube or LinkedIn, definitely give us a follow or subscribe. It definitely helps us spread the word and let other people know as well that we have a community that we would love to welcome them into.

We are a growing community of about 50, 000 people so far. So we would love to keep growing that and keep spreading the good message of cloud security and how to do that this was a conversation we had at KubeCon North America, where we were a couple of days ago. And thank you to everyone who came in and said hello and took pictures with us and took [00:02:00] videos with us and was kind enough to come on my video of the LinkedIn videos that I post for my daily vlogs for conferences that we attend.

Now, the next conference we're attending is AWS re:Invent. And if you would be there as well, I would love for you to come and say hello and reach out maybe if you are available. So if you give me some heads up, I should definitely be able to make some time to meet you at re:Invent. And I look forward to taking more pictures and videos with you folks.

When I see you at AWS re:invent, I'm always grateful when you folks give me hugs, say hello and say thank you for all the work we do. It really means a lot. So thank you. Thank you. Thank you for everyone who came and said hello to us at KubeCon and can't wait to say hello and hug more people at AWS re:invent as well.

If you are there, definitely reach out. I hope you enjoy this episode. I'll see you in the next one. Could you tell the audience a bit about yourself.

Tim Miller: Sure. I am right now currently a technical marketing engineer at Outshift by Cisco. So I work on our CNAPP solution Panoptica. I go way back in terms of the Linux community.

I started playing it with it in college. Red Hat Linux. [00:03:00] Not Enterprise Linux, but the Red Hat Linux before that, and I've done a lot of interesting things from high performance computing in academia to being a data center networking guy, then running an entire network for a university.

Yeah, and that's all before I joined Cisco, and then I just settled down on just doing sales. Wow. From a specialist perspective.

Ashish Rajan: You worked on HPCs as well, that's pretty awesome. Yeah. But in all the HPC, you didn't share your name. So what's your name?

Tim Miller: That's a secret. Some of those HPC environments were at National Labs.

Yeah, I was gonna say

Ashish Rajan: is that why you didn't share your name?

Tim Miller: No, sorry about that. Yeah. Tim Miller. Tim Miller.

Ashish Rajan: Nice to meet you, Tim. Yeah. So first of all, I want to start because we are at KubeCon and one of the C words from Gartner that's floating around the moment is CNAPP. How do you describe CNAPP?

For people who probably have never heard that.

Tim Miller: Probably the easiest way to think of CNAPP is one of the P's in CNAPP. It's Cloud Native Application Protection Platform. And the whole point of that, of a CNAPP, is to have multiple aspects of security related to your [00:04:00] cloud native app. I really view CNAPP as being a solution approach.

Where you're covering multiple different domains of security around your cloud native app, which, Gartner's definition of CNAPP is workload protection, identity and entitlement management, cloud security, posture management, if I'm trying to remember all the C's, because there's a lot of C's in there.

So it's all about all these different domains of security that you have to be aware of. Yeah. But to have in one place and to draw. Context, meaning and intelligence from all that being in one place,

Ashish Rajan: Maybe it's also a good point to talk about how do people normally approach security? Because I think obviously there's one part is, hey, I'm going to install CNAPP and all my Kubernetes problems resolved.

And then there is the actual problem of how do you even look at security in Kubernetes? How do people look at traditionally at the moment into this space?

Tim Miller: Yeah, traditionally they start off with the basics, right? They start off with container security. With the executive order from the White House, right?

A software bill of materials, supply chain security. Of course that [00:05:00] put it big on everybody's radar. Now I have to secure my containers. I have to get an SBOM. What is this SBOM thing? And, oh, this SBOM can give me vulnerability information. So they're just really getting their heads around all of this to understand where they could have vulnerabilities come into their own software.

So yeah, that's where they start at.

Ashish Rajan: And is that normally a place where you would put a CNAPP?

Tim Miller: No, not really. That does become part of the CNAPP solution, right? All that information can flow into a CNAPP, but a lot of folks don't start there. They just start either in one or a few places, because it's a mere mature technology.

Yeah. Generating SBOMs, generating the vulnerabilities from all the databases. Everybody's got it, including where the developers live. The GitLabs of the world. You have those actions and runners that... We'll generate that stuff for you. So it doesn't really start off with the CNAPP. CNAPP usually comes into play when the next buzzword hits them.

Ashish Rajan: Hey, I'm missing that. I should probably get that one of those. The reason I ask is also because a lot of people may come to a KubeCon or [00:06:00] may come to a Kubernetes security conversation and just simply assume that oh, all the CNAPP conversations, I need a CNAPP in my life. Otherwise, I would not be able to do this.

Clearly to what he said as well, that's a more mature, once you have, if you have one cluster, you don't really need a CNAPP to secure one cluster. Once you start growing, start having production workload and there's a little bit more maturity that comes in. Another term that comes in around that same time, which is confusing for people, is CSPM.

What is that and what does that have to do with CNAPP?

Tim Miller: That particular acronym is where CNAPP starts showing up in customer environments. Cloud security posture management is really about looking at your cloud and doing a security posture management. Yes, it's very difficult and technical.

It's complicated. But yeah, it's about when you put those workloads into a cloud. And that can be lift and shift, right? I've got that cloud first, cloud only mentality. Or, I'm refactoring my application in a microservice based way, right? Either way, you're deploying resources in a cloud and [00:07:00] you've got to figure out, am I doing it securely?

Because, speaking of Gartner, which we all have to do, 99 percent of security incidents in the cloud are misconfiguration related. You've got to collect the complete inventory of your cloud environment and then do the security assessment of all those configuration settings, all those assets. And that is by definition what CSPM does.

Ashish Rajan: Yeah, okay, because I guess maybe another way to put this also is the fact that a lot of people listening to this conversation may have come to KubeCon. They're building workloads in Kubernetes at the moment. Yeah. And they're automatically being. Okay, if you want to do this at scale, you need one of these, right?

Where should they start considering that? Hey, are you at a CSPM stage or are you at a CNAPP stage? Like a lot of people may just be confused by the fact that, oh, that's great. So Cloud Security Posture Manager, should I take that first or should I take CNAPP first or should I take just the CNAPP?

Tim Miller: I usually would recommend meet them where they're at, right?

So if they're primarily [00:08:00] just doing Kubernetes. Let's talk about workload protection, but at the same time, you still need to have that cloud posture management assessment that goes with it. So if you are putting workloads in a cloud, you really do need a CNAPP. The question then becomes, do I need all the parts of a CNAPP, right?

Don't try and the analogy of how do you eat an elephant, right? One bite at a time. So don't go into deploying a CNAPP. With the intention of turning on all the knobs and widgets, right? Yep. So CNAPP, if you're in the cloud, you probably should start with a CNAPP because you'll have all the components there.

And then you start working through your environment in an intelligent way. So Kubernetes, we're at KubeCon, right? So Kubernetes workloads, let's turn on the workload protection. Let's get visibility into that. And then use the Kubernetes security posture management. There's another, but it's not C, it's K.

KSPM. KSPM, yep. Let's use the KSPM capabilities from CNAPP to assess how we're consuming that Kubernetes service from the [00:09:00] cloud provider.

Ashish Rajan: Typically, a lot of people look at network security. Typically, people look at, has an attacker gotten access to my network? Has access been provided by a misconfiguration?

We spoke about the whole attacker path analysis kind of thing. What is that? And I guess. How is that better than what we were doing traditionally?

Tim Miller: Sure, and looking at that network access is a great way to look at it. When I have a CSPM, it will show me a security group that has, I'll show my age, the IP NE, right?

Yeah. Yes, I do work for Cisco, I have been in a firewall in an iOS device. We'll see the fact that they don't have any access control. Okay, that's probably not good. Another red alert in my CSPM and CNAPP tool. And then I look at the EC2 instance, right? And I do a vulnerability scan on it. And, oh look, there's a hundred red things that are all CVSS scores of 9. 8, right? Yep. I've got all of these different parts and pieces from all the parts of the CNAPP platform [00:10:00] that are giving me tons of red. Which one do I fix first? Naively, I might think IP NENE is not a good idea, wide open access. But if it's my application I'm serving to my customers I have to do that.

Yeah. How about we look at it the way an attacker would look at it? All right, let's explore the external attack surface. So we do a scan of your environment, right? We see what's available. We use some publicly available tools to do a posture assessment of that external access. Discover potential vulnerabilities.

Oh, there's an EC2 instance. I get into that. Now I do what an attacker would do. Oh, look, there's all these other vulnerabilities. Oh, look at all this access I have from this EC2 instance. Let me escalate my privilege and go get that S3 bucket. Oh, I've got wide open access now. Why don't I just encrypt this and make a lot of money with ransomware?

That path through those assets, the security findings is an attack path.

Ashish Rajan: And that helps you prioritize as well, because at the moment, what you might look at as a wall of red, but in that [00:11:00] haystack, there's a needle that you probably want to pull first. That's a great way to say it. Yes, that's exactly right.

I appreciate that. I think also because I feel another place a lot of people would get confused would also be the fact that. What would an example kubernetes attack path look like? Because a lot of people talk about, to what you said, NE NE as well. That yes, it could be applicable. Are there like kubernetes specific ones that you can think of that could you could share?

Tim Miller: Sure, so we've got an application. We're exposing to the internet. So that would probably come through ELB. I'm gonna speak AWS Yes, a lot of folks are familiar with AWS. So we're exposing an IP address through elastic load balancer Yeah, because on the Kubernetes side we've used that as our service type, right?

Yep. And so that exposes a service slash workload, whether it's deployment. Ultimately, it becomes a pod, right? And so we've exposed that pod directly to the internet. Yep. So there could be our potential attack path, the ELB, the access with the security groups [00:12:00] through into that Kubernetes environment, and then the vulnerabilities that are on it.

That pod is a workload. We always talk about workload protection and compute protection, but really it's serving out data. Yeah. Everybody's gotta be concerned about the data and 'cause that's what the attackers want. And so ultimately that could have privileges to an RDS. , does it have just what it needs for a public facing service or did that pod get read-write access.

Yeah. And so that's leveraging, say, RDS and Amazon. And so that would be your attack path. It would be. Public Internet, ELB, goes to this Kubernetes service, goes to this deployment, this pod, this container image. Container image is probably a fork off that, but then from that pod out to RDS and there you go.

Ashish Rajan: Wow, and in terms of teams as well, I think maybe what they're calling out, every time I talk about, because you're an open source community and there's a Open source version of CNAPP is a paid version of CNAPP is a same for CSPM otherwise as well when people who are leaders in the organization thinking about, okay , [00:13:00] I have a lot of Kubernetes workload.

I need to start looking at this more seriously because now we're putting production items in their mission critical applications in there as well. What kind of teams should people consider having in their organization? Should people know Kubernetes in the first place? Buying CNAPP is enough.

Tim Miller: Depending on the CNAPP solution you look at, buying a CNAPP is often not enough because there's this assumption that the security teams understand the technologies that it's testing, right? And so you really, if you're going to go that road of choosing a CNAPP platform, you really need to have one that kind of explains things in security terms, but not the special kind of CKS, the certified Kubernetes security specialists, right?

You need to explain. Why this is a problem because we all know a security engineers about privilege escalation. We know about secrets. We know about leakage, data leakage, data loss prevention, those do exist in a Kubernetes cluster, the technologies down there and the combinations of them are, do they really need to know that?

Do they really [00:14:00] need that certification or do they just need to understand the risk so that they can then work with the right teams to fix it?

Ashish Rajan: Yeah. And, we have a final question, where can people learn about this space, this whole CNAPP, Kubernetes security? What do you advise people to start with? You've been in this space for a long time, since we spoke about the, showing our age, talking about NE.

How did you start learning about Kubernetes? What would you advise other people to do the same?

Tim Miller: Yeah, to learn about Kubernetes Believe it or not, it was kubernetes. io. But the docs on the Kubernetes website were fantastic. I did, at that time when I was really diving into this, KubeCon, and I think they still do have a virtual component to this.

Take advantage of the virtual option. We've got a lot of great content at the conference. And I found in my life, in my experience, Gunning for certifications is a great motivation for actually learning the capability. So that's how I got into Kubernetes. It was being interested in this space. I love complex problems because of my academic background.

So that was a [00:15:00] natural fit too. It was all about really managing containers and applications and developing these things in cool different ways. So there's lots of different angles you can come at learning Kubernetes. Its from the, I don't want to, because again, showing the age, I don't want to deal with package management on an OS and deploying that OS.

And so wrapping up an app in a nice clean container is appealing. Deploying it at scale, watching it auto scale out and back. If you've worked in IT, all of these things, which were very hard problems with real physical load balancers, firewalls, and that all just became so much easier. So if you're a serious IT engineer, at least in my opinion, you couldn't help but gravitate to some of this cool technology.

Ashish Rajan: I wonder how many people actually have any memory of what data centers looked like as well. Or the hot rooms that are air conditioned rooms, depending on which part of the a datacenter you were going into.

Tim Miller: It's we were just having a conversation with some friends last night, and it's the two extremes.

A lot [00:16:00] of us don't ever walk into a physical space anymore, but as we're talking AI, we're talking about the extreme datacenters you need now because of all the density of heat and power that needs to go into the GPUs that drive it. So you either never see a datacenter, Or it's now, hotter than the sun.

Ashish Rajan: No air conditioning in the world is going to solve that problem for you. Exactly right. Awesome. Now that was most of the questions I had. Cool. This is the fun section. I'm just going to get some boogies, literally, we'll find out. Alright, so three. For you, three for me. Oh, three. Yeah, so we basically, the way we're running this is basically, one question, oh my god.

Oh. So I got green ones, but we'll find out. So anyway, pick any. We'll have these first, share what you think it is, we'll talk about it towards the end, but

this is not bad. What's yours like? Mine is like a well, I won't say peas. It's like a sweet pea.

Tim Miller: Popcorn. That's what it is. It's got

Ashish Rajan: popcorn flavor. Popcorn flavor. Mine is not popcorn flavor. Mine is definitely, [00:17:00] Mine is sweet, has a bit of a cherry onto it as well. It's green in color, but good flavor.

First question. What do you spend time most of the time not working on this cloud native Kubernetes kind of thing?

Tim Miller: It's something that I did infrequently before, but after all the work from home and The home office, it's, I got to get out. So I garden, I actually raise tomatoes and green peppers.

I go hiking. As soon as KubeCon's done, I'm going camping. Oh, nice. Wow. Yeah, I'm from North Carolina, so we've got all four seasons there, and now it's starting to get a little chilly, so I got to get it in before it's too late.

Ashish Rajan: Fair enough. Thank you for sharing that. Alright, next one. I'm going to get the yellow one this time, not the green one.

It's it's weird. Oh, this is soap.

Tim Miller: Not like super sweet.

Ashish Rajan: Oh, you want a sweet?

Tim Miller: Yeah. I can't place the flavor, but it's got a slight sweet taste to it.

Ashish Rajan: Mine is definitely soap. I don't think I've ever had soap in my life, but...

If I had soap, I think this is what it would taste like. So second question, if you could have superpower, what would that be?

Tim Miller: Ooh, [00:18:00] superpower. Wow. I think I'm going to have to go with flash super speed. Ooh,

Ashish Rajan: go from point A to point B in a matter of a flash. Yep. That's a good one. Where would you want to go in a flash?

Tim Miller: Oh, everywhere. It's one thing that from my grad student days, I got to travel everywhere for conferences and working on experiments and stuff. So I got the travel bug, but In my current role, I get to travel a lot more, but there's been a long period of time where I just don't travel a lot, so I guess, to pick a location...

Camping site.

Ashish Rajan: Go to the camping site in a flash, before it snows.

Tim Miller: Yeah Grand Canyon would be great. I actually haven't been to the Canadian Rockies, so that would be fun too.

Ashish Rajan: Yeah, I've seen pictures of it, I've never been as well. That would be great, because you watch something and go...

I wonder what that's and just be able to go there in a flash. Yeah,

Tim Miller: and not having to be on a 5 hour or 6 hour plane flight would be good.

Ashish Rajan: Yeah, or 8 hours for us, would be good. Alright, last one. Alright. Oh my god, this is worse. Oh yep. Okay, this is like booger. It's been getting worse, I'm telling you, since morning.

Oh wait,

Tim Miller: okay, I think mine got better. Mine's marinara.

Ashish Rajan: [00:19:00] Mine is not getting any better. I'm like, oh my god. Gonna have some water after this. Last question. What's the best part about coming to KubeCon? I'm gonna cry. Like

Tim Miller: best part of coming to KubeCon? Yeah. Oh my goodness, meeting people. My opinion, the whole point of having conferences, yeah, learning is good, but it's really mostly about meeting people.

Hearing their experiences, both from a technology perspective, as well as, an industry, as well as an industry perspective. Yeah, it just started off so weird. It's yeah, a spaghetti sauce on a jelly bean. It's it's not as horrible as the three you got.

Ashish Rajan: Yeah, I might have been worse, but it's not even your thing.

Meeting people.

Tim Miller: Yeah, just meet in the community. There's a lot of very interesting people here, a lot of great ideas, and just being able to be exposed to new things, because we're all doing our day job, right? We're all tackling the problems in front of us, but we're just a small microcosm of what everybody else is doing.[00:20:00]

Tens of thousands of us experiencing 10, 000 different other problems.

Ashish Rajan: Yeah. Yeah. It's a good place to collectively have that information dissected. Exactly. Yeah. Thank you so much for sharing that. Where can people find you on the internet if you want to know more about CNAPP and what you do

Tim Miller: oh, okay. I'll start with my contact info first this time.

I'm at BroadcastStorm on X slash Twitter slash whatever we're going to call it. Oh, that's a good name, BroadcastStorm. I like that. At LinkedIn, I'm there at LinkedIn. There's, you might be surprised, there's a lot of Tim Millers. But if you look for Timothy E. Miller, After all the LinkedIn, I'm sure you've got like show notes or anything.

Yeah, I'll put the show notes as well. And of course, I work at Outshift by Cisco. So we've got a lot of content there. And I can't help but mention the product we work on, which of course is Panoptica. app. That is, we've got a lot of great information about the industry, the challenges there. So there's helpful information as well as things to learn more about our product.

And also we'll be at AWS re:invent . So we've got a booth for Outshift. And so you can meet me, several of my colleagues, as well as [00:21:00] see what we're doing that's interesting in that space.

Ashish Rajan: Would all of them get boogies? Oh,

Tim Miller: I think I'll hold off on that. They've got enough challenges with cloud native security as it is.

Ashish Rajan: But I appreciate you coming, man. I thank you so much for coming. You bet. Thanks.