Kubernetes Runtime Threat Detection and Response – Falco, Sysdig

View Show Notes and Transcript

Episode Description

What We Discuss with Dan “POP“ Papandrea:

  • 00:00 Intro
  • 03:20 What is Kubernetes Security?
  • 05:30 Why there is a need for runtime security?
  • 07:41 Where does Falco fit in?
  • 10:05 Commons Threats in Kubernetes
  • 10:58 What is Falco?
  • 14:26 Attacks relevant for Kubernetes Run Time?
  • 16:32 What’s an ideal production environment?
  • 19:24 Is there a skills gap in Container security?
  • 21:12 Difference in Managed vs Unmanaged Kubernetes?
  • 23:10 Do I need Kubernetes for all use cases?
  • 25:10 Forensics for compromised Kubernetes
  • 28:50 Why the POP Podcast?
  • 31:26 What is CNCF Ambassador
  • 34:49 Kubernetes in a diverse environment
  • 38:56 Fun Section

THANKS, Dan “POP“ Papandrea!

If you enjoyed this session with Dan “POP“ Papandrea, let him know by clicking on the link below and sending her a quick shout out at Twitter:

Click here to thank Dan “POP“ Papandrea at Twitter!

Click here to let Ashish know about your number one takeaway from this episode!

And if you want us to answer your questions on one of our upcoming weekly Feedback Friday episodes, drop us a line at ashish@kaizenteq.com.

Resources from This Episode:

[00:00:00] Hey and cloud security podcast listeners. Thanks for getting it. Sonya. It’s the opportunity sponsor of the show. If somebody does exactly three things like connecting to existing data sources, assuming this gives customers a comprehensive asset inventory, both cloud, and on-prem it. Then uncover security apps and finally it automatically validates and enforces policies.

Thanks again, and check us out.

Ashish Rajan: Hey, Dan, thanks for coming in, man,

Dan “POP” Papandrea: Thanks for having me Ashish. This is awesome. I’m a fan. I saw you with one of my buddies, Mark Manning and Magno. I mean, so like really amazing, amazing stuff you’re doing. So,

Ashish Rajan: dude, the pleasure is all mine. The pleasure is definitely all mine to come over here. And make new friends online . . That’s the best part. I just want to get it straight into this as well. Cause I definitely feel a lot of people are quite keen to understand the space.

So before we go into it, who is Dan and what’s for the “POP” man.

Dan “POP” Papandrea: So my name is Dan Papandrea. So “POP”, like I’ve been called “POP” since I was a little kid. So I’m the director of open source ecosystem and community , for SysDig. So part of the Falco [00:01:00] project, which is a to CNCF incubated project for like runtime, behavioral monitoring, analysis and detection.

Prior to that, I was a field CTO with, they can, prior to that, I was on the OpenStack Helion project with HPE. And prior to that, I was a director of it for software company managing, you know, the network and security for an investment portfolio company called SSNC or advent software.

Ashish Rajan: That’s pretty awesome, man.

So primarily you’ve been quite close to the open source community for some time, then that’s pretty awesome to hear.

Dan “POP” Papandrea: Yeah, it was a contributor to OpenStack prior to this. And that’s like where I found my love of containers and the initial stages Kubernetes was, you know, being in that ecosystem. And I’ll tell you like, again, CNCF and, and in Kubernetes is the greatest community in the world.

And this is why I hope your, your listeners are keen on this from an open source perspective, like security to me, the future of security is open source. Like having, like being able to detect actors and, and be able to detect. Like things that are happening from runtime [00:02:00] perspective, not only for in general, but be able to have this overall security posture.

You need to think about open source first.

Ashish Rajan: Interesting since we are on the topic. What is Kubernetes and What does Kubernetes Security mean for you?

Dan “POP” Papandrea: So to me, Kubernetes is open source container ecosystem, right?

That manages it, scales it. Originally obviously the design by. Our friends at Google. So that was, you know, Borg. And they’ve also been on shameless plug podcasts. They’ve been on my show, you know, in terms of the original godfathers, the Kubernetes. But to me, it was like, you think about that Nirvana.

That was OpenStack or infrastructure as a service or SAAS or platform as a service to me, that Nirvana is found with Kubernetes. You have the ability to scale up these containers, which as you know, are these atomic units for being able to run your code in a way that’s, you know, might run less than five minutes, but still have the capability for you to, you know, run your applications and all of those things.

So that to me [00:03:00] is what Kubernetes is. And then you asked about Kubernetes security. Well guess what out of the box. Not a lot of options for you from a Kubernetes perspective, you’re not going to get the things you would expect. Like there’s not going to be, you know you know, protection from a runtime perspective.

It’s not gonna be protection from a vulnerability management perspective. You said earlier about drift detection, all of those things, you don’t have those things out of the box. However, You know, there are things that are protecting you like pod security policies and those things, which are they’re deprecate at this point, but they’re moving on to different things like, you know, OPA or Kyverno or other projects in the ecosystem.

So to me, you know, Kubernetes security is the elements that are making up every part of this. Managed orchestration system. So like the underlying OSS, the nodes and all those things. So if you need a, you know, Kubernetes secure posture, you need to think about all of those elements.

Ashish Rajan: Interesting.

So now, since you’re on the Kubernetes security posture, it’s an open source and it’s obviously not there out of the box, but what, why is there a need [00:04:00] for something at a runtime? Like, as in, I always thought that I was going by the previous job, sort of you’ve done that.

Or Kubernetes, you define something it’s it’s job is to maintain a state. Right. I mean, I thought that’s the job. So why is it changing in runtime? Why is there a need for having something like an insecurity runtime and it just maintains the state.

Dan “POP” Papandrea: So, I mean, again, you think about the reactive aspects of it.

You think of this pipeline, Hey, I’m going to deploy this pod out to this orchestration system called Kubernetes on various. Clouds. It could be, AWS could be, Google could be my, my private cloud. And from that perspective, it’s like, I need to protect all of those elements. I need to understand what’s happening at runtime.

And so if you look at it this way, Hey, I’ve deployed XYZ application from a, maybe a secure software supply chain of some sort, God willing. And you hear that, that buzzword. And what have you, because you put, you don’t have that protection. And so what I look at, and again, when I mentioned Falco again, a CNCF incubator project, and you’ll see that little logo here.

There’s a reason [00:05:00] why I love this project and love the community around it, because this is a capability out of the box that allows you to tap into the kernel, to be able to see all of those abnormal things that happen even after you’ve deployed this code. Like you said, it’s managing those applications, but is it’s truly securing you.

It really isn’t. And so from this perspective, creating rule sets that say, this is a deviation from what happened, drift, like you said, Hey, somebody added an exec permission to this. Maybe I need to understand what’s on somebody’s catting, the se shadow on the thing on the I think the K8s API is touching something.

It shouldn’t be touching. I need to know what’s going on. And so from that perspective, that’s the reason why you need runtime protection.

Ashish Rajan: Ah, so wait, would this be people who are, I guess, They find out today that they have Kubernetes in their environment and going, holy shit, this like this little lot of humanity clusters here.

And because what we’ve spoken about so far has always been about, Hey, you should do admission controller OPA and all [00:06:00] these things that you kind of define it should. The assumption is it’ll make sure that you don’t, you deploy a good, I guess, Kubernetes Cluster which is safe enough, but this sounds more like, what if you already have, like, the Pandora’s box is already open and you’ve gone.

Okay. I need to find some kind of sense to what my security or how bad security for me , in production. Would that be the kind of situation where Falco fit in or is it most of the more if you’re starting today? Like how else would this be kind of fitting in.

Dan “POP” Papandrea: So if you think about back in the day and I’ll mention Wireshark, right.

And everybody’s used it TCP dump, right? You had your switches, you had your network, you know, NICS that were communicated, but you didn’t know exactly what was going on. You didn’t. We were unable to filter by specific details that were going on. That to me is use case and why I mentioned Wireshark, a TCP dump our founder. Of Sysdig. That’s also one of the the original creators of Falco. Right. So like there’s that, that lineage there. And then if you [00:07:00] tap into these functions, I’m sure you’ve heard the term, a kernel module or here, here at E BPF. Yeah. And I know, Manning was on, he talked a little bit about that.

That’s true visibility because, because again, you have , a orchestration system like Kubernetes, you’re not just protecting those applications or those networks you’re protecting everything. So that means the node, the operating system, that’s running it, the Kubernetes elements that are run of the control plan that’s running on top of it.

You may also need the, the underlying cloud provider aspects of it. You may also want to look into Kubernetes audit log to say, okay, I shouldn’t have been running this. Namespace. He shouldn’t have been able to create that. I want to know what’s going on and maybe I want to react to it. So the things you mentioned earlier, like OPA OPA is a fantastic tool and, various end-users are using OPA and Falco and other tools in this amazing ecosystem that we have.

In the CNCF to be able to have that protection. But what OPA is giving you is that like, kind of almost like a security policy kind of [00:08:00] engine, that’s kind of providing you like a scripting language, something called Rego. That’s giving you that capability to say, okay, this is what I will allow and disallow from like a pod security policy or nodes security policy perspective or, or, you know, overall Kubernetes DSL.

Ashish Rajan: All right. And since you’re on that topic. Okay. So we found out that I guess that there’s a bit of a skill conversation as well there. Okay. I need to be okay to be getting your hands dirty with the tool and everything. Right. So some people might be going a bit, Ooh, I don’t know. Maybe I’ll start with, what are some of the common Threats I need to look out for before I make a decision on going on a tool?

What do you recommend as some of the a low-hanging fruit people can be going for there to look at drift detecion? If that’s your thing.

Dan “POP” Papandrea: I mean, look, if you go to falco.org right now, and you go into the docs page, the first thing you’ll see is privileged escalation. That’s the number one thing, somebody, you know, going in.

And if you think about drift in general, what is it? I want to be able to say, like Ashish went to here, [00:09:00] he made himself the God, and he went in and added whatever he wanted and then came out of it. So, I mean recently the community wrote like a series of blogs called this file correspondence engine, basically, where we have this thing, it’s a called Falco sidekick.

And it’s basically, you know, it has that rule set that I told you, like, let me explain what Falco is to everyone.

Ashish Rajan: Well, actually I see a good grade to start that. Yes, let’s do that.

Dan “POP” Papandrea: Yeah. So basically Falco taps into use a system called so taps into something called the BPF or like the kernel module kernel itself to get these system calls.

And then you have a rules. Ascertation like rule sets there Yammel that you can go in and say, this is a violation of what happened. So something like, again, somebody doing a privilege, escalation you know, something like you, you know, mutating login capabilities, somebody like, excuse me, going into the bin at sea.

So many going into some of the underlined cube elements as well. And so then having it output to something like standard out assist log. [00:10:00] You, you know, and those types of things, or we Falco psychic is from an architecture perspective is again, remember I told you that core engine, there’s the rules there’s outputs, and you can send that out to this sidecar.

That’s basically now taking this data and sending it out to something like slack PagerDuty or, or, and this is what we did with response engine. Maybe I want to send it to Lambda to do something like take a look, take a mem dump. If this thing happens. And so the possibilities are endless. So here’s what I’m saying to you.

This is why, remember I said the power of open, the, the future of security. What security should be is incorporating open source, because what I can do with that is now I have this capability to create a bunch of rules for whatever the ails the patient, by the way, we provide 120 out of the box. So there’s, these are best practices built on MITRE attack framework, and some other things like CVEs for, you know, from those perspectives.

And then the other [00:11:00] thing we do again, is on top of that is then you can also create those functions and that’s that response engine set up blogs. If you go right now at the falco.org/blog, you’ll see response engine. So we have K native as an example, and that’s an engine for serverless. And so this might seem daunting to you.

Security experts. They’re like, wow, how do I get started? It’s simple. Spin up a mini cube, K theory ass or something like this, then all you have to do is it’s a helm chart, simple one-line helm chart, you run it and you have all of the elements there and you have a snazzy UI to see exactly like triggered some of these rules.

It’s right there. It’s all in that, in that blog, it’ll be right there. Step-by-step you know how I know that? I had to validate all of those blocks and team Madden goes in here. He did some contribution is about, he’s a great member of the community as well. So that’s what I love about is the community has been,

Ashish Rajan: I’ll tell you this.

I can definitely feel the passion that you have for opensourcing security and getting more security people like except open source as a [00:12:00] new future for security. They’re definitely, I’ll give you that much. And I’ll also say this, that a lot of people listening in going. Okay. I want to go onto that. Cause Falco is open source as well.

So I definitely also wanted to add the fact that getting your hands dirty, people should not be afraid of getting their hands dirty insecurity, especially in cloud native world. And that’s where people seem to be going even more. The more you’re going to containers, Kubernetes serverless Lambda, that’s all code in a way.

The. Security industry, at least some parts of it has to move towards the direction of being able to code as well. So you’re able to kind of create things such as specific for yourself and keeping that in mind, you mentioned a ATT&CK MITRE and I know MAGNO spoke about this last time as well, what are some of the specific Attack relevant for runtime?

Dan “POP” Papandrea: I mean, there’s a ton in terms of like, you know, privileged escalation, the visibility aspect of it and those types of things. But again, like the MITRE attack framework, there’s those blogs that were written on those [00:13:00] specifically, like persistence privilege, escalation credential, access discovery, lateral movement.

But the beauty of this, and I know there’s a new iteration of MITRE. The beauty of this is because of the rule set. The Falco syntax rule set is based on this really easy, you know, syntax for a DSL (it’s YAML). All right. You can go in and update and say, okay, this should be, you know, I need you to always look at/etc or /bin or whatever the thing needs to be.

And so you can update those rules. So again, the re the proactive aspect of this, right. If a CVE comes out and there was one recently for Kube API, right. And were able to like create rule sets all based on contribution from the community. So this is the whole point of this security people need to know from an open source perspective, they don’t have to depend on a closed source company for them to adjust to these external hackers that every single day are attacking every single element of not only [00:14:00] Kubernetes.

But also underlying nodes, underlying clouds as three buckets. You know what I mean? All of that, that whole posture needs to be protected. And I believe Falco’s a rule set is the best out there. And I, and there’s a reason. At where I’m at right now is because I strongly believe in it.

Ashish Rajan: , that’s pretty awesome, man. I was trying to trying to think of like, oh my God, you have, so you’re so passionate about this. It’s come through. So, so well as well a lot of people and. I’m going to flip this around a bit more as well, because we’ve kind of spoken about threats. We kind of spoke about, okay.

Falco can help me run to do runtime security. What about what’s an ideal production environment, like, I’m thinking more for like, from a threat detection. Okay. I’ve realized today that there’s basically tons of humanities in my I guess , in my environment and I’m trying to go, okay, how do I do Thrat detection and maybe in forensic or like, how do I set that up?

What’s an ideal scenario for us to start. [00:15:00] Is someone looking at this today and going, okay. I don’t know where to just a lot of information. Clearly Dan seems to be like knowing a lot about this as well, and just really passionate about Falco, but I want to take a step back and how do I, like what’s my first step.

Dan “POP” Papandrea: I’m going to everybody watching this right now. Okay. There’s not one tool or one open source, like project out there. That’s going to resolve all these things. You have to think about it from the full life cycle. You have to think of it. Underlying nodes. You hear about, you know, CIS benchmarks. You need to ensure that those things are addressed from either the node level or the Kubernetes level.

That’s first vulnerability management. You need to ensure that in your pipelines, you’re scanning, you might use things like there’s Trivvy out there. Very good tool, right. Ancore another one. And so like again, don’t think that one of these tools is gonna be make work there’s commercial tools that honestly they have like, you know, the breadth of the of the solution from a, from that perspective.

But you need to think of the overall posture, then there’s the runtime, Falco and [00:16:00] forensic post-mortem. You can use that, but then also dumping that in some type of SIEM. Using the cloud providers, you know, the AWS is of the world. You know, GKE Openshift, right? Another one, if you think of like, you know, OpenShift recently with OpenShift plus, right.

They’re using something called StackRox, . StackRox actually is using the underlying Falco libs to be able to do some of that, you know, per progression of details. So I think, again, my advice to you all is don’t think one solution fits them all. Be open to, you know, there’s commercial tools.

Obviously they do all this, but be open to, if you’re going to put these things together, there’s not one tool that does all of them. What I think would be ideal for your listeners. Take a look at the CNCF. It’s basically the CKS is the certified Kubernetes security test, because that shows you how to protect the Kubernetes from all of those elements and all the projects that are used are part of the testing.

And it’s a practical test. So wishy-washy way to answering your question. That’s [00:17:00] just because, but I just think again, if there’s one sound bite, I want you all to listen to. There’s not one product or project that’s going to do all of it. That is based on open source that doesn’t, you know, you need to think of that overall posture.

Ashish Rajan: Oh my God, but I I’m a hundred percent with you on that one. I think also worthwhile calling out. You need to start looking for skills in the team, which can do some of this task as well, because do you feel there’s a skill gap for this as well? In the whole container security, Kubernetes security in that space, you feel there’s a skill gap as .

Dan “POP” Papandrea: So. I would say this in turn, there’s two parts of this too. I think the developer has to also understand how to be secure out of the box. It’s like, and that’s the other thing is it’s more of the the knowledge gap between like understanding like what’s out there. So developer and an operator also has to understand this.

It might not have a security background, right? So they, you know, they’re not looking at this, like they don’t look at this, like it’s, oh, I got to do anti antivirus. I’m just going to put an anti-virus in my container. Yeah. And that just makes me secure. Right. [00:18:00] I’ve worked with like large, you know, again, when my previous life, when I was a field CTO, I’ve worked with large investment banks, healthcare pharmas, all those types of things.

There’s every single customer or end user I went to was completely different. There might’ve been where the developers were so savvy that they may have included, okay, this is how we’re protecting XYZ, or they have, you know, a dev sec ops team that’s doing this or the best ones that I’ve seen integrate this.

They don’t have the security team here. The developer here, the operators here, it’s all integrated. When we do a build, we’re integrating security into our overall workflow. And if they don’t do that, they suffer because what happens. External actor comes in, because guess what? You have all of those holes.

Like we talked about earlier, there’s the cloud, there’s the nodes running on cloud. There’s the control plane elements on it. There’s the applications. All of those things need to be protected all of them,

[00:19:00] Ashish Rajan: but you just want the cloud one. Do you feel there’s a drastic difference between say. Like a managed version where if you, if someone was used, look at runtime in a managed Kubernetes versus like a non-managed Kubernetes, would there be a dramatic difference between or considering it’s platform agnostic?

I guess, according to quote on quote it wouldn’t make a difference would be the same thing.

Dan “POP” Papandrea: So cloud providers will already, you know, if you look at like GKE, for instance, you know, they have auto updating. Nodes, it’s right. That are, and I think EKS has a similar convention as well, but so like basically it’s, if a certain thing that you need to patch your underlying container hosting, all those things, that’s fine.

But you still, as a cloud provider is not going to have, you know, doing, they might have services that are doing like vulnerability management to a certain degree, but there’s not nothing stringing all of those things together. Right. And then from a runtime perspective you know, a lot of, these tools are looking at things like Falco.

[00:20:00] Like I know, like, you know, Falco’s looked at to be integrated in some of the, you know, things that are happening out there already. And so it’s just a matter of, you know, looking at this more of do I need to build a platform for this or incorporate some of the managed services here. I can understand why somebody is like, you know what.

You spend this up on EKS. Why do I have to manage a bunch of EC2 hosts running Kubernetes? And then I have to look at this and, and address this. And this is why, again, you need to have that overall workflow because you need to use the CIS benchmarks and you need to, you know, do the vulnerability regiment versus, okay.

Maybe I can just put this in ECR or GCR, have it, do that level of scanning. That’s only going to do like. Non-operating system and not full, like full scanning and all of those things. So it’s still, there’s a gap there. So , if you have a savvy team, it’s like, Hey, I’m going to do this and I can do this for less cost.

Or if I’m just wanting to get spun up, I might use Kubernetes. And you’re going to ask me this [00:21:00] question next. I’m sure you’re going to ask me, well, what do I need Kubernetes for all use cases. And you know, and my thing is, look, and I say, this is my Tywin Lannister. If you all remember game of Thrones, there’s a tool for every task.

And there’s a task for every tool. If your application is a simple stateless application run, that in Fargate run that in CloudRun. It doesn’t matter. Get the job done. If you’re looking at a scalable infrastructure, if you’re looking at a managed infrastructure that is, you know, has best of breed, that is a control plane that could run anywhere.

And there’s no lock-in think about spinning your own Kubernetes, but I can tell you right now, Do not look at this, like Kubernetes, I need to get my application Kubernetes to get it. No. Think about your architecture and say, I do, I need it maybe. And Alex Ellis out there. Right. We’re at a really amazing it was a, I think it was a blog he wrote about like, do I need Kubernetes?

[00:22:00] And so you gotta to ask yourself, you know, do you really need it? And I look, look, I’m a contributor to Kubernetes, right? Like I’m not, I don’t want to, I’m not saying my baby is ugly. I’m just saying. Look at your end goals and figure out is this right for you?

Ashish Rajan: Interesting. I love it, man. I think we’ve been talking about Threat detection.

I also want to get you up.

Dan “POP” Papandrea: I love this stuff, man. This is my,

Ashish Rajan: I can only see a pumped about this man. I was going to say from a response perspective and forensic perspective, like I think. It looks like most of the conversations that we’re having around this seems to be a lot of focus on there’s no permanent IP.

Like there’s a lot of things that are ephemeral and going. Oh, great. Okay. So where does response look like? So we kind of, we spend some time in Threat Detection. What kind of responses are we talking about in Kubernetes space, like, so. I love the passion that you have for Falco. I’ve taken Falco, deployed it, and now are like great.

I’ve realized one of my Kubernetes clusters is probably compromised [00:23:00] or something like what , what are some of the examples of forensics and probably a response that you’ve seen in the community for those that I would love for you to share some of those examples as

Dan “POP” Papandrea: well. I’m going to use the obvious one and that’s the crypto miners.

All right. There’s an example that we’ve had in the past where somebody was coming in, there was a honeypot and addressing, and they were, this was where somebody actually contributed a brand new set of rules based on that attribute attribution. So having to be able to like look at that and say, okay, this is how they were able to get in.

They were able to use, you know, the, the specific strata protocol to come in and be able to like, you know, take a look at that. And that was one where again, somebody contributed that rule back. And so since everybody benefited from it, that’s first and foremost, it’s, it’s the sharing of rules that address those things.

And then also it’s the, okay. Using thing cools beyond just Falco. Gatekeeper OPA pod security policies, to be able to say slap [00:24:00] done out in the response engine blogs that I told you about a second. We have something where if something’s detected, it kills the kills, the Pod. Guess what? Kubernetes is resilient.

Everything is going to still be up. So at that aspects compromise, we then within the daemons that that’s deployed. That person gets kicked out. Those things would just go back to whatever the code that’s there. And so then you go back and you say, okay, now I need to make sure my code is not vulnerable.

That’s vulnerability management. Right? So if I’m, again, the scenario I mentioned earlier,Ashish, somebody goes in and goes into that pod and it’s terminal into the pod and it’s going in and looks in that two, shot them doing the thing. They do try to drop their things in there. If they tried to do that, a Falco is going to find it immediately.

At the speed of syscall and then run that function. I told you about earlier using sidecar, getting, knocking them the hell out.

Ashish Rajan: Oh, I love it because there’s the unique aspect which you [00:25:00] dropped in. The it’s probably worthwhile repeating the whole aspect that it maintains a state. So even if you kill a pod, as long as the code is not changed, if we’re just redeployed the same thing, but in a clean way.

And if we’re just take out who are the bad guy, bad person, wasn’t there.

Dan “POP” Papandrea: At the node level right now, let’s think about it at the node level. And this is again where look being able to have that level of detection, you can automate this, or, you know, you can use downstream again, there’s a lot of commercial tools that are out there’s there’s things like Aqua sysdig, right.

That had that built in capability to like address, you know, the same thing as Falcon could do is notify your SIEM. And if you see these signals. Then what do you do? You take that action, right? And then you are all the wiser for them. Putting policies in place. OPA policies, vulnerability management policies, Falco rules.

Because you, I mean, again, remember the thing I mentioned earlier, now, one of these tools and I’m going to do all of it.

Ashish Rajan: Yep. And I love the passion and [00:26:00] I do want to take some, a few moments to talk about your podcast as well, and get into a bit more about what you talk about in your podcast and what got you started podcasting.

I think we’ve kind of touched on the whole sort of Threat detection and response and a runtime con responsibility for capabilities and how. People can use that to be part of the entire flow and not just one component of runtime. So I love that passion as well that you have I, I did want to introduce people to your podcast as well and take an opportunity to go.

What got you started with the podcast? First question let’s start with that first. I want to know I’ve got a lot of questions about the podcast.

Dan “POP” Papandrea: So I saw you started yours. I’m like I had to keep up with this guy. No, I’m just kidding. So. You know, I was like, I was visiting a lot of end-users and customers and I just, I, you know, my thing is I loved seeing how different people did their thing.

Meaning like, Hey, you know, you’re going to, you know, I don’t know a gaming company and they’re like, wow, this is how we’re deploying. And then you go to the next customer is a completely different way of, for them to do it. So I’m in my house and in my basement right here. And I’m [00:27:00] like, I got some friends in the community.

I just like to do a podcast just to interview and have some fun. And so like, it just blew up and the way that I look at it. And so it’s called The Popcast. If you follow us at Twitter @podcastpop we’re on, it’s a, it’s an audio and video as well. So it’s you know, various things like apple and Spotify, and it’s basically the people behind the code.

So I’ve had, you know, the godfathers and godmother of Kubernetes. I’ve had Liz Rice on who’s, you know, you know, she’s very cool person. One of my good friends. I’ve had just a ton of people and, and it’s all based on again, it’s not so much, you know, we talk technically we can get in the weeds when we have to, as you know, we did here, but I look at this more of there’s the people, the humans that make this community great.

I want those stories out there. And we’ve had a hell of a, an amazing time with, I just, I love it. And it’s, it’s my passion. And I’m sure it’s the same thing as you, you never thought you’d be like, do it as allowance, just like. I love it. Like, I, I, I’m consistent now where like, [00:28:00] I’m, I’m an episode behind you, buddy.

Ashish Rajan: Don’t worry, man. It’s not a competition, but did I tell you about the three episodes, which are audio only, which is not even on the website. So I guess it’s technically four, but not that it’s not competition, it’s funny last December I had two cybersecurity podcast hosts come in.

. And what I realized was that there they’re actually not that many cybersecurity podcast as well, but then I met, I saw your podcast and I actually, this is how many are cloud native. I actually don’t know that many, unless, I mean, there’s a Kubernetes podcast and how many are cloud native and I don’t know if people know this, but your is CNCF ambassadors.

What does that really mean? And for, I mean, I guess in case people haven’t seen the previous episode, where does CNCF as well?

Dan “POP” Papandrea: It’s the cloud native computing foundation. And so The CNCF itself is basically this governing overseeing kind of aspect that basically looks at all of these projects out there.

Incubator projects like [00:29:00] Prometheus and Kubernetes, and gives us . This governing aspect gives it marketing user capabilities. They put on something called KubeCon every year in cloud native con it’s where like, and again, it’s one of the best communities in the world. If you remember, like, You know, I, I was part of the OpenStack community, you know, we’re all part of like the mugs or, you know, those types of things.

This is a little more different. It’s very inclusive. It’s diverse. You know, I have friends, like I said, Magno, and so that’s the beauty of the community is doing these things and, and being able to, you know, there’s people that are, that are brilliant out there, but they’re also so. Giving with their time to tell you about things. So I’ll give you an example, like before I got into, you know the , first couple of years I was at Sysdig, we were having to manage Mesosphere and Kubernetes and Docker data center and to be able to deploy it to all the various customers out there, and then Kubernetes ended up being like the de facto.

And so that’s when I started contributing to it and all that I sent Joe beta godfather. I was like, [00:30:00] how do you do this, this and this on Twitter. I had a response within five minutes within five minutes.

Why did we do this thing with like health checks?

I was like, well know, can you explain what the idea was behind it? And it’s just, but it’s always from that spot forward. In a, with a regardless of podcast, regardless of why I’m in there. All of these projects I am in love with. I love seeing Prometheus. I love seeing cloud custodian. I love seeing all these new projects because it makes it better for everybody, everybody to be able to contribute and do things.

And so CNCF ambassador. So basically it’s talking about, you know, projects, you know, being at meetups, being able to talk about intelligently, not only about the projects that your involvement I’m involved with, obviously Falco, right? So it’s basically all also about all of the various projects out there.

And also like, you know as a CNCF ambassador, It’s we’re also looking to bring in folks for them to understand, like you said, like Ashish, like, I want you to understand this, to talk to your listeners about security and [00:31:00] when you have that question, But you’re like, Hey, POP. You know, I really want to understand how to do Falco rules, or I really wanna understand how to use Prometheus for some of the security goals I want to, or maybe a last a search, but I’m building my own SIEM.

I don’t want you to feel like, or any of your listeners to feel like they can’t ask anybody in our community. Join our tag security group and CNCF slack. Ask those questions. You all will be all the better. You’ll make this community even better than all the communities out there that we’ve had. And that’s what I believe a good CNCF ambassador does.

Ashish Rajan: It’s pretty awesome by the way. I’ve got a question from Roxanne here. She’s she’s oh, it’s been a long time, this and that. So she has a question. Is there too many tools and policy engines? OPA gatekeeper, Kube-bench Falco. Oh my God. You’re so right. Roxanne. How could all of these be managed at scale in big enterprise?

The diverse number of dev teams?

Dan “POP” Papandrea: Roxanne. You’re absolutely right. There’s a time, but all of those things can maybe give you false positives and it takes the team to understand what each [00:32:00] of those tools do, because each of those tools do a different thing. You think about kube-bench kube-bench is doing CIS benchmarking.

If you think about Falco or runtime, , drift prevention as part of any of any tool that’s out there, all of those things that are out there, this is where you need some type of collective backend. That’s going to take that data and say, what is a false positive? And what’s not. And I can tell you if a vendor tells you, they can give you all that in one space group, they’re bullshitting you.

And I can tell you that right now. So Roxanne, you have an absolute point. I think it’s getting better, but also I’m going to swing it back to you, go back and contribute to these projects and say, these are signals that really do matter. We’re fortunate from the Falco perspective is we did, you know, folks tributing from that perspective, that’s 120 rules are the rules.

We saw map to MITRE attack framework, things that were happening in the world. Like I said, the crypto jacking example, some of the CVE things that are out there, you’re going to benefit from there. But I agree. There is a lot of signals, but [00:33:00] there’s also organizations are doing things. Every organization does things very much differently.

So you’re going to have to figure out what are the signals that matter to you?

Ashish Rajan: I feel like just to Roxanne’s point. Cause I think it’s a great question. Cause that’s been a common theme with a lot of conversations that I’m having as well as there’s so many tools. And like I think cloud, this is kind of where cloud was in the beginning stages were great.

We have AWS only. So you just do some CIS benchmark and there was no one consolidating it. When you start scaling out, there was no one consolidating and suddenly you feel like, oh my God, this is like so much happening in the cloud and is suddenly. Someone was able to pick up I guess that gap and give a visibility layer right across it.

And I feel like Kubernetes at that stage where we’re still, even though it’s been there for four or five years just been growing exponentially, it still hasn’t reached that stage where a lot of people have had enough pain points shared that someone’s going, you know what, I’m going to make something and just give it out there.

I feel that that’s [00:34:00] missing and. This is going to be the standard that as security field, we’re still like, holy, wow, this, this gap, this caught up really quickly. We need to do something about it. They’re just having that realization now.

Dan “POP” Papandrea: That’s the thing, like you said, it’s like, if you think about, like, if I was.

When back in the day, when a director of it, and I’m looking all the signals, I would have to look at it from, oh, the windows host perspective, the Linux host perspective. And, and so I, I agree with you again, there’s a lot, a lot of signals. I mean, she didn’t even mentioned, Kyverno was another like, you know, policy like that.

Right. I mean, there’s a lot of them out there and I do agree, but I will tell you this, and this is something, I think the Kubernetes security group that’s not specific to tag security, but the Kubernetes SIG security, right. Looking at all this and trying to look for ways to ensure that like, okay, there’s there, all of the underlying components are as safe as possible.

So that’s the first part of this, but yeah, I mean, look. There are a ton of policy engines to target part of those tools. But I do believe that, you know, you’re ultimately [00:35:00] always going to need some type of SIEM or some type of machine not learning. I don’t want to use that term, but like some type of user based intervention for the things that are happening.

Ashish Rajan: Right. And, and I think for hopefully that gave you some light on for the answer, I guess so extended, but we definitely need to have a conversation about this a lot longer. We definitely need to be talking a lot more about this as a community. That there’s definitely gaps that need to be filled. More people need to contribute.

And I’m going to switch gears here. We’ve been talking about that. We spoke already a podcast as well, and we spoke about I guess how trends can be detected and responded in the runtime. I would want to. Give opportunity for people to understand you as a person as well. And this is usually called the fun section and I’ve got two questions for you, man.

It’s pretty straight forward. First one being, what do you spend most time on me not working on Cuban days and contributing to communities or like Falco. What do you do outside with all this?

So. [00:36:00] Beyond my podcast and I, you know, my children, obviously making sure I have two kids and you know and so just making sure that they’re taken care of and take care of my wife and all that fun stuff.

I’m a musician. So I went to school, go this way, musician. Yeah. I went to school for music originally and I was fixing the Mac. Mac computers. And that’s how I got my first like computer job. And so, yeah. So like, you know, like that’s kinda my, you know, my passion. So like the opener of my show, the play that we play is actually my band playing it live at CBGBs in New York.

Really? Yeah.

Wow. Well, I’m glad, I’m so glad. Cloud native has Ms. Musicians actually you’re the second person. I know who at least who’s come on the show who has a musician past have you got songs as well, early on to share an album with something. If you guys have released a few songs as well, where can people find some of your old band songs?

Dan “POP” Papandrea: Whenever you watch the podcast at Popcast pop, you’ll see the opener. That’s pretty much there. I mean, like [00:37:00] again, it’s, you know, it’s, it’s, I’m not one to like, I play live and stuff like that, but it’s not something like, it’s like, I know it’s this isn’t my day job. Like, you know what I mean? Like this, my day job is this.

So like, I’m not like looking at like sell an album or anything like that, so, oh,

Ashish Rajan: right. I didn’t mean it that way. Cause I think about we had a guest from capital one and he has a whole album that he had released. Spotify and I wanted to support him. So we kind of got into that as well. They all go, there you go.

I had no idea. Magna is used to be a DJ as well, so he wants to make a band. This is, this is amazing. So we have a guitarist and we have a DJ, so I’ll be a very different kind of band, but sure. Let’s maybe you can go down the pub.

Dan “POP” Papandrea: That’s another thing is like, I think musicians make amazing. Either developers, engineers in general, because that patients that they have, like, there’s a person, their name is Jace out there and Cooper long-time Kubernetes contributor, one of the best saxophonists I’ve ever heard, like, like, and just brilliant, [00:38:00] brilliant musician.

So it’s, it’s awesome. And again, I just think I’ve a lot of engineers and probably special, probably the person from capital one. It’s that? Discipline. You know, to be able to, it’s just the same thing. It’s repetition and that’s what makes a good musician. I think that’s also what makes a good engineer as well.

Ashish Rajan: I actually, that reminds me of another person. Tiny Jenka. She used to have a band as well. She’s come on the show as well. So if anyone wants to check out some of the past guests who are musicians, she used to actually have a band before she joined it because people said, you seem to know it stuff. And she like really?

I mean, I just, this is like everyday stuff. Like, no, this is definitely not everyday stuff for a band person to know that’s how she got into it. And then going into cybersecurity. I’ve got one more question. Actually two more questions. What is something that you’re proud of? Part is not on your social media.

Dan “POP” Papandrea: Wow. That’s a really good one. I would say obviously, besides, you know, my kids, I, you know, I’m, I’m very proud of my kids, but Yeah. I mean, I’m, I’m proud of the success of the podcast, you know, and, and also we’re launching something on cloud native [00:39:00] TV. It’s a on CNCF Twitch. Wow. Yeah, we should talk later.

Anyway. Yeah, there you go. And if this is just I’m executive producing that I’m loving it because we’re putting together a group of just different shows in the various aspects of. You know, cloud native. So we’re having, like, how do you contribute to a cloud native project every week? It’ll be a different person from the project on and contributing to it.

We’re having a game show, like family feud, it like having that, the various products of the CNCF kind of duke it out for like, you know, supremacy and the CNCF cup. Yeah, it’s just, and you know, there’s somebody like there’s somebody doing a one-on-one track, somebody doing how to do like certifications, you know, it’s and also we have what I love again, talking about the inclusiveness and the diversity of CNCF.

We have some show talking about you know, LGBTQ and people of color and how they, they rose above and being able to address, like, being able to be huge contributors to what’s. To the amazing thing that isn’t CNCF, because [00:40:00] again, the diversity and inclusion is something the previous general manager, his name is Dan Cohen.

God bless his soul is very new Yorker as well. That was something that he, he wanted. And that’s what I think the CNCF and large strives for is diversity inclusion.

Ashish Rajan: That’s pretty awesome. And thanks for sharing that as well, man. I am definitely looking forward to, so like he’s looking, who’s watching and participating in this.

This is a big ride they’re back by, right, right back at you. Last question. What’s your favorite cuisine or restaurant that you can share now since New York is opening up, I’m assuming there’s a few more restaurants, but if restaurant or queer favorite cuisine

Dan “POP” Papandrea: Magno, it’s not pineapple pizza. So first off I just thought shout out because well, last year I did that sole to get to, I promised everybody in cloud native that if I, if I had 2000 followers on Twitter, Which now we’re up to 8,000.

I would eat a pineapple pizza, full pineapple pizza live. And so this is really out there. It’s pretty, pretty heinous.

[00:41:00] Ashish Rajan: Oh my God. Okay. You suggested you also go pineapple pizza. Okay. But I love it. I love it.

Dan “POP” Papandrea: Sorry. I love the fact that you used subjected myself, which was, she was tortured, but that’s okay.

But I’m such a fan of world cuisine. So like, again, my father growing up, we own 10 pizzeria, so it was pizzerias and restaurants. So I grew up in the pizza business. Right. And so once I got out, I gotta, I want to be able to like try different cuisines. So I love like. Indian cuisine and Persian cuisine and Vietnamese different things.

So like, you know, it’s, it’s really dependent on it. Like being again, going through different cities and trying even the local cities that you go to. Like, if I go to Seattle, you know, oh, there’s amazing salmon there or something like that. We’re going to San Francisco crab and you know, those types of things.

So it really depends on it. That’s the hardest thing to tell an Italian. What their favorite cuisine is. Cause you know, this is, it’s a tough thing, man. Especially being from Melbourne. There’s a lot of Italians there. Isn’t there a shit?

[00:42:00] Ashish Rajan: Yeah. I think not enough. Not, not enough versions. I think we definitely have, I definitely try a lot more of Persian food, which I haven’t been able to explore.

I definitely look, look for some recommendation for that for someone. And if anyone has a recommendation of Persian food, I’ll definitely take that. But I think we definitely have a massive Adalynn population, a lot of as a large African population as well. And Chinese, like, I think Asian food is like, cause we so close to Asia as well.

Like so much of it, man. But yeah,

Dan “POP” Papandrea: he told me like, there’s something called the chicken Parma and it’s a specific type of chicken parm sandwich that’s in Melbourne. Is this

Ashish Rajan: like, you guys

Dan “POP” Papandrea: don’t have chicken bummer. Now we have chicken parm, but it’s very specific. He was like, not like my buddy was from Melbourne.

He was like, Nope. Not as good, not as good as in Melbourne. This is, that was that’s way better.

Ashish Rajan: Chicken parm. The same as the chicken snitzel with some sauce on top. It’s like a deep fried chicken sizzle.

Dan “POP” Papandrea: Yeah. It’s the same concept, but they were saying the bread is different and I’m like, [00:43:00] and this was like five years ago.

I heard this and I’m like

Ashish Rajan: to come and try it. I mean, it, it definitely is like a, it’s like a very popular thing in any Bob, Bob, you walk into I do want to at least leave with this though, with folks who want to reach out to you and talk more about runtime tradition response, where can they reach you?

Dan “POP” Papandrea: Fantastic question. So we’re on Kubernetes. Slack is a Falco channel there. So if you want a Kubernetes second and Falco also part of tags security. Tech security is all, you know, Magna is part of it as well. And again, so in place, if you want to understand, they, they wrote a best practices guidelines document it’s.

If you go to CNCF slack, there’s a tag tag dash security channel. Fantastic. And again, it’s just want to get your feet wet, ask those questions, because again, it’s about. That’s what I love about CNCF. It’s a community that’s inclusive. Come in, ask those questions because guess what? We all were beginners.

I mentioned that [00:44:00] story earlier about Joe Beda. We all wear those beginners. You go in there. Magna’s a tremendous help. Andres Vega you know brand alum, Emily Fox, all the people that are in that group all want to help you be more secure. Also your expertise, Roxanne, that question you had about the false positive come in the channel and help out, come in contribute.

Tell us what’s wrong. Tell us how we can help it. If you had this experience, it’s awesome to be able to share it.

Ashish Rajan: Oh, nice. And you still don’t mentioned maybe PR people can reach out to you the sort of tax security, but you’re specific in more on Twitter or LinkedIn were very hang out more,

Dan “POP” Papandrea: Either of those.

So yeah, so I’m Dan pop in YC or at podcast pop CAS T pop for the show and stuff like that. But yeah, mine, either of those, you know, giving my unique. Impressions on the world in, in either of those on Twitter.

Ashish Rajan: And I think I’ve brought my my friend here, my new friend here Roxanne’s friend.

Tell me if you’ve Persian restaurants as well. So I [00:45:00] was saying, you can remember my next best friend now. So thanks for that. Pop is everywhere. Magna definitely loves you as well, man. So thank you so much for coming on the show, man. I really appreciate this and I know it was last minute as well.

So thanks so much for taking, spending time as well. And

Dan “POP” Papandrea: I am a big fan of yours, man. I really think, well, I appreciate that, man. I wish I had that

Ashish Rajan: number of episodes behind

Dan “POP” Papandrea: now. We’re we’re brothers. We’re brothers.

Ashish Rajan: Totally, man. I think I, I I’ll give you that much. I totally enjoyed the passion that you bring to the show, man.

I think it’s, it’s like. I, I, if I were to kind of like put like a scale, you’re definitely right up there for passion, man. And I it’s all the 71 episodes or 72nd one, this one, I definitely feel your passion level is right up there. You can just feel it coming through like the awesome mic that you have as well as you spoke about offline.

So it’s like, it’s like meant to be, but I really appreciate you came over women. So thanks so much for coming in and we [00:46:00] should definitely do some crossover more like on the Twitch TV as well.

Dan “POP” Papandrea: You have a spot anytime you want. Let’s talk all

Ashish Rajan: us. Thanks, man. And for everyone else feel free to subscribe to the channel if you are like what you’re hearing and we’ll talk to you the next weekend.

Thanks so much for coming in. See ya.