About this Episode:
Need to secure your AWS environments? In this episode, Jonathan Helmus, author of the new book AWS Penetration Testing, will share some tips on AWS penetration testing and security best practices. Discover some of the commonly exploited vulnerabilities in AWS and how to prevent them. Listen in to learn more about cloud penetration testing tips, and tricks.
TestGuild Security Testing Exclusive Sponsor
Micro Focus Fortify is the recognized market leader in application security and is the most comprehensive and scalable application security solution that works with your current development tools and processes. Try it today
About Jonathan Helmus
Jon Helmus is a security engineer, educator, author, and cloud hacker who has been working in engineering, security, and information technology for 10 years. He specializes in Penetration Testing, Threat and Adversarial Assessments, Vulnerability Management, Cloud Technology (AWS), and also has experience as a Technical Educator and University Level Professor.
Jon is known as “the granola” hacker due to his consistent attitude of “giving back” to those trying to get into pentesting, as well as helping others “bypass the gatekeepers” and get into cybersecurity.
Recently, Jon became the author of the new book “AWS Penetration Testing”.
Connect with Jonathan Helmus
- Company: www.cobalt.io
- Website: www.cobalt.io
- Twitter: @Moos1e_Moose
- LinkedIn: /jon-helmus-474146103/
- YouTube: channel/UC43JcMMRzft9mqDR-LSaA
- ** Buy AWS Penetration Testing: Implement various security strategies on AWS using tools such as Kali Linux, Metasploit, and Nmap
Full Transcript Jon Helmus
Joe [00:01:48] Hey, Jon! welcome to the Guild.
Jon [00:01:52] Hey, Joe! Thanks for having me on the show. Glad to be here.
Joe [00:01:54] Really excited to have you. Jon before we get into it I'm always curious to know, first of all, is there anything I missed in your bio that you want the Guild to know more about?
Jon [00:02:02] You know, you pretty much hit it on the head. The long-winded version of my concise bio is that I have a very vast background and different experiences, which I've given lots of talks just based on, like how to get into pentesting based off of my vast experience and how individuals can with that, as with their vast experience. I really like the granola hacker bit where you mentioned, you know, my way of giving back is kind of being this hippie pentester where I'm more about keeping the peace and helping out those who want to get into pentesting rather than being a gatekeeper and making it more difficult than it needs to be to get into pentesting.
Joe [00:02:39] Did you find that that is a barrier for some people, that there are gatekeepers in this particular niche?
Jon [00:02:44] Oh, yeah, absolutely. Like, if you go on to various different social media platforms, you'll see a lot of passive-aggressive toxicity within the field where there's a lot of individuals poking at each other based on what they think is a valuable metric to, I guess, qualify someone as a pentester or not. And you see that all over the Internet.
Joe [00:03:06] Got you. So, you know, I'm always curious to know when people write books. I mean, I tried writing a book five years ago. It took me a year of my life and I published it. Why write a book? And your book is huge. I think it's about 300 pages.
Jon [00:03:16] Yeah, that's a good question, because, like you said, it took you a year. I think for me, it took about a year from signing to actually writing to getting the book on the digital shelves. Why write a book? This was something where I was actually hit up by the publisher to write a book based on pentesting, and they wanted to get more focused in cloud. And I had done a lot of AWS pentesting at my current employer that I was with and my previous employer. But the thing I noticed was there was not a lot of material on pentesting the cloud and more directly with pentesting in AWS. And as we've seen, AWS is something that is really taking the market by storm. They're the market leader for cloud. And, you know, also like biasedly I live in Seattle. So that's where AWS is. And I just saw it has this area of like, “Hey, this is a really good way to get my niche in there as a cloud Pentester, but to also, more importantly, give back to the community and be one of those, the four front runners of the founders of setting up the beginning of formal writing for cloud pentesting rather that be in a technical sense or in a theoretical methodology kind of sense, which AWS pentesting really does a good I think it rides the knife-edge pretty well of doing both where it teaches you things at a theoretical level and a conceptual level, but also at a practical level as well.” And yeah, so writing a book has a way to really help encompass all that and give back and then start the conversation around, “Hey, we need to actually start formulating the way how we do cloud pentesting.”
Joe [00:04:53] Absolutely. So yeah, it seems like an area that there's not a lot of information. And so, you know, we mentioned cloud security a lot, but this is AWS. If someone bought this book and read it, would a majority of it still apply to any cloud-based type of environment like Azure or is it specifically AWS all the way through?
Jon [00:05:12] That's a really good question. So the beginning of the book talks more about AWS. If I remember correctly, looking at the book, the first few chapters I believe they go over like this is AWS and pentesting AWS. The second chapter goes more towards how to set up AWS and then it goes and dives into pentesting certain services such as Lambda S3, APIs, and things like that. So that's all geared towards AWS. However, the last half of the book, the third portion where it's the good dos and don'ts of cloud pentesting that is a portion of the book and that's a good one hundred pages of the book that allow you to take those concepts and actually formulate those into new ideas that can be cloud-agnostic.
Joe [00:06:00] For folks who are listening, I believe there are 12 chapters and there are three main sections like you mentioned. I think the first one is, as you said, again building AWS environment. Section two is pentesting the cloud exploiting AWS and Section three is lessons learned. So I thought that we dive into each section and try to pull some information out, to really tease it out, to get people to really buy the book. So the first one is the first chapter is Building Your AWS Environment. And I'm curious to know from a Security perspective as people are building their environments, is there anything they could do from the beginning to make sure that building a more secure application, rather secure infrastructure rather than wait until the end to do a Pen Test against it?
Jon [00:06:39] That's a great question. And in the book, this book doesn't actually go into it. There are ideas for future literature based around that idea. But in the sense of securing let's go with AWS, in general, because AWS is one of those things where, comparing it to something that's also big in the market, like Azure, Azure tries really well to mimic on-premise kind of assets, as opposed to AWS. Learning AWS, it is an entirely new skill set altogether because it's an entirely new application. And so in regards to ways to build better, let's go with AWS applications, I'm a big advocate for saying bring Security in early on rather than later on. And that doesn't mean just having someone like a security engineer there to do the tech review. I'm a firm believer that if you have an internal pentesting, a lot of bigger corporations now do deploy internal teams, have someone there at the beginning of the conversation to start talking about Security ideas and concepts based around whatever it is that you want to deploy. Because what that's going to do is that Pentester or Offensive Security Engineer, whatever title you want to give them is going to bring in that adversarial mindset approach. And they think about things in that light as opposed to a security engineer, like a typical one might just look at it as a hardening Security control kind of basis. The Pentester is actually going to say like, “Hey, like if you don't incorporate, I don't know, detection or detection on a certain part of the service or monitoring or if you don't have deletion protection. These are the impacts that can happen through misactions, misuse, or even compromise. And so having a pentester there is going to build that security awareness where it's going to have a trickle effect, where a year down the road, everyone at the table at the pre-deployment table, so to speak, is going to understand like, “Hey, these are things that we need to look out for. And these are things that we need to share with other departments within the organization to ensure that they're following a similar protocol to make sure essentially ensure that we're deploying assets that have Security built-in rather than Security being an aftermath.
Joe [00:08:50] Absolutely. And I guess because it's in the cloud also. It seems like if you work for a large organization, you have multiple groups setting up environments unbeknownst to you. Could that expose bigger companies to issues if they have you know, they don't have a full grasp on the magnitude of how many people are using AWS for services like that?
[00:09:08] Oh, yeah, absolutely. When the cloud scales so fast that the other thing is, I guess this is kind of more of like a not a correction to the question, but more of a potential solution is when you're having multiple heads in on the cloud and it can become cumbersome really, really quick because the cloud is continuously scaling, continuously scaling internally with the environments. And it's one of those things where companies are just like executives and leadership loss often look at it's like a lift and shift where the engineers are like, “Oh, no, we don't even understand how to use AWS yet. You got to give us some time to catch up on that.” I think one of the big things is that there needs to be before or during some kind of cloud development, there really needs to be company-wide or at least department-wide awareness training on the cloud. And what that means is like, “How are we educating ourselves? How are we measuring the effectiveness of the Security training that we're giving to the individuals that are going to be utilizing and deploying the cloud?” And then, you know, based on that those two metrics, how are we going to deploy effectively and efficiently after we've educated and determined if our users or our environment is cloud-ready? How do we deploy it securely and then how do we build off of that? And that was like a four-step program or a four-step process that I had actually introduced at a talk at the Wild West Hacking Fest. And they had a Cloud pentesting Summit that I talked about last week with a group of other Cloud Pentesters. And yeah, that's something that I think is going to have to be incorporated to ensure that we maintain this or the users maintain proficiency with the scale of cloud computing because like you said, it's everywhere. You can bring up an entire infrastructure in less than a day. So we have to make sure that the engineers and the users and everyone who's using it is caught up to speed.
Joe [00:10:54] So I also hear people talk a lot about excessive privileges. So if someone's building an AWS environment is that when this comes into play that they need to be cognizant of that or is that a different step of the process?
Jon [00:11:06] Oh, no. The number one reason for compromise in AWS is because of misconfigured services and typically it's on the user level or the group or role level. And so because if you look at AWS it's a billion-dollar company, right? So they're spending that a lot of money on ensuring that their services are hardened and that they're secure by default. And when we're targeting it, we're not targeting the services themselves. We're targeting the things that the human element implements. And the biggest thing is the roles and the user-level attributes that are being assigned to the AWS environment. So, for example, you know, you might find a role that's overly permissive and Engineer Tim is actually able to execute group level privileges because for some reason he accidentally got Admin John's privileges as well. And now he's got full keys to the kingdom. And believe it or not, it sounds kind of I guess silly is a good point to put it, but it's that's a huge, huge issue when deploying AWS.
Joe [00:12:09] So if I have an AWS instance, it's in the cloud and I work for a company and I have it so someone can easily get into my particular instance, is there any way they can hop over into someone else's instance that may be on the same network or…?
Jon [00:12:22] Oh, yeah, absolutely. Absolutely. Sorry, I didn't mean to cut you off, but yeah, no, so in the cloud, they use what's called a virtual private cloud. It's the VPC network and think about that as like a subnet in the cloud. And so if you have two instances let's go with like two I don't know, Red Hat hosts that got something hosted on Amora and someone gets access to that. Absolutely, if that host can talk to the other hosts inside that VPC, again remember the VPC is like a subnet. Absolutely, you can definitely talk to that other one. There are potentials if that instance isn't kept up to date. The other one that the user's not on that instance hasn't kept up to date or if it's got some kind of vulnerability on it because you still have the administrator, engineer, user, whatever you want to call it. It's your job to maintain the updates and patches for that host operating system. If you let those lapse, then, yes, that becomes vulnerable. And if you have one instance become compromised, then the other one can be compromised. And even to further that attack, there are such things called VPC hopping where essentially it's just hopping from one subnet to another subnet in the cloud. And that kind of goes in line with privileges getting exploited if something's overly permissive. There are potentials that a compromised host in one subnet or VPC could actually hop into another VPC and then start exploiting things, host services within that other VPC.
Joe [00:13:46] It's pretty scary. So I guess this discusses kind of into the second section of your book, which is exploiting AWS. And I think you have a chapter on S3 buckets. Is S3 buckets the main area that people will try to exploit to start with when they're dealing with AWS?
Jon [00:13:59] Yeah, for the most part, you see a lot of stuff with S3. If you go and you do like a Google search or a Google scholar search, you'll find a lot of formal and peer-reviewed articles on exploiting S3 buckets. And it's very, it's company agnostic in a sense. And what I mean by that is that you would think, “Oh, it's probably going to be the smaller companies that are going to be getting their S3 bucket object storage exploited.” But it's not if you go and you look, you'll see companies small, medium, and then even the large, such as like Facebook, the NSA, DOD contractors getting exploited via their S3 buckets storage just because of weak policies or compromised keys, which keys are essentially the keys that allow you into the AWS environment. Yeah, so S3 buckets are the big thing. And it was a big thing that I wanted to touch on in this book just because it's one of those things that we need to become very, very familiar with because it's such a common service and it's one of the most exploited services in AWS.
[00:14:59] Now is that good? Does that go back to when you try (??) to be misconfigured? Is that what makes it easy to hack? Or sometimes people just misconfigure it to make it so it's easier to penetrate or to get into?
[00:15:11] Yeah, that's a great question. So with S3 directly, there are two issues to the puzzle there. There's the user level issues where you might have a user that's overly permissive and able to get into a bucket. And then there are the buckets that are put in the public by accident. And what that means is essentially think about it as your Windows file server. You have Windows folders on your computer. Imagine being able to store all the data you want and you create that into a shared drive that you share with other individuals inside of your company. Well, imagine being able to give that folder an API and make it accessible by the public Internet. That is a big common issue that happens with S3 buckets is that they'll be given public access. And I can't tell you how many times I've done consultations with companies where they make them public and they think, “Oh, well, I thought it was public internally on the Internet, not the Internet.” And so these buckets essentially, which are file storage, are put on to the open Internet. And what that means is that anyone on the Internet can go and query the URL to the API and grab whatever is in the bucket. So it's a user level issue and a service level issue, that API issue is the service level issue.
Joe [00:16:22] So besides S3 buckets, I guess you also mentioned RDS. Is that something similar with the… so people can get through an API or to exploit it?
Jon [00:16:30] Yeah. So RDS's are the relational database systems of servers. Essentially think about that is like a database. It's essentially a database within the cloud and you can give that a public API as well. And then the same issue if an RDS database is public, someone can potentially get into it. The other issue too is that if there's, think about it with anything like if you have a vulnerable front-end web application interacting with your RDS database, then you incorporate a whole new level of issues too just like you would with an on-prem back in database server. It's the same thing. You can start exploiting that database via the vulnerable web interface.
Joe [00:17:08] How common is it between S3 or the RDS that people attack? Are both of them just as common or is one more familiar to attackers than others?
Jon [00:17:18] Oh, no, it's S3 all the way. If you Google AWS attacks or AWS compromise, compromises, or things like that, I would guarantee you I would put 10 free books on it that nine times out of ten, if not more, it's going to come up with some kind of S3 bucket breach.
Joe [00:17:33] Now, how about Lambda? I've heard a lot about Lambda. Once again, I'll state my ignorance here. I just thought it had to do specifically with code. I didn't know you could design it in such a way that it could also open you up to be exploited as well. Can you talk a little bit about Lambda Services and how maybe they could make you vulnerable to a Pen Test?
Jon [00:17:50] Yeah, absolutely. So in my book, we do some cool things where we actually exploit the service itself. There's another book out there that it's called, I think it's called Hands-on Penetration Testing with Kali Linux or Hands-on AWS Penetration Testing with Kali Linux. And it's very Lambda heavy. So for this book, I didn't want to focus too much on that because the other book was such a great resource. And there's a lot of good resources on the Internet for Lambda because like you said, it's essentially, think of it as like an IDE in the cloud that can execute your code and you can execute it to do various things, interact with services, interact with APIs and do a lot of different things. It's also not directly related to one language. You can use a bunch of different programming languages inside of it. In the book, I focus on Python because I'm proficient in Python, so we use Python for that. So Lambda service is essentially like I said, it is an IDE within the cloud that allows you to execute code on a triggering basis. And what that means is it only executes when a certain triggering function is executed and then the lambda function responds. However, there are lots of issues with Lambda where there's a lot of roles that can be overly permissive. And one of the big issues is that integrating Lambda can be kind of difficult and cumbersome. So one of the things that are often incorporated into it is that they add what's called a star attribute, where that essentially allows anyone and anything to execute the function. And for a developer, it's a lot easier to do that than have to assign roles and make sure that whatever that Lambda function is operating or what working with is also on that same role. It's just much, much easier for them to say, “Okay, well, let the Lambda function interact with whatever we need to and we'll build Security afterward. Kind of like what we were talking about at the beginning of all this. And so what that does is that allows anyone to exploit the Lambda service. And you could do anything with the Lambda service from writing your own code into it to using it to escalate privileges to another user. And in my book, what we actually do is we use a vulnerable Lambda service to execute a reverse shell into the Lambda function. And what that means is we essentially set up a persistent connection to the host operating system of the Lambda function. And what you can do with that is you can actually use that as an undetected pivot channel in a Pen Test or Red Team exercise that allows you to pivot off of, say, you have exploited easy two servers. You could use that Lambda function as a pivot to go to somewhere else in the cloud and be undetected. And so we touch on that in the book.
Joe [00:20:36] Awesome. So this book is like I said, there's a lot of information. You cover all these points so far in detail in each chapter. So I just want to start wrapping up. I guess the next question I always ask everyone is tooling. Is there any specific tooling you use for AWS? Is it just your standard Metasploit that any pentester would already be familiar with?
Jon [00:20:54] Yeah. So in this book, we focus on Metasploit a lot because I wanted to make this book beginner-friendly because for that simple fact if there's not a lot of information on here, so I'm a big believer in you've got to start walking before you can run. And the people running all over the place with material on how to do this but it's not concise and formatted. So it's like, “Okay, this is the perfect time to start formatting it at an entry to intermediate level and start building from there. And so Metasploit is a great tool because it has some proprietary AWS tooling or tools in it. However, you know, in the book we use Burp Suite we use Nmap a lot. We use some open source tools that we find on the Internet. There's a lot of S3 tooling out there that you can find on GitHub. I don't have any one specific tool that I can recommend. But you can definitely do a GitHub search for S3 bucket tools. There is Pacu, which is a tool developed by some individuals up here in Seattle. You can download that, that's a good tool to use. We didn't talk anything about Pacu in the book just for the simple fact of I didn't want to be redundant of the other book, the one other AWS Pentesting book. And that book is very heavy in Pacu. And so that's a good tool. And then there's especially for S3 if you want to start doing your own tooling, you can use the Boto 3 library that's built for Python and just start taking scripts down from the AWS documentation that you can find publicly online through AWS and start actually creating your own S3 bucket scripts. So yeah, Metasploit is a good one. Pacu is a good one. And then just using some of the more traditional ones but starting to get in start learning to do scripting yourself and start scripting using various libraries that are more directly related to AWS.
Joe [00:22:40] Nice. I think the three areas we touched upon are basically specific to AWS. Any best practice in general for cloud-based Penetration Testing to maybe help people hardening the systems against attacks?
Jon [00:22:51] Yeah. So in the book we mentioned a term that I coined over the course of the past year, year and a half called Functional Testing. And I gave a talk again about this last week and it touches on it a little bit in the book. And we're actually going to be expanding on the idea in future books. It's called Functional Testing. And that is where we as Pentesters need to actually stop thinking about pentesting the cloud in a traditional way. Like you were mentioning with tools, what pentesting tools can we use? A lot of the tools that we use are actually ineffective for assessing the cloud. That's why I mentioned you can use Metasploit. You can use Nmap and things like that but really get into scripting because that's where you're going to start formulating your own tools that directly relate to the cloud. But the other thing is, too, is that as a pentester assessing the cloud we actually need to start thinking outside of the box. And what that means is that we actually need to start looking at this as like a Quality Assurance Engineer. And we need to actually ensure that we're getting access to the consoles of the cloud platforms that we're accessing. So in this instance for AWS we need to actually ensure that we're getting access to the consoles of the organizations that we're assessing and actually start going through and start looking at the custom roles that are made, the custom groups that are made, the custom policies, see what they are tied to, see if there are any issues there. And then we also need to start looking at not the individual services at a very, very granular level, but at least look at the services and how they have them configured through the console. And we'll discover a lot of issues there where like I said, you might see that monitoring isn't enabled on a service. You might see that deletion protection's not enabled on a service. You might see that there's a sudden uptick in EC2 servers from a log. And, you know, that might be a cause of concern because you're saying, “Okay, why are you scaling this fast? Like who's authorizing you to scale this fast? Who knows that you're scaling this fast? And what are you doing to secure yourself if you're scaling this fast?” So we really have to take our offensive Security mindset as pentesters, but really start to look at it at a more holistic cloud Security engineering perspective and really meet those too at full peak to properly, effectively, and efficiently assess cloud environments for our clients.
Joe [00:25:11] Now, will this apply to a company that's developing software? They're trying to shift activities left in the software development lifecycle? So should a QA person or a test to be somewhat aware of these, to begin with, if you were in a Scrum Team to have a checklist maybe of, “Okay, we're running in the cloud. So at a high level before we actually have to get to a security team, let me make sure I have one through 10 of these checked off on my checklist here.”.
Jon [00:25:33] In theory and in a pipe dream absolutely. Does it work like that? No, because, I mean, that takes up time, resources. And then also the thing is, too, is that we want to kind of like always define a title to it and figure out like, “Okay, if we're going to have this person do this, we need to just make that a title and have them be that one person.” And then, you know, that's a…but then you're like, “How do you define that job title? How do you define what that person needs to do? And how do you define what do we pay this person and how much is too much for them to do? Do we need more than just one?” Because that in of itself is its own skillset but yeah, I mean, in a perfect world absolutely, absolutely.
Joe [00:26:11] Okay, Jon, we just skimmed the surface of the book, so I'm really excited to get people to learn more, get their hands on the book. Before we go, though, is there one piece of actionable advice you can give to someone to help them with the Security AWS testing efforts? And what's the best way to find, contact you or get our hands on your book? The AWS Penetration Testing book.
Jon [00:26:29] Yeah. So if you're looking to get into cloud pentesting or you are actively pentesting for a company and you're assessing organizations, cloud environments, the best thing that you can do for yourself, for your company, and for those that you're working with is to ensure that you think outside the box and you look at especially AWS from a different perspective. You cannot just run your traditional five-step, six-step process, whatever you want to call it on the cloud. You actually really need to start thinking about it in a holistic cloud Security engineering kind of way with a pentesting mindset. So as we've talked out throughout the show that means that you're going to really need to start looking at it in a different light. And if you want to find out more about how to do that, you can pick up my book AWS Penetration Testing. It's available currently on pachube.com and Amazon.com. I think it's about forty-five dollars for the paperback copy right now, which I highly recommend that you get the paperback copy instead of the E-reader version just because you can have something tangible on you that you can make notes, things like that. And then if anybody listening would like to reach out to me, I'm on LinkedIn at my name Jon Helmus. That's J-O-N H-E-L-M-U-S or I'm also on Twitter at moos1e@moose. That's M-O-O-S-1-E underscore moose.
Rate and Review TestGuild Security Podcast
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page. Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.
