In this post, I would like to respond to the SANS post titled, “The Cloud is Evil…“, as authored by John McCash. In reading the post, I felt rather compelled to dispel the myths/misstatements of the article that the cloud is evil for the 5 stated reasons. While I can certainly understand that line of thinking, definitely cannot argue one’s personal experiences that may have lead to such conclusions (I personally have had very lackluster experiences with a lot of lower tier cloud providers), and I am fairly certain he does not think the cloud is really “evil” per se (he actually states/clarifies that later in the post), I still feel a duty to respond with my experiences and facts as compiled from my interest and dedicated development of IR within AWS (and to some extent Azure).
1/18/16 Update: The link actually appears to be down right now. I wonder if it got taken down or if the site as a whole is just experiencing difficulty. Regardless, the page is Google cached here:
*Important Note*: Please PLEASE know that I mean zero personal attack, harm, or ill intent toward the author of the blog post of which this one references! If for some reason it does somehow come off that way, I ask that you please know it’s not the case and instead attribute it solely to my passion toward IR in the cloud and its usefulness and power to responders. I actually hope to hear from the author to discuss this and learn more about his position and how he came to the stated conclusions, as the priority for the DFIR community is education and healthy discussion, not pointing fingers and/or putting each other down. I much prefer someone author a contentious post and just take the time to feed back into the community for its discussion then authoring nothing at all. I am sure I will do the same (if not already done so). So, it is important that, regardless of outcome, we continue to foster participation and discussions within the DFIR community. We’re in this together.
Without further ado…
The referenced post is sub-titled, “A Forensicator’s Perspective”, but I have to say my perspective pretty much disagrees with most of the content here. Maybe that just means I’m not a “forensicator”? Now, I would like to give the benefit of the doubt and approach this from the perspective that maybe the author has no exposure or experience with AWS or Azure which is why such claims are being made without knowledge of their fallacy. However, regardless of the perspective, I’d like to address each of the claims in this article from my perspective and extensive DFIR experience with AWS (and to some extent Azure).
To begin (before the numbered statements/claims), I found the author’s below statement rather incongruent with my experience, in reference to Troy Larson’s SANS DFIR Summit presentation on “Defending A Cloud“:
“The Cloud (Specifically MS Azure), and describes numerous features that can be greatly advantageous to security professionals. Unfortunately, as I discovered upon submitting a question at the end of this talk, most of these features are currently only available to the Provider of the Cloud Service, and not to the Customer?. “
I’m not sure what was asked or discussed, but I know for certain this is not the case for AWS. A ton of features useful to IR are available to nearly anyone/everyone. It would be worthwhile knowing a bit more context for this and perhaps whether or not Azure mirrors the facilitations AWS provides. Though, from my experience, they both have very similar features and capabilities, just different ways of going about them. So, I would be very surprised to learn if Azure provides minimal features useful to security professionals.
At any rate, on to the numbered statements/claims.
“1. You can’t Forensicate the Virtual Hosts.”
- “Even in cases of Infrastructure as a Service, if the customer desires these capabilities, he will typically have to build the associated forensic imaging support directly into his host configurations, rather than leveraging the intrinsic rapid access & direct snapshotting capabilities available to the Cloud provider.”
You absolutely can “forensicate” virtual hosts within AWS, and very easily. In fact, I contend that it is actually easier to acquire and analyze a system within AWS than it is in most any on-premises environment. Specific to AWS, all that is involved in acquiring a system is simply creating a Snapshot of it, which is a bit for bit image of the host. From there, you create a new volume from that snapshot and then attach the new volume (read-only) to your forensic host. It then appears as any other physical drive would to a forensic host, upon which you can go on your merry way with analysis right within the cloud. All of this typically takes only minutes to perform with typically no immediate need to download or transfer data outside of the cloud. No waiting hours for dd to complete and the image to transfer to another system (maybe even a day or so at this point, depending on your network speed). It’s literally as simple as point, click (or execute a couple commands), and begin analysis.
“2. You typically have no access to logs of administrative access to the Virtual Environment.”
- “The customer may, or may not have access to logs of the activity performed using administrative accounts provided to him, and those logs may or may not contain sufficient detail to determine what actions were performed, and from what originating IP addresses.”
AWS, by default, logs API access to/from most (if not all) of its resources, namely to the console and EC2 Instances (systems/machines/hosts you provision and use) which are in the scope of this discussion. You have access to all of those logs (assuming you have the privileges within your IAM role/policy to do so), not to mention the capability to log further specifics about the machine’s performance, actions performed actually within/on the host itself, and a multitude of other items otherwise not easily achievable (or almost unachievable) in many other environments. These systems are in no way black boxes of known activity. I actually content that more logging is enabled by default within AWS than in most on-premises solutions I’ve seen.
“3. You have no way to monitor traffic to and from the Virtual Environment.”
- “Similarly to #1 above, in the case of IaaS, it’s possible to set up netflow or full packet-capture architecture, internal to your virtual hosts, but such configurations are a pain to maintain, scale poorly, and can use significant resources.”
It’s actually very easy to set up netflow captures within AWS – they’re called VPC Flow Logs. In fact, it’s actually scarily simple to the point that I’d recommend alerting on VPC Flow Log creations as an attacker could also harness this capability to snoop on what’s going on within your network (again, with great power comes great responsibility and accountability!). Here is a nice little writeup by Jeff Barr from a while back on just how easy it is to begin monitoring all traffic over an interface (or multiple) with a few simple clicks. It scales, it stores, and uses very minimal resources. I tend to set these up during response to monitor for and identify specific known IOC’s/network indicators, a process of which is often much more arduous or impossible in many on-premises networks and architectures. They’re also great for network/endpoint baselining!
“4. Because of items two and three, an attacker (who might break into your internal network, but eventually be discovered after operating for a typical period of only 18 months or so) need only perform a single, brief, undetected intrusion. He just has to obtain Cloud access credentials or implant a backdoor in your Cloud software, and then get back out, deleting most of his tracks in the process. Subsequently there is only a miniscule chance of the existence or usage of this access ever being noticed, and he effectively has access to all Cloud-stored data in perpetuity.”
Sure, an attacker can compromise your network with elevated privilege credentials, but this is really no different than any other deployment or network. The reason DFIR sustains as a huge business is that attackers do this all the time, and it’s nothing at all unique to the cloud. As to deleting their tracks, I contend that (with the right monitoring and logging) your AWS environment can not only be instrumented to make this incredibly difficult and noisy for an attacker to perform, but also impossible. See Daniel Grzelak’s article on “Disrupting AWS Logging” for some tips on how to use the same tactics to protect against attacker log modification and deletion.
Daniel Grzelak has done a lot of awesome research and blogging on the attacker/PenTester side of the coin for AWS. I highly recommend following his stuff!
“5. The provider’s boilerplate contract agreement will typically absolve them from any possible responsibility for customer damages or losses due to data compromise.”
This is one point on which I totally agree, presuming we are talking about data compromise that occurred due to customer or user fault. However, I look at it in a positive light. It seems reasonable to me to have no preconceived notions that anyone owes me and/or is responsible for irresponsible, uneducated, or otherwise poor behavior in using what they’ve provided.
If the above statement is in reference to data compromise due to AWS operation that is outside of customer/user control, I’m not sure I’ve ever seen or heard that contract clause. So, I would be keen on getting that reference to read myself.
My last point of address is the following quote:
“Cloud providers are not necessarily evil, in and of themselves, but they enable appallingly self-destructive behaviors on the part of their customers. While there are many useful IR capabilities they could make available, there’s an unfortunate lack of demand. The businessperson customers continue to drone the mantra “Cheaper! Cheaper! Cheaper!”, and so the providers acquiesce to these demands, and streamline their offerings, reducing (or failing to develop in the first place) ‘frills’ such as security & IR capabilities.”
To the first sentence, this logic appears to obviate accountability and is essentially (in my personal view) taking the victim stance of “They let me do this.” To me, allowing a customer freedom to do what they want is not a detriment and cause for finger pointing when a poor decision is made; rather, it is the basis of what has made and continues to make AWS and Azure so successful and attractive to consumers. It is this very thing when done properly promotes both self-empowerment and drives accountability. Regardless, I don’t feel that you can make previous points of claiming fault for hand-tying and then go on to claim fault for giving too much freedom to do bad things.
To the second and third sentences of this claim, I’ve worked very closely with the AWS security team in developing our AWS IR service line. AWS provides a plethora (and, if you’ll please excuse me, I dare to say a metric sh*t ton) of incredibly useful tools for security and incident response (the baseline of which are better than most on-premises deployments I’ve encountered). However, they make one thing very clear – AWS does not perform security. Rather, AWS gives you the tools and resources you need to either perform it yourself or make an educated decision not to. If you choose “appallingly self-destructive” behavior, it is just that – a choice you have made. Further, it is a choice you have made despite the tools and resources available to you, not because of them. This is why and where accountability is so important.
Ultimately, cloud providers like AWS and Azure do so very much to enable and facilitate all levels of operation and technical (or non) capability with great freedom. They give YOU the tools to be successful (or shoot your eye out).
I highly encourage everyone to take some time to learn and understanding the offerings available from whichever top-tier cloud provider suits you best. And, if beneficial to your organization, use it and take the utmost advantage. But, whatever you do or do not do, be accountable for your own actions.
“So: Is The Cloud Evil?”
I contend that it is not, not even in the least, with the right provider.