When investigating a potential attack on cloud services, Daniel Bohannon often has to deal with detailed Amazon Web Services logging, an issue that can allow an attacker to hide in an avalanche of data.
While AWS typically produces only a single event for each programmatic API call, accessing the management console via the web leads to an exponential increase in events, says Bohannon, a principal threat researcher at the cloud identity services provider Security Permit. In one session, for example, 81 clicks on users, workloads, and roles resulted in 5,370 events recorded in the AWS log file.
The sheer volume of events causes so much noise that it becomes difficult to determine what a user is actually doing in the AWS console, the threat researcher says. For this reason, Bohannon and fellow threat researcher Andi Ahmeti plan to release an open source tool at the Black Hat Asia conference in Singapore to help security managers and incident responders consolidate cloud log events into a record of user actions.
“The idea is that you input your raw logs – and we do [show] 100% of those raw logs, but then we enrich the data on top of that, which… contains all the information about the events that led to those events,” he says. “We all have the complete information about the signal, which… .. it has the summary, the labels, all that kind of stuff.”
Historically, the volume of data in log files has made it difficult to determine the events that led to a compromise. Sometimes, the problem is that the cloud service doesn’t log enough events to determine what’s happening, like last year’s criticism Google Cloud Platform failed to record adequate data when a user accesses a storage instance.
In other cases, the specific ways in which cloud services communicate information to their customers can lead to a lack of visibility, especially considering the differences companies face when using multi-cloud. More than half of companies have open doors that undermine their security and take about two months to patch the vulnerabilities.
AWS produces an avalanche of events
For businesses using Amazon Web Service, the number of events produced in a log while using the web console can be significant. Simply clicking on a list of users produces 18 events for just three identities, and that’s a mild example of AWS’s verbiage, Bohannon says. A user who clicks on the AWS console to view users in the identity and access management (IAM) console will see more than 300 events produced in the logs for CloudTrail, Amazon’s auditing feature.
“That number can actually go up to 100, 300, or even 700 events, depending on your web browser settings, all with just one click,” he says. “So at a fundamental level, every single action you take produces at least one event, but often there are dozens or sometimes even hundreds of additional events associated with it.”
The researchers’ open-source tool, Cloud Console Cartographer, aims to turn the list of events captured by CloudTrail into a short timeline of actions taken by the user. The program adds comments to the cloud log that classifies a series of captured events into signals: actual user actions.
“We want to show all of our mapping to remove as much noise as possible, but still retain all the raw events,” Bohannon says. “So anything that isn’t mapped is great, it’s still evidence and defenders can make the most sense of it.”
No plans for other clouds
The open source Cloud Console Cartographer tool, which will be available on GitHub, produces an enriched log of events and has a web interface that lists signals in a table. They have currently created more than 240 rules to classify collections of events into user actions, i.e. signals, which will be used to enrich log files.
The two threat researchers plan to continue working to expand the number of classifiers and hope others will do the same.
Bohannon and Ahmeti could move on to developing the tool for other cloud platforms, but because different cloud providers have different ways of logging, what works for AWS won’t work for Microsoft Azure or Google’s cloud platform, they say. AWS is very verbose, but Azure is the opposite: Its logs are so terse as to be useless, Bohannon says.
“I feel like each platform, each cloud platform presents unique challenges that will have to be addressed in different ways.” he says. “So in the future we may find that we can integrate other cloud platforms, but [for now] we at least have plans for additional AWS-related GUIs that we will work on after the initial release.”