Data Analytics with AWS S3 Logs
What is AWS S3?
AWS Simple Storage Service (S3) is storage for the Internet, designed to make web-scale computing easier for developers.
AWS S3 is a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web.
Why S3 Logging is important?
An AWS S3 bucket can be configured to create ‘access log records’ which contains details on an access request made, such as
- Request type
- Resource worked with
- Turn-around time
- Processing time and date
Server access logs give bucket owners an insight into the nature of requests made by users not under their control.
HOW DNIF HELPS
Customized Dashboards for real time traffic flows.
Easy to implement use cases, as per security, business, and user requirements
Perform AWS S3 log analysis
Monitor all data that resides within your AWS S3 bucket
Index, search and analyze performance and access logs
Generate reports and determine AWS usage patterns
By creating a single dashboard, system analysts or required stakeholders can have a comprehensive view of their storage platform. The dashboard can have multiple widgets, depending upon business or security use cases.
Shared below is the dashboard view, with some useful widgets, that you can create easily within DNIF:
Widgets created for the dashboard above include : - Source based heatmap - Request monitoring - Error monitoring - Frequent visitor list - Total event count
These widgets benefit immensely while performing data analytics for effective threat hunting.
Source Based Heatmap
Geo tracking helps identify good traffic patterns and adjusts your AWS resources, as required, to serve particular regions better. S3 logs can provide actionable insights on where, when, and for how long your users were active in the last 24 hours. This information can prove helpful for cyber security professionals to perform possible data analytics.
Here, we’re monitoring the user activity over the last 24 hours, by keeping track of the HTTP request methods and the count for these method calls.
A wealth of critical information about your network and applications resides in the errors monitored and logged in AWS S3 bucket over the last 24 hours.
You can take a look at the complete list of Error Codes.
Frequent visitor list
List of all the Source IP Addresses that tried accessing a specific bucket in the last 24 hours, and their frequency.
Total Event Count
Count of all AWS S3 bucket logs for the last 24 hours.