SIEM: The Complete Guide

Learn all about Security Information and Event Management

Need a crash course on SIEM? In today’s demanding cybersecurity landscape, security information and event management has become an essential part of every security operations center (SOC) toolbox, helping businesses across the globe protect themselves from cyber threats.
Enterprises have been struggling with complex SIEM deployment for over a decade now, as deployment for traditional SIEM platforms can take as many as 6 months before all essential applications and integrations are in place. In this guide, we’ll share all the strategies and concepts you need to know, no matter if you’re a beginner or a seasoned cybersecurity veteran.
In this guide, you’ll learn:
  • The basics of SIEM and how it operates.
  • What’s changed over a decade of SIEM evolution.
  • Use cases for beginners and seasoned veterans alike.
  • Things to look for when buying a SIEM solution.


How does SIEM work?

All about data sources, SIEM architecture, correlation and analytics in SIEM

Learn More


Evolution of SIEM

Challenges in traditional SIEM, SIEM developments, benefits of next-gen SIEM platforms

Learn More


Incident Response and Automation with SIEM

About incident response, the benefits of security automation and orchestration

Learn More


Advanced SIEM Capabilities: UEBA and SOAR

Beyond alerting and compliance, detecting threats never seen before

Learn More


SIEM Use Cases

Identify patterns, connect the dots faster, detect unknown threats

Learn More

CH 01

The Basics of SIEM

Businesses are becoming more digitized, and this shift has caused an explosion in the amount of data being generated. Every component in the IT landscape (viz. networks, security devices, infrastructure, endpoints, antivirus, applications, databases) generates logs for various events. In our interconnected world, they are exposed to an increasing number of threats, making the data in these logs essential to organizational security.
That’s why organizations today are searching for ways to proactively prevent security incidents, rather than reacting to attacks after the damage is already done.
However, the tools used to monitor network security generate an unprecedented amount of data themselves, to the point that critical alerts can easily be missed.
The goal of any security plan is to predict the who, what, when, where, and how of an unknown threat and thus avoid risks that lie ahead. The key to doing this is the ability to aggregate all logs in a single repository and process them in real time. This makes it possible to identify and respond to events as they occur.

What is SIEM?

Security information and event management (SIEM) is a security technology that aggregates log data from multiple sources, identifies suspicious activity and takes appropriate action. The most basic and important function of a SIEM platform is to centralize security notifications from multiple security tools (like firewalls, IDS/IPS, wireless access points, antivirus software, etc.) that each generate their own alerts every day.
A SIEM solution helps you collect all these alerts in one place. It creates a single set of reports in a centralized system for generating notifications. This is also referred to as a log aggregation system.
Gartner defines SIEM as follows:
Security information and event management (SIEM) technology supports threat detection and security incident response through the real-time collection and historical analysis of security events from a wide variety of event and contextual data sources. It also supports compliance reporting and incident investigation through analysis of historical data from these sources. The core capabilities of SIEM technology are a broad scope of event collection and the ability to correlate and analyze events across disparate sources.
While you may be familiar with SIEM software as a tool to help with regulatory compliance, SIEM offers much more than this. It can form the foundation of your security strategy, a fact that more and more organizations are becoming aware of.

As Gartner explains, SIEM software is a class of security software built on two previous generations of security technology:

  • Security information management (SIM): A first-generation security solution that generates a report based on log data it retrieves and analyzes.
  • Security event management (SEM): data enrichment, threat validation
  • Correlation and analytics: fA second-generation solution that analyzes events and log data in real time to provide event correlation, threat monitoring and incident response.

Why do companies need SIEM?

log management
The first and most important reason companies need to implement a SIEM solution is to monitor logs and generate reports when suspicious events occur.
Most organizations generate far too much data for their security teams to handle manually. A SIEM platform makes security more manageable by sifting through all the data and alerting the security team to the most critical security issues.
SIEM also has a crucial role to play in regulatory compliance, as with HIPAA and PCI DSS. Many organizations use SIEM to protect sensitive data and meet compliance requirements.
A SIEM platform receives logs from multiple sources and generates a single report from these events; since compliance reporting often requires centralized logging capabilities, a SIEM solution is extremely valuable in this regard. Without such a system, organizations would be faced with the daunting task of retrieving logs from each individual source by hand.
SIEM Compliance
why siem
Apart from log monitoring and compliance, a SIEM solution is extremely helpful in incident detection and management. While SIEM software may not be able to stop an attack on its own, it can communicate with other security tools in the network, such as a firewall, IDS/IPS systems, and so on, directing them to block any activity found to be malicious.
Additionally, SIEM solutions can also be integrated with threat intelligence providers, enriching log data enriched with information from threat intelligence feeds.
This not only helps in detecting threats that might otherwise be missed, but also in taking action to prevent attacks before they are even attempted.

SIEM Market Share

According to Gartner, SIEM is the fastest-growing segment of the security solution market.
At about 2.5%, SIEM accounts for a small but increasing proportion of global security spending. In total, organizations spent about $2.4 billion on SIEM out of a total security spend of about $98 billion in 2017, but this is expected to rise to $3.4 billion in 2021, according to Gartner.
Traditional SIEM offerings tend not to offer many features beyond those described above. Attackers are constantly developing new kinds of threats, making it difficult for older platforms to keep up. Traditional solutions also rely on relational database technology like MySQL for storage. These technologies aren’t able to efficiently handle the volume of data generated in modern SOCs.
Keep reading to find out everything you need to know about DNIF and other modern SIEM platforms.

Next Gen SIEM

Organizations need a big data platform that can parse and store data in a variety of formats. For detecting modern, complex threats, technologies like user and entity behavior analytics (UEBA) are also key.
Next-generation SIEM platforms can do everything a traditional platform can, and much more. They offer integration with multiple security tools, full security orchestration, automation and response (SOAR) capabilities, and machine learning techniques to identify patterns for advanced threat detection.

CH 02

How does SIEM work

In this chapter of our Complete Guide to SIEM, you’ll learn what a typical Security information and event management (SIEM) architecture looks like and how SIEM solutions are able to derive relevant insights from millions of log events in real time.
You’ll read about both traditional SIEM solutions and modern, next-gen solutions.
Before diving into how SIEM works, we’ll start with the most important features of a SIEM platform, and then take a look at how these features are implemented in practice.

In this section you will learn:

  • Ingesting and interpreting logs: data collection, ingestion, indexing, storage, management, log retention
  • SIEM and threat intelligence feeds: data enrichment, threat validation
  • Correlation and analytics: from analyzing millions of events to deriving meaningful alerts
  • Security analytics and data visualization: contextual information, reporting, dashboards, advanced security analytics
  • SIEM Deployment Models: self-hosted/self-managed, managed security services (MSS) options

10 SIEM Capabilities At A Glance


Log data aggregation

Collect and aggregate data from multiple data sources, like network devices, security devices and cloud services.


Threat intelligence

Custom data enrichment with external threat intelligence providers.


Correlation and analytics

Connect the dots between related security events to see the complete picture.


Real-time security monitoring

Monitor key metrics and traffic profiles to identify anomalies.



Detect issues and notify incident handlers for further investigation or remediation.



Give your team a unified security overview, making it easier to identify anomalies with threat storylines.


Incident response

Identify and investigate incidents faster by bringing in relevant context and threat information.


Security compliance

Keep your organization audit-ready with comprehensive reporting.


Threat hunting

Derive insights from your log and event data by writing queries.


Security Automation

Detect, validate and respond to threats without lifting a finger, thanks to built-in security orchestration, automation and response (SOAR) functionality.

How SIEM Works

SIEM log data flow

Ingesting and Interpreting Logs

A SIEM at its core comprises of a log collection and storage frameworks, that facilitates log management and analysis.
Data collection and Ingestion
SIEM platforms can collect data from a variety of data sources, such as network devices, servers, applications, DLP/IDS/IPS engines and BYOD platforms. Usually, data collection is performed in four ways:
  • Log forwarding software collects and forwards logs from a device to a SIEM platform. This is known as agent collection.
  • Logs are forwarded directly over HTTP(S) or via APIs.
  • Devices forward logs directly in the syslog format.
  • Event streaming protocols like NETFLOW or SNMP can forward events to a SIEM platform as they occur.
Data indexing and storage
After data ingestion, the log data is parsed and stored in a database. Traditional SIEM solutions typically use RDBMS storage systems; today, in response to their schema and scalability limitations, next-gen SIEMs have replaced this technology with distributed, horizontally scalable architecture and storage setups.
Log management and retention
In most organizations, SIEM platforms must rapidly ingest huge volumes of log data in a variety of formats. Over a period of time, the stored data generally goes through two stages, as determined in a log retention policy:
  • Tiered: Most tiered data has been recently collected. Given its relevance to the current situation, it must be readily accessible, such as via queries. Historical data that may be needed in an investigation must also remain accessible alongside the most recently gathered data. Therefore, these data sets must be stored on storage media with high throughput rates, which improves performance..
  • Archived: Data sets older than the log retention threshold can be transferred to archival storage (using less expensive storage media), as these data are unlikely to be useful for heuristic analysis and investigations.
Next-gen SIEM platforms are already based on a data lake architecture, making it easy to reduce storage costs with ElasticSearch and Hadoop.

Unlimited Scalability With Data Lake

Data storage can be very cost-effective when you use a next-gen SIEM as a data lake, built on ElasticSearch and Hadoop.

SIEM and threat intelligence feeds

Data enrichment
Most SIEM solutions today have the ability to enrich stored log data in some way. This added information may include geographic location, inter-departmental metadata, and much more.
Modern SIEM solutions have provisions to enrich event data in real time with information pertaining to user risk scores and anomalies in usage patterns as well.
SIEM log data flow
SIEM log data flow
Threat validation
Validating threats and alerts is crucial to threat hunting. Traditional SIEM solutions tend to have limited threat correlation capabilities. This means that more manual investigation is required, relying heavily on analysts’ experience and skills. In addition, this approach is time-consuming and can result in a high rate of false positives.
Most SIEMs now are able to augment event data with threat information (in form of Indicators of Compromise IOCs) from third party intel providers.
Given the broad range of scenarios that benefit from automated validation, the use of a next-generation SIEM platform significantly reduces the overall time to respond. By providing relevant threat information and focusing on mission-critical events, analysts can investigate more rapidly.

Correlation and analytics

Traditional SIEM solutions rely on pre-written correlation rules for threat hunting purposes. These basic correlation rules are prone to producing false positives and false negatives, because rule-based correlation is geared toward finding known threats and patterns. Pre-written rules cannot adapt to today’s constantly shifting threat landscape, in which APTs and insider threats have become more prominent. The result is unreliable security alerts, as well as missed alerts that may quickly lead to catastrophic incidents.
Data-driven cybersecurity analytics, however, now plays a crucial role in helping vendors and security analysts do much more with their log data. The ability to correlate event data using machine learning (ML) techniques while monitoring user behavior has made it possible for modern SIEM platforms to address risks beyond the scope of traditional, signature-based detection models.

Security alerts and data visualization


Contextual information

Real-time event data enrichment reduces overall investigation times, giving analysts a complete contextual threat timeline in the form of a dashboard featuring drill-down analysis.



Most SIEM platforms feature dashboards, which allow analysts to visualize and interact with data. Security teams can customize dashboards to include information and tools relevant to any given situation.


Security compliance and advanced reporting

Regulatory compliance remains a key driver for SIEM deployment, and SIEM solutions continue to be go-to tools in deriving insights and automatically generating reports for audit purposes.

Traditional SIEM

Traditional approaches to SIEM were expensive, vertically scalable, and RDBMS-based. Creating a complete security management solution required adding tons of proprietary plugins and query languages. Given the large volumes of data involved, custom hardware was necessary to implement such a solution.
With over a decade of evolution and the advent of big data technologies, traditional SIEM solutions no longer live up to the expectations and challenges of modern SOCs. There are many drawbacks associated with the use of traditional platforms in today’s security environments:
  • Integration with non-standard, customized devices and applications requires analysts to fine-tune parsers.
  • Relational database management systems like MySQL have schema limitations that make it difficult to store and access large volumes of data.
  • Enriching event data with contextual information is impossible.
  • Detection strategies are primarily static or point-based.
  • Vertical scalability promotes rip-and-replace architecture, which is expensive.
  • Traditional platforms weren’t designed with ML in mind.
  • Scaling is difficult and expensive.
  • Creating and customizing context-based reports is difficult.
  • Once ingested, data cannot be easily used to derive insights beyond the scope of security.
  • Deployment is complex and time-consuming, with timelines in excess of two months.
  • Options for custom or external event enrichment and data validation are limited or nonexistent.

Next Gen SIEM

Modern SIEM platforms, powered by big data technologies, have overcome these drawbacks. Below, you’ll find examples of the features they include:
  • Real-time threat detection: Malicious activity can be detected as soon as it begins, and appropriate measures can be taken to limit the resulting damage or negate it entirely.
  • Improved decision-making and predictions: The accuracy of modern solutions improves over time by “learning” from wrong decisions. These metadata systems are often further enriched to improve decision quality. Preparing for the future gives businesses a competitive edge, providing a more flexible framework for decision-making and risk handling.
  • Enhanced security posture: Using historical data, organizations can understand what is typical and what isn’t. By detecting these outliers and prioritizing security concerns correctly, an organization can improve its overall security posture.
  • Extended data storage: Storing more data for a longer period means there’s more time to analyze patterns and proactively plan for future security incidents.
  • Data lake-based storage: unlimited scalability at low cost, with improved fault tolerance and high throughput, is perfect for organizations experiencing issues with centralized data management.

SIEM deployment models

SIEM is not a deploy-and-forget tool. To maintain its effectiveness, any SIEM implementation must be supervised, maintained and fine-tuned over time. Correlation rules must also be periodically updated in response to changes in the security landscape.
There are two key questions that determine the best SIEM deployment model for a given environment:
  • Where do you want to deploy your SIEM platform? (Cloud, on-premise or hybrid)
  • Will an in-house security team be monitoring and managing your SIEM solution?
Two common deployment models are available, depending on how you’d prefer to host and manage your SIEM solution:

Self-hosted, self-managed

In this scenario, an in-house team takes care of both deployment and management. Purchasing, hardware provisioning and data archival are also typically handled internally.


Managed security services (MSS)

This is the most widely adopted model. Purchasing and provisioning hardware for are handled internally, while. other activities (deployment, event collection setup, correlation, analysis, alerting and dashboards) are handled by your MSS partner.

CH 03

The Evolution Of SIEM

While the concept of SIEM itself is nothing new, recent years have produced exciting developments in the underlying technologies.
These developments have improved the range of threats that modern SIEM platforms can address, as well as making SIEM more affordable and accessible for organizations of all sizes.

In this chapter you will learn:

How today’s solutions address the drawbacks of older ones. You’ll also read about some of the new technologies being incorporated into modern platforms, and how you can benefit from them.

Problems Associated With Traditional SIEM Platforms


Lack of integration

The volume and variety of data that today’s SOCs handle can be a challenge for traditional SIEM solutions. Data often arrives in inconsistent formats, since SOCs often use products from numerous vendors.

Additionally, modern security teams are increasingly leveraging external data sources as a supplement to their internal data.
Many traditional solutions lack the integration support they need to make use of this external data, forcing analysts to spend time manually correlating separate data sets.


Reliance on human intervention

One of the most significant drawbacks of traditional SIEM systems when compared to their modern counterparts is their reliance on human intervention: they can ingest data and generate alerts, but they stop short of taking any action themselves.

This places a tremendous burden on the shoulders of the SOC team, who must manually respond to each and every alert.
In today’s high-volume SOCs, analysts working with traditional solutions can quickly find themselves buried under a mountain of alerts as a result.


No dynamic profiling

In today’s constantly shifting security landscape, signature-based detection isn’t enough. Relying too heavily on these strategies leaves networks open to more sophisticated attacks, including advanced persistent threats (APTs) and exploits involving zero-day vulnerabilities.

Modern SIEM platforms incorporate statistical analysis and machine learning techniques are applied to establish baselines for what constitutes normal activity.
When the software detects unusual activity, like an unusually high login frequency for a particular account, it generates an alert. These baselines are also continuously updated to avoid generating an unacceptable number of false positives.


Limited capacity for validation

In an SOC using a traditional SIEM solution, each alert requires the attention of an analyst, who must investigate manually to determine whether a threat is present at all (and what to do about it, if one exists).

Over time, that translates into hours and hours of analysts’ time spent investigating threats that don’t exist, thereby increasing the time it takes for the SOC to respond to real threats, too.
To help make security teams more efficient and effective, modern SIEM platforms can do some of the investigative work on their own. By taking advantage of external threat intelligence sources, like lists of malicious domains and file hashes, the software can validate many threats independently. This reduces the amount of time spent manually investigating threats and enables SOC staff to respond more rapidly to confirmed threats.


Inefficient resource use

In light of the standard set by modern SIEM platforms, traditional platforms lead to inefficient resource use at a number of levels.

In a traditional setup, analysts are regularly distracted from legitimate threats by false positives. Legitimate threats, in turn, must be validated and responded to manually.
This approach to security puts constant pressure on analysts, who must work quickly and without stopping to prevent a backlog of alerts from forming. By contrast, next-generation SIEM solutions are able to reduce the pressure on analysts, ensuring that the SOC’s resources can be focused where they’re needed most at any given time.

How Has SIEM Evolved

Since its introduction, SIEM systems have come a long way. Today’s platforms incorporate technologies like SOAR (security orchestration, automation and response), machine learning and artificial intelligence, reducing the amount of manual work SOC teams are forced to do and giving them the tools they need to respond effectively to a variety of threats.
Significant advances have also been made in the approach SIEM solutions take to data storage and retrieval. While older solutions used relational database technologies like MySQL to store ingested data after parsing, modern solutions have shifted to using data lakes.
Data lakes can include data in so-called “semi-structured” formats, such as XML and JSON, making them more flexible than relational databases. Data lake storage is also more scalable, making it easy for organizations of all sizes to get started using SIEM without investing in expensive dedicated storage hardware.

Download Ebook - Why you need a next gen SIEM

The advantages of next-gen SIEM platforms:

If you’ve run into any of these issues mentioned above personally — and even if you haven’t — you’ll be pleased to know that today, they’re little more than relics of the past.
Next-generation platforms can not only consolidate information from both internal and external sources in a variety of formats, but also begin analyzing it independently. This means that alerts can be validated and prioritized automatically in many instances, saving analysts time and effort.
Advanced technologies like UEBA make it easier to detect both known and unknown threats; paired with SOAR functionality, the software can act automatically to limit the damage an attacker is able to cause.

CH 04

Incident Response and Automation With SIEM

What is incident response?

Incident response includes a variety of activities whose purpose is to detect, investigate, and handle security incidents in an organization. Incident response activities can be broken down into two main types: reactive and proactive.

Reactive response

Reactive response activities are those begun after the presence of a threat has become apparent. Depending on the progress and nature of the incident, this may include limiting the damage an attacker can cause, denying an attacker continued access to the network, eliminating malware, restoring from backups, patching vulnerabilities, and so on.

Proactive response

Proactive response activities are those begun before the presence of a threat is apparent. In contrast to reactive response, proactive response is about actively searching for signs of a cyberattack or the means by which an attack could occur. For example, network traffic can be monitored to discover the presence of a backdoor before an attacker is able to exfiltrate sensitive information.

The SOC’s tools in practice

With an extensive array of tools at its disposal, the ability to apply the right tools at the right time is critical to success in an SOC. A SIEM system is one of these tools, keeping analysts informed by aggregating information forwarded from multiple sources as it arrives. This includes internal sources of information, like firewall logs; as well as external sources of information, like threat intelligence data from third parties.
Based on this information, the SIEM system generates alerts to let security staff know about suspicious activity that may require their attention. In a modern SOC, it is common for analysts to be faced with a high volume of alerts, which makes triage essential. In this context, triage is the process of assessing and prioritizing alerts based on their severity. Some alerts will also be false positives — identifying them as such early on is key to ensuring efficiency and keeping valuable SOC resources available to handle any critical incidents that may arise.

What is case management?

Your SOC has identified a security incident. Now what? Case management is the next step. In case management, security teams gather information about an incident that will allow them to respond as effectively and efficiently as possible. In a modern SIEM system, the team can open a case as soon as they’ve identified an incident. Then, analysts can pull information about the incident from a variety of sources to develop a comprehensive overview.
If there are several ongoing incidents, SOC teams can also use case management features to prioritize incidents and track the status of each one. This ensures that analysts are able to coordinate their efforts to address the most pressing issues first, before handling non-critical incidents.

Tying It All Together: Incident Management and Collaboration

In today’s demanding security environments, information alone isn’t enough, and that’s where SOAR — security orchestration, automation and response — comes in. SOAR bridges the gap between information and action, giving SOC teams the means to assess security incidents and handle them. To see how this works, we need to take a closer look at the components that make up SOAR.


The first component of SOAR is orchestration. Orchestration involves retrieving and/or generating data about a potential incident in order to evaluate it:

  • Some of the needed information will typically already exist locally, like log entries ingested by a SIEM solution.
  • Some information may need to be retrieved. For example, connections to suspicious domains can be verified against prepared lists of known malicious domains.
  • SOAR tools can also generate information themselves, such as by running a suspicious executable in a sandbox to observe its behavior.


Automation and response

If orchestration confirms the existence of a security incident, a SOAR tool can often take action without human intervention. Using sets of rules called playbooks, a SOAR tool matches the incident to one or more rules and takes the action called for by those rules.

  • If a malicious file is discovered on an endpoint, for example, the endpoint can be automatically isolated from the network before any damage occurs.
  • If the file arrived as an email attachment, the system can delete or quarantine other emails from the same address.
  • If a user downloaded the file from a website, the system can block access to that website indefinitely.


Automated alert handling

In traditional environments, analysts were usually forced to manually triage and investigate every alert generated by the SIEM system. Now, systems with integrated SOAR capabilities can automatically triage alerts.

By correlating internal and external information, false positives can often be sorted out automatically, greatly enhancing efficiency in SOCs that deal with high volumes of alerts. Validated security incidents can be forwarded to case management with more contextual information in place, so security staff can spend more time responding to incidents and less time poring over pages of log data.


Journaling and record-keeping

Modern SIEM solutions can keep records of every investigation from start to finish. These records can become a historical reference for your organization, providing an overview of the types of incidents encountered previously and how they were handled.

Besides functioning as the SOC’s own self-writing history book, SIEM records can be used as part of a regulatory compliance strategy.

CH 05

Advanced SIEM Capabilities: UEBA and SOAR

User and entity behavior analytics (UEBA) and security orchestration, automation and response (SOAR) are two new categories of security solutions prevalent on the SIEM market right now. These technologies are powered by machine learning and artificial intelligence to identify anomalies, detect risky behavioral patterns and automate response actions.


UEBA involves observing the normal conduct of users and entities, and detecting any anomalous behavior by looking for deviations from established, “normal” patterns. UEBA evolved out of UBA (user behavior analysis), which was limited to analyzing user activity. Soon, the need to apply behavior analysis to security events became too great to ignore, which gave rise to UEBA solutions.

Advanced Applications of UEBA

While most SIEM platforms offer some basic functionality in this area, traditional solutions are limited in their ability to account for varying patterns of behavior from one attacker to the next.
Fortunately, the UEBA features of modern SIEM solutions are markedly more effective at addressing these threats. Using statistical models powered by machine learning and artificial intelligence, next-generation SIEM platforms can establish dynamic baselines for what constitutes “normal” behavior.
When the software detects a deviation from those baselines, an alert is generated, so that the security team can investigate and take action. Below, you’ll find a few examples of UEBA features and their applications:

Time-based profiling

One application of UEBA involves detecting login attempts outside of specified office hours. If some users in an organization can only be expected to log in within a given period, and a login attempt is detected outside that period, this may indicate that an insider is trying to perform a suspicious activity.

It is, however, important to keep in mind that an alert produced by such activity could also be a false positive, so dynamic, constantly updated baselines are essential to making UEBA efficient.
In the case of an organization with offshore clients, a user logging in at 4 A.M. may be considered a normal event. Traditional SIEM tools that with lacking UEBA support might assume that any login at that time is suspicious, generating spurious alerts and distracting analysts from other tasks..


Monitoring login/logout frequencies

By monitoring how many times users log in and out in a given time period, a UEBA-equipped SIEM platform can identify behavior likely to be associated with unauthorized account use.

Using a statistical model, the software learns how frequently users typically log in and out, and this serves as a baseline for detecting unusual behavior. For example, a user who wishes to gain access to confidential information or protected resources may be able to obtain the account credentials of a more privileged user. The extra logins from the unauthorized user will be detected, generating an alert..


Monitoring dormant user accounts

Sudden activity on an account that has not been used in some time may also indicate an attempted attack. If the credentials to an old or disabled account with root privileges are leaked, for example, an attacker may attempt to use the account to infiltrate an organization’s network.

Even if the login is unsuccessful, as in the case of a login attempt on a disabled account, the resulting alert from the SIEM platform serves to notify SOC staff that an attack may be in progress..

Security Orchestration, Automation and Response (SOAR)

Security orchestration, automation and response — or SOAR, for short — refers to a range of powerful technologies that streamline the incident response process and help you make the most of your security data.
Integration with third-party threat intelligence providers like VirusTotal and Kaspersky gives you the context you need to make informed decisions, and automation tools save your analysts time by responding to many alerts without requiring human intervention. Need a crash course on SOAR?

Download Ebook - What is SOAR and why you need it?

SOAR + SIEM: Integrated Solutions

One example of an integrated solution is DNIF, a unified SIEM solution based on modern data lake technology. With built-in SOAR and UEBA functionality, there’s no need to spend time and money buying and configuring plugins to complete your security toolkit. To better understand how an integrated solution like DNIF works, let’s first take a look at a typical security pipeline.
Existing Cybersecurity Pipeline
A cybersecurity pipeline starts far before an event is classified as a threat — in fact, it starts right where events are received. Response is at the very end of the pipeline. Since most events don’t indicate the presence of a threat, relatively few events actually reach the end.. A typical pipeline within a SIEM system might look like this:
  • Collection: events are received using various protocols
  • Parsing: events are broken down and entered into fields
  • Enrichment: contextual information is added to events
  • Indexing: events are stored in the database
  • Correlation: similar events are connected and analyzed
  • Validation: event details are checked across multiple sources
  • Response: action is taken to counter a threat
Many variations on this pipeline are possible. For example, vendors may choose to add stages or split one stage into several stages. Even so, the core functionality remains more or less unchanged. Next, let’s examine the phases that benefit from SOAR to see how it affects the process as a whole.

Enrich Events and Add Context

The enrichment phase adds context to events, which makes finding correlations between events easier and more productive.. Adding geographical information and whois records are common means of enriching data. However, much more can be done with SOAR:
Data enrichment
Advanced examples of event enrichment include:
  • Layering a user’s UEBA score on top of Active Directory or proxy events
  • Extracting and adding IP context from AD, and adding it to proxy events
  • Aggregating third-party threat intelligence and enriching events with context
  • Adding user attributes extracted from AD to database users
Adding context makes correlation more productive, enhancing the capabilities of your SIEM platform.
SIEM log data flow
Using multiple data sources for enrichment
SIEM log data flow
Validating cyber threats with internal and external data sources
Validate Threats Using Lookups
Previously, correlation rules were limited to simple models and groupings. There were no opportunities to automatically validate correlation results or incorporate third-party threat intelligence. Today, SOAR can also validate correlations, ensuring that the right decisions are made in the response phase.
By calling on external sources to validate decisions and add context to correlated outcomes, SOAR makes more effective validation systems possible. Examples of ways threats can be validated include:
  • Validate domains / IPs / file hash etc using remote threat intelligence providers.
  • Validate and contextualize event details with additional fields from either ITSM or Active Directory.
In the validation phase, SOAR reduces investigation time by removing the need for manual checks that differentiate threats from false positives.
Trigger Outbound Events and Respond to Attacks
Response is the last phase in the event pipeline. Any event reaching this point has become a confirmed alert that requires action. Cross-verification of the threat has already been performed, so action must be taken to counter the threat.
Traditional systems stop short of this stage, simply raising an incident or ticket on a handler’s screen. It is then the handler’s responsibility to validate and respond to the threat manually. This time-consuming approach lets attackers remain in the system longer than necessary, by modern standards.
Examples of triggerable responses include:
  • Raising a ticket in the ITSM application
  • Notify users using email or chat apps like Google Hangouts / Slack etc
  • Trigger a vulnerability scanner on an unrecognized device
  • Choke a process, block disk and network access using an endpoint product
SIEM log data flow
Active response framework triggering actions

Benefits of SOAR

  • Automate data enrichment: SOAR not only minimizes response times, but also enriches generated alerts.
  • Security playbooks reduce manual investigation: Security playbooks can be built based on gathered intelligence or trend profiles based on suspicious IP addresses, domains or even file hashes. Automated threat validation from external threat intelligence providers makes investigation and remediation faster. This helps security analysts investigate every incident while giving them ample time to address alerts.

CH 06

SIEM Use Cases

Security information and event management (SIEM) systems are the cornerstone of organizational security infrastructure. By collecting and analyzing data across an entire organization, SIEM serves as a foundation for a variety of uses and applications.
Below, you’ll find a few cutting-edge use cases for the modern threat landscape that go beyond traditional detection strategies:

Process Whitelisting

Security teams have long been haunted by thoughts of a threat slipping through all their defenses undetected, perhaps leaving some malware active within the network in “stealth mode.” Process whitelisting significantly reduces the risk of such a situation occurring, making it one of the most powerful applications of SIEM.
How it works
Implementing process whitelisting is simple. There are three steps:
  • Scan a clean system’s folders and drives to detect the applications and processes you wish to allow. Executable files detected in this scan will be added to a whitelist.
  • If necessary, modify the generated list. For example, you can remove an automatically detected application from the list.
  • Enforce the whitelist, putting the system into a protected state. An application that attempts to run on your system will first be checked against the whitelist. If it’s not on the list, the application will not be allowed to run.
How process whitelisting helps
  • Monitor process integrity: Discover new processes and identify malicious executables before they can run.
  • Enforce compliance: Maintain and enforce a standard software usage policy across your organization.
  • Control access: Identify and respond to unauthorized access to files and systems via rogue or trojanized processes.
  • DGA Detection

    Detecting the use of a domain generation algorithm is unquestionably critical. Attackers implement DGAs in malware to periodically generate a large number of domain names.
    Malware authors register a small set of the generated domains as means for the malware to “phone home,” potentially receiving updates or relaying information to an attacker. Repeated attempts to connect to invalid domains may indicate the presence of DGA-equipped malware, so detecting those connection attempts can be critical to stopping the spread of malware and preventing the exfiltration of sensitive data..
    How it works
    Detecting DGA-related network activity is summarized in the following steps:
    • Monitor domain names present in log data.
    • Validate or enrich domain name information with external threat intelligence sources for reputation, registration date and domain owner.
    • Correlate the domain information with traffic data to identify anomalies.
    • Generate alerts and/or block connections to flagged domains. This can be done automatically using built-in automated response tools.
    How DGA detection helps
  • Identify compromised systems: Once you’ve detected the use of a DGA, you can easily see which systems have been compromised by malware.
  • Block C&C servers: Modern SIEMs can correlate network traffic with user behavior to identify communication with external entities, such as command-and-control servers used to facilitate communication between infected systems and an attacker.
  • Detect zero-day threats: DGAs are a common component of modern malware. Using statistical analyses and machine learning models, DGA detection can reveal the presence of these threats, reducing reliance on traditional, signature-based approaches to malware detection.
  • Data exfiltration: If not detected quickly, malware equipped with a DGA may be able to exfiltrate sensitive information. Because small portions of information can be transmitted to seemingly unrelated domains, this type of attack is difficult to spot using more traditional solutions.
  • Detecting DDE Injection

    Dynamic Data Exchange (DDE) injection is another technique attackers use to infiltrate environments where Microsoft Office is used. DDE is a technology that facilitates information sharing between applications.
    It is sometimes used to populate Excel spreadsheets with information generated elsewhere, for example. However, the ubiquity of Microsoft Office makes it an attractive target for attackers, who have been known to use DDE as a means of executing malicious code..
    How it works
    • Monitor Sysmon event IDs for new process creation and registry changes.
    • Monitor Microsoft Office processes and correlate the actions of these processes with the above events.
    • Trigger an alert if such processes are found.
    How detecting DDE injection helps
  • Detect phishing attempts: Decrease the risk posed by phishing attempts within your organization. If someone within your organization falls for such an attack and ends up opening a malicious file, you can limit or negate the resulting damage.
  • Monitor file and process integrity:Most SIEM solutions allow you to run automated profilers around the clock to identify suspicious files and processes.
  • User Entity and Behavior Analytics (UEBA)

    Security experts must also be attuned to the risk of insider threats. Profiling user behavior makes it possible to spot suspicious account activity and respond to it preemptively, rather than waiting for a security incident to occur.
    How UEBA works
    UEBA profiling involves monitoring user behavior to detect anomalies. Common components of this approach include:
    • Monitoring failed login attempts.
    • Creating a profiler to monitor and analyze common trends related to successful logins, login/logout frequency and session lengths.
    • Monitoring dormant user activity.
    • Provisioning privileged access and monitoring user access.
    How UEBA helps
  • Detect insider threats: user behavior profiling is key in identifying threats within organisations.
  • Set up dynamic baselines:Creating profilers not only gives organisations a complete visibility into its user activities but helps understand common operating procedures or baselines into their user behaviors. It creates dynamic baselines based on activities profiled over extended period of time and helps in identifying outliers which are more precise than traditional static baseline methods.
  • Threat Intelligence: Data Enrichment and Threat Validation

    Modern SIEM solutions support integration with third-party data sources. Incorporating external threat intelligence helps to refine datasets, adding context and making it easier to validate threats.
    How it works
    UEBA profiling involves monitoring user behavior to detect anomalies. Common components of this approach include:
    • Events forwarded to your SIEM solution are supplemented with threat intelligence from your chosen intel providers in real time.
    • This data can be further used to validate threats and provide additional insight during incident investigation.
    • Using integrated security automation tools, internet resources and users can be blocked automatically after a threat is validated. Alternatively, an alert can be sent to the relevant team for further investigation.
    How it helps
  • Contextualize data: Contextualizing data helps a great deal in identifying what, why and how during investigations. This helps security teams triage alerts faster.
  • Reduce response times:Faster triage means faster remediation, since incident handlers now have a contextual storyline to help them connect the dots.
  • Leverage actionable threat intelligence:Tapping into an expansive pool of indicators of compromise (IOCs) from multiple sources helps you learn from others’ mistakes and detect known threats rapidly, without much manual investigation.
  • Next-gen SIEM and advanced use cases

    Next-generation security information and event management platforms are the product of a decade of evolution in the field of SIEM, and have been built from the ground up with security teams in mind.
    Advanced analytics and automation tools make advanced applications like those described above possible with modern SIEM solutions. By leveraging these tools, you can perform more sophisticated threat hunting, prevent data exfiltration and respond to threats never seen before, even in situations where traditional security tools aren’t able to detect what is going on — let alone raise an alert.

    Download our ebook on - Five most effective SIEM use cases