Advanced Security Analytics for digital enterprises

My previous blog on Advanced Security Analytics dwelled on a compelling business case for it. Please refer Making the case for Advanced Security Analytics

This sequel goes a couple of levels deeper and discusses why such a platform is extremely potent in addressing holistic cyber security challenges in today’s highly dynamic and insecure world of business, and how traditional point-security solutions simply cannot fit the bill, and produce minimal to negligible ROI for the customer!

The Challenge

According to the HaveIbeenpwned website, thousands of organizations have been hacked despite security point-solutions in place for years on end – SIEMS, firewalls, end-point solutions – you name it! In today’s world, billions of signals are coming in, and data is flying all over the place!

So how does one get one’s hands around this gargantuan problem?

Real-Time Data Enrichment

An Advanced Security Analytics platform that ingests billions of signals – internal and external events, whether it is raw network traffic, raw packet-capture, end-point data, VPN, Proxy, firewall logs, SIEM data, structured data, or log files, ingesting all this data and analyzing it in real-time with context to the data by enriching it with dimensions such as geo-IP location, geo-coordinate specifics, who owns that block of IP, IP vs threat-intel-feeds to see whether that particular IP is malicious or not, reputation of that particular IP etc.  Sound like a leap of faith? Not really.

With the maturing of AI and ML technologies, this is very much a reality to counter and transcend today’s complex cyber challenges. A Risk Score is provided, known and unknown threats are identified based on ingested data, and extracted based on behavioral analysis.

Fidelity

All this data ingestion is of no value, unless one can assure absolutely no loss of data, and integrity is maintained to perform a proper investigation. In other words, fidelity is a must! The need to be able to present cohesively to the Security analyst in one place on the screen to investigate and follow process internally of whatever problem they are investigating is paramount!

Context is King!

If one doesn’t have context at the time when the event is happening or when investigating an incident, gaps remain in one’s story. One loses time and effort. And that’s one more reason why enterprise customers continue to get hacked!

Software platform solutions (not appliance-based) can be installed on a bare-metal machine or in a virtualized environment, and on-premise, hybrid or on the cloud – from an implementation of hardware and software perspective.

The product should not summarize the data. For example, 50 DNS requests should be stored as 50 individual requests, and not by aggregating them.

Advanced Security Analytics Processing

Enriched data enters a real-time stream-processing engine which is distributed across many different machines. There is a “known threats” rules engine. This results in alerts that the SOC team can investigate. In addition, the SOC analysts have the ability to build their own rules, and those can be processed and stored in real time (in Elastic Search and Hadoop). It is used for querying for any anomaly. A subset of the data (features extracted from ML Jobs) is used and stored in Hadoop, where ML jobs are run to detect unknown threats. Looking at the base behavior of assets on the network, any deviation from the baseline gets flagged and associated with a Risk Score, and is exposed to SOC analyst on the dashboard. Secondary analysis explains the anomaly, and the reasons as to what events led to it, why it is anomalous, and then it is classified to reduce false positives. Feedback is obtained to reduce false positives, which goes into a Case database. Alerts go into the dashboard and the investigation starts.

Concerns

Organizations would naturally be curious to know where this is installed, is my data being taken, is it being migrated to the cloud or are you exposing my security.

The simple answer is that it is installed within the security perimeter of the customer’s enterprise. On their own premise, within their own cloud environment or within their perimeter. Data does not leave the premise of the customer environment.

Real-Time Analytics

To summarize, the following functions are performed in real-time:

1. Detection of known threats, utilizing a Rules Engine
2. Data enrichment
3. Query engine that allows the Security analyst to query the databases on what they’re looking for
4. Reports -> some pre-built and some customized
5. Threat hunting
6. Asset mapping

Real-time Asset Mapping

This is a very important process and involves reading the data Source IP address, connecting to a Destination IP address. The source IP is enriched with Active Directory information, and the IP is mapped to that particular machine. So when the analysts are looking at alerts, they know that particular IP has a specific machine name when they’re looking at it in real-time (associating when the event is actually happening). Since machine IP changes are dynamic (DHCP), real-time mapping is very important for the association of machine to IP address to happen. Also, the log in and log off to that IP/machine, and the user logged in at the time the event is happening is mapped.  Real-time mapping of unknown threats is based on behavioral analytics, and provides deep network visibility.

Real-time vs Batch Analytics

SIEM solutions are mainly used as tools that generate alerts. The challenge is that the security analytics run in batch mode. If the SOC analyst wants to investigate something, s/he runs a query, picks up the data and starts enriching it only at the time when the query is sent. That takes time. Real-time enrichment while ingesting the data is a big differentiator. It is always available for querying, and it minimizes the time to get results.

Scalability, Versatility, Productivity

Any such solution needs to be scalable. Why is this important? When customers install, their needs change over time. There are more data sources to ingest, different departments asking for different business requirements. Scalability is very important. Software platforms can help scale horizontally, no matter what and how many data sources, without breaking the architecture. Scale is achieved by spinning additional hardware to accommodate additional ingestion points and additional data to ingest and analyze. Deployment becomes versatile – On premise, on the cloud or in a hybrid environment.

It improves productivity of the Security Department within any org. Customers have the ability for its Level 3 engineers to focus on important tasks they are doing, and build a play-book for their Level 1 and Level 2 to follow standard operating procedures (SOPs)  of investigation before escalating to the next higher level.

SIEM Augmentation

Augmenting existing SIEM product solutions is a benefit of this technology. Since real-time enrichment at the time of ingestion is occurring, it is an upgrade to these existing platforms to be able to perform further analysis. One can query the attributes stored in the SIEM, and end-point data coming into it. So there is no need to ingest end-point data again into the Advanced Security Analytics platform. If we want to get this endpoint data for a particular asset, the SOC analyst can click a button, and it can retrieve attributes from the SIEM database and tag it to whichever asset is being investigated for a holistic view of that asset, and provide a rich visualization layer. This results in quick identification of threats and it should integrate easily with Phantom or other orchestration platforms. It provides visibility to all network events, and stores data in the SIEM without duplication.

Ideally, it can be used as a Threat Hunting Platform to augment what an organization may already have in place. Within their own environment, the SIEM database can be queried. One can go to the specific asset icon, click on it, and get the context menu. There is an entry that says, “Query SIEM for end-point data”. When integrated with the SIEM API, it pulls the information and enriches the asset information.

The true value for customers is that in this type of implementation, there is no data duplication required in a separate system for querying and incurring storage cost and compute. A loosely coupled architecture allows one to take system network traffic into the platform and query that and any other data in a separate database from a separate system e.g. endpoint traffic, without any duplication. Hence this is an augmentation of their implementation, and not a rip and replace.

In conclusion, it is these clear differentiators that create a significant RoI for organizations by employing a solution that provides holistic network and endpoint security across the enterprise.

Arvind Benegal

VP, Digital & Security Practice