||Add To My Personal Library
November 4, 2005
Vol.27 Issue 44|
Page(s) 12 in print issue
Simplifying IDS/IPS Log Data Analysis
A Look At Ways To Sift Through & Find The Data That Matters
In an effort to proactively monitor and protect their networks, most companies have some sort of intrusion detection system or intrusion prevention system deployed. Many of those companies also make the mistake of believing that the IDS or IPS is some sort of automatic silver bullet that will safeguard their network out of the box. The truth is that IDS products are primarily passive and require human intervention to interpret and respond to suspected issues. Even IPS products, which by default are more proactive than an IDS in responding to attacks, require a good amount of oversight and monitoring to ensure that they are properly reacting to suspected attacks and that there isnt legitimate traffic being rejected on accident.
So Much Data, So Little Time
At the core of IDS and IPS tools are the volumes of log data they accumulatehundreds, thousands, or in some cases even millions of entries for alerts detected and/or actions taken to prevent suspicious traffic from affecting the network. For many network administrators, this log data represents somewhat of a Pandora's box. It holds the keys to understanding the threats to the network and providing better defenses against them but only for those who have a firm grasp of what normal network traffic looks like, so that they can identify anomalies, and a thorough understanding of the IDS or IPS product being used.
Ben Rothke, a senior security consultant with ThruPoint and author of Computer Security: 20 Things Every Employee Should Know, says that the log information can be classified into five categories, with Level 1 being simply informational log entries and Level 5 being critical. A big enterprise can have tens of millions of log entries a day. Attempting to store and correlate that data is a huge job. [Analyzing] only the Level 4 and 5 data is the most beneficial way to go.
Rothke also adds, More importantly, companies need to have trained staff and security analysts that can take the log data and interpret it. The biggest mistake companies make is to think that once they install an IDS it is all plug and play and that they dont need someone to manage the system.
Handling IDS/IPS Log Data
Managing the vast volume of log data IDS and IPS devices generate is a daunting task. Much of the data may be meaningless, but some of it may be critical, and it is up to you to determine which is which. The primary keys to successful IDS/IPS log management include:
Educated staff. As Rothke points out, the technicians administering the IDS/IPS must have a thorough understanding of both the network and the IDS/IPS device itself and how to understand the log entries.
Management backing. Management backing is a key component of virtually every aspect of IT, and IDS/IPS is no different. Management has to support IDS/IPS logs as a priority or analyzing them will just get shoved to the back burner.
Adequate budget. The budget to purchase and deploy an IDS/IPS system must be sufficient to buy all of the sensor hardware and adequate server and hard drive capacity to make sure the product runs optimally. The budget should also have room for any required training for the network staff to properly administer the system.
Documented processes. You should comprehensively and clearly document the process for configuring and administering the IDS/IPS system and analyzing and responding to log data.
Integration into incident response. In order for the IDS/IPS system and log data to be valuable, your company must review it and act upon it. The process for responding to IDS/IPS log data should integrate smoothly with the companys documented incident response plan to make sure that suspicious or malicious attacks identified by the IDS/IPS are handled appropriately.
Using Tools To Automate Log Analysis
Using a SIM (Security Information Management) or SEM (Security Event Management) tool can help to automate some of the log analysis. Products such as netForensics nFX Open Security Platform or ArcSights ESM (Enterprise Security Manager) are great tools for analyzing not only IDS/IPS log data but logs from other sources as well and distilling the information down to what administrators need to see.
Reflex Security offers a slightly different approach with its RCC (Reflex Command Center). According to Reflex Securitys Mike Casey, The Reflex IPS does not generate logs but rather aggregates the attacks based on attack type/source, etc. with the ability to drill down. This approach was developed to eliminate the need to review the detailed logs by allowing the security professional to view a network IPS security dashboard that aggregates the attacks into a summarized presentation for quick review and response. This approach is similar to that of the SIM/SEM products, but the RCC does not have the ability to aggregate or correlate log data from firewalls, routers, or other network devices at this time.
No Substitute For Knowledge
Whether you review the IDS/IPS logs manually, implement solutions such as the Reflex RCC that dont rely on logs, or deploy a full-scale SIM/SEM product, it is important to realize that there are no silver bullets. All of these products or solutions are simply tools to help automate the process and make it more efficient. They are not plug and play solutions that will magically protect your network just by turning them on. Choose the tools that will provide the most value in helping to analyze the overwhelming amount of data available about the network, but understand that, in the end, a knowledgeable, rational human must still configure, administer, and maintain whatever solution you choose.
by Tony Bradley
Tips For IDS/IPS Success |
IDS and IPS products generate tons of log data. Depending on the configuration of the network, what other security measures are in place, and where the IDS/IPS sensors are, attacks may be detected almost constantly. To sift through the data and make sense of what matters, follow these tips:
Categorize log entries: Log entries should be categorized by type. By assigning a priority to log entries, you can focus your attention on only the most critical or urgent entries and not waste time reviewing harmless information.
Normalize data: If you are viewing entries from different applications or devices such as firewalls, IDS or IPS systems, or other network equipment, the log data may not be apples to apples. In order to correlate and compare data, it first must be normalized so that you are sure you are comparing and contrasting the same information.
Implement a SIM: A SIM (Security Information Management) or SEM (Security Event Management) tool can help to automate the process of correlating and comparing data from various sources and do much of the grunt work of reviewing the log files from IDS/IPS sensors. Like IDS and IPS solutions, though, SIM/SEM products still require knowledgeable administration to provide any real value.