What can log data do for you?

Written by Lagis Zavros

Organizations today are deploying a variety of security solutions to counter the ever increasing threat to their email and Internet investments. Often, the emergence of new threats spawns solutions by different companies with a niche or a specialty for that specific threat - whether it is a guard against viruses, spam, intrusion detection, Spyware, data leakage or any of the other segments within the security landscape.

This heterogeneous security environment means that there has been a proliferation of log data generated by the various systems or devices. As the number of different log formats increases coupled with the sheer volume of log data, the more difficult it becomes for organizations to turn this data into meaningful business information.

Transforming data into information means that you know the “who, what, when, where, and how” - giving you the ability to make informed business decisions. There is no point capturing data if you do not use it to improve aspects of your business. Reducing recreational web browsing, improving network performance, and enhancing security, are just a few outcomes that can be achieved using information from regular log file analysis.

To achieve these outcomes, it is important for organizations to have a log management process in place with clear policies and procedures and also be equipped with the appropriate tools that can take care of the ongoing monitoring, analysis and reporting of these logs.

Having tools that are only used when a major problem has occurred only gives you half the benefit. Regular reporting is required in order to be pro-active and track patterns or behaviours that could lead to a major breach of policy or impact mission critical systems.

10 tips to help organizations get started with an effective proactive logging and reporting system:

1. Establish Acceptable Usage Polices
Establish policies around the use of the Internet and email and make staff aware that you are monitoring and reporting on usage. This alone is an effective step towards reducing inappropriate usage, but if it’s not backed by actual reporting, employees will soon learn what they can get away with.

2. Establish Your Reporting Requirements
Gather information on what you want to report and analyse. Ensure this supports your obligations under any laws or regulations relevant to your industry or geography.

3. Establish Reporting Priorities
Establish priorities and goals based on your organization’s risk management policies. What are the most important security events that you need to be alerted to?

4. Research your existing logging capabilities
Research the logging capabilities of the devices on your network such as proxy servers, firewalls, routers and email servers and ensure they are producing an audit log or event log of activity.

5. Address shortfalls between your reporting requirements and log data
Open each log file to get a feel for what information is captured and identify any shortfalls with your reporting requirements. Address any shortfalls by adjusting the logging configuration or implementing an independent logging tool such as WebSpy Sentinel.

6. Establish Log Management Procedures Establish and maintain the infrastructure and administration for capturing, transmitting, storing and archiving or destroying log data. Remember that archiving reports may not be enough as sometimes you may be required to go back and extract from the raw data.

Ensure data is kept for an appropriate period of time after each reporting cycle and that the raw data related to important events is securely archived.

7. Evaluate and decide on a Log File Analysis Product
Evaluate log file analysis and reporting products such as WebSpy Vantage to make sure your log formats are supported, your reporting requirements are met and that it is capable of automated ongoing reporting.

Ensure it can be used by business users as well as specialist IT staff, removing the dependence on these busy and critical staff members. Make sure the vendor is willing to work with you to derive value from your log data. Often a vendor that supports many different log formats will have some insight that may help you in obtaining valuable information from your environment.

8. Establish Standard Reporting Procedures
Once a report product has been decided on, establish how regularly reports should be created, who is responsible for creating them, and who is able to view them. Store user reports in a secure location to ensure confidentiality is maintained.

9. Assign Responsibilities
Identify roles and responsibilities for taking action on events, remembering that responsibility is not only the security administrator’s domain.

10. Review and Adapt to Changes Because of the metamorphic nature of the security environment it is important to revisit steps 1-9 regularly and fine tune this process to get the maximum value.