ModSecurity

Logging

Debug Log

Use the SecFilterDebugLog directive to choose a file where debug output will be written. If the parameter does not start with a forward slash, Apache home path will be prepended to it.

SecFilterDebugLog logs/modsec_debug_log

You can control how detailed the debug log is with SecFilterDebugLevel:

SecFilterDebugLevel 4

Possible log values are:

  • 0 - none (this value should always be used on production systems)

  • 1 - significant events (these will also be reported in the error_log)

  • 2 - info messages

  • 3 - more detailed info messages

Note

ModSecurity uses log levels up until 9 internally but they are only useful for debugging purposes.

Audit logging

Standard Apache logging will not help much if you need to trace back steps of a particular user or an attacker. The problem is that only a very small subset of each request is written to a log file. This problem can be remedied with the audit logging feature of ModSecurity. These two directives:

SecAuditEngine On
SecAuditLog logs/audit_log

will let ModSecurity know that you want a full audit log stored into the log file audit log. Here is an example of how a request is logged:

========================================
Request: 192.168.0.2 - - [[18/May/2003:11:20:43 +0100]] "GET /cgi-bin/printenv?p1=666 \
HTTP/1.0" 406 822
Handler: cgi-script
----------------------------------------
GET /cgi-bin/printenv?p1=666 HTTP/1.0
Host: wkx.dyndns.org:8080
User-Agent: mod_security regression test utility
Connection: Close
mod_security-message: Access denied with code 406. Pattern match "666" at \
ARGS_SELECTIVE
mod_security-action: 406

HTTP/1.0 406 Not Acceptable
========================================

You can see that on the first line you get what you normally get from Apache. The second line contains the name of the handler that was supposed to handle the request. Full request (with additional mod_security headers) is given after the separator, and the response headers (in this case there is only one line) is given after one empty line.

When the POST filtering is on, the POST payload will always be included in the audit log. Actual response will never be included (at least not in this version).

Note

Take care when handling audit log data. The files may contain unfiltered binary data received over the network. Such data may be dangerous if not handled properly (e.g. it may contain terminal escape sequences.)

At this time, the audit logging part of the module will log Apache 1.x error messages, on the line below the Handler: line. The line will always begin with Error:. This functionality will be added to the Apache 2.x version of the module if possible.

Note

The audit log subsystem does not log request timeouts.

Note

The audit log entries do not contain the output headers Date and Server. This is because Apache is adding these response headers to the response at the very last moment in response processing making it impossible for a module to get to them.

Choosing what to log by response status code

As of 1.9 ModSecurity supports the SecAuditLogRelevantStatus directive, which is used to selectively log requests to audit log even if they did not cause a warning or an error. It is especially useful to establish a simple communication channel with the applications deployed on the web server. For example, if you know the application always responds with a HTTP status code 500 whenever an internal error or an attack occurs you can configure ModSecurity to log such requests in full:

SecAuditLogRelevantStatus ^5

This directive accepts one parameter, a regular expression that will be matched against the response status code. If there is a match the transaction will be considered relevant, and logged.

Unique request identifiers

If you add mod_unique_id to the Apache configuration mod_security will detect it and use the environment variable it generates (UNIQUE_ID). Its value will be written to the audit log. You could write the unique ID in an error page to the user and use it later to track and fix a false positive.

Choosing what to log

The SecAuditEngine parameter accepts one of four values:

  • On – log all requests

  • Off – do not log requests at all

  • RelevantOnly – only log relevant requests. Relevant requests are those requests that caused a filter match.

  • DynamicOrRelevant – log dynamically generated or relevant requests. A request is considered dynamic if its handler is not null.

Getting ModSecurity to log dynamic requests can sometimes require a little bit of work depending on your configuration. In Apache theory, a response to a request is generated by a so-called handler. If there is a handler attached to a request it should be considered to be of a dynamic nature. In practise, however, Apache can be configured to serve dynamic pages without a handler (it then chooses the module based on the resource MIME type). This will happen, for example, if you configure PHP as instructed in the main distribution:

AddType application/x-httpd-php .php

While this works, it isn't entirely correct. However, if you replace the above line with the following:

AddHandler application/x-httpd-php .php

PHP will work just as well, Apache will have a handler assigned to the request, and audit logger will be able to log selectively.

As of 1.9 the audit logger takes the response code into account when deciding whether something is relevant or not. At the time response codes 4xx and 5xx are treated as relevant. This makes it easy to perform audit logging on request from (any) web application. If an error occurs just respond as you normally would but just change the response code to (for example) 500.

New Audit Log Type

New audit log type was introduced in ModSecurity 1.9. The old audit log type remains useful for ad-hoc logging but it is considered obsolete and it may be removed from ModSecurity 2.0. The new audit log type was introduced to increase performance (one file per transaction is created, avoiding the need to synchronise writes between concurrent requests), increase the amount of information logged, and to allow for real-time audit log aggregation (a proof-of-concept piped logging script, modsec-auditlog-collector.pl, is included in the distribution). The new audit log type can log the response body too.

Note

For the new audit log type to work the mod_unique_id module must be active.

Example configuration:

# Yes, we want to use the new format
SecAuditLogType Concurrent

# Directory where the files will be stored
# MUST NOT BE THE SAME AS THE APACHE LOGS FOLDER
SecAuditLogStorageDir /var/www/audit_log/data/

# The index of all files created
# YOU MUST NOT ALLOW NON-ROOT USERS TO WRITE
# TO THE BASE FOLDER
SecAuditLog /var/www/audit_log/index

# Choose what to log – everything (default is ABCFHZ)
SecAuditLogParts ABCDEFGHZ

For each audit log file created, an one-line entry will appear in the index file. Those who wish to implement real-time audit log aggregation should configure a script to receive information about audit log entries via the piped logging mechanism.

Note

You have probably heard that it is dangerous to allow non-root users to have write permissions in the Apache logs folder. In the same manner non-root users should not be allowed to have write permissions in the folder where you place the audit log index file. To be safe create a subdirectory beneath the main audit log folder and allow the httpd user to write there.

A typical audit log entry looks like this:

192.168.2.101 192.168.2.11 - - [15/Jul/2005:11:56:52 +0100] \
"POST /form.php HTTP/1.1" 403 3 "http://192.168.2.101:8080/form.php" \
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 \
Firefox/1.0.4" G3yTd38AAAEAAAM7BLwAAAAA "-" \
/20050715/20050715-1156/20050715-115652-G3yTd38AAAEAAAM7BLwAAAAA 0 1031 \
md5:dc910f6d647d47b32ae6e47326f0ca42

The line begins with a "vcombined" log format, but it then adds the following fields:

  • unique_id

  • session_id (not used at this time)

  • filename

  • offset

  • size

  • hash of the audit log entry (MD5 hash used at this time)

A typical audit log entry may look like this:

--67458b6b-A--
[15/Jul/2005:11:56:52 +0100] G3yTd38AAAEAAAM7BLwAAAAA \
192.168.2.11 4236 192.168.2.101 8080
--67458b6b-B--
POST /form.php HTTP/1.1
Host: 192.168.2.101:8080
User-Agent: Mozilla/5.0
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://192.168.2.101:8080/form.php
Content-Type: application/x-www-form-urlencoded
Content-Length: 5

--67458b6b-C--
f=111
--67458b6b-E--
403 (Response body)
--67458b6b-F--
HTTP/1.1 403 Forbidden
Last-Modified: Fri, 08 Jul 2005 14:25:30 GMT
ETag: "decb4-3-34b96a80"
Accept-Ranges: bytes
Content-Length: 19
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html

--67458b6b-H--
Message: Pattern match "111" at POST_PAYLOAD \
[id "1"] [rev "2"] [msg "3"] [severity "4"]
Apache-Handler: application/x-httpd-php
Stopwatch: 1126536042708000 11024 (7276* 7375 9842)

--67458b6b-Z--

Note

Take care when handling audit log data. The files may contain unfiltered binary data received over the network. Such data may be dangerous if not handled properly (e.g. it may contain raw terminal escape sequences.)

Available audit log parts:

  • A – audit log header (mandatory)

  • B – request headers

  • C – request body (present only if the request body exists and ModSecurity is configured to intercept it)

  • D - RESERVED for ntermediary response headers, not implemented yet.

  • E – intermediary response body (present only if ModSecurity is configured to intercept response bodies, and if the audit log engine is configured to record it). Intermediary response body is the same as the actual response body unless ModSecurity intercepts the intermediary response body, in which case the actual response body will contain the error message (either the Apache default error message, or the ErrorDocument page).

  • F – final response headers (excluding the Date and Server headers, which are always added by Apache in the late stage of content delivery).

  • G – RESERVED for the actual response body, not implemented yet.

  • H - audit log trailer

  • Z – final boundary, signifies the end of the entry (mandatory)

Note

ModSecurity does not log response bodies of stock Apache responses (e.g. 404).

Guardian log

Since 1.9 ModSecurity supports a new directive, SecGuardianLog, that is designed to send all access data to another program using the piped logging feature. Since Apache is typically deployed in a multi-process fashion, making information sharing difficult, the idea is to deploy a single external process to observe all requests in a stateful manner, providing additional protection.

Development of a state of the art external protection tool will be a focus of subsequent ModSecurity releases. However, a fully functional tool is already available as part of the Apache httpd tools project (http://www.apachesecurity.net/tools/). The tool is called httpd-guardian and can be used to defend against Denial of Service attacks. It uses the blacklist tool (from the same project) to interact with an iptables-based (Linux) or pf-based (*BSD) firewall, dynamically blacklisting the offending IP addresses. It can also interact with SnortSam (http://www.snortsam.net). Assuming httpd-guardian is already configured (look into the source code for the detailed instructions) you only need to add one line to your Apache configuration to deploy it:

SecGuardianLog |/path/to/httpd-guardian

By default httpd-guardian will defend against clients that send more 120 requests in a minute, or more than 360 requests in five minutes.

Custom logging

Since 1.8 it is possible to use Apache custom logging to log only those requests where ModSecurity was involved. This is because ModSecurity now defines an environment variable mod_security-relevant whenever it performs an action. To use a custom log file, add the following (or similar) to your configuration:

CustomLog logs/modsec_custom_log \
"%h %l %u %t \"%r\" %>s %b %{mod_security-message}i" \
env=mod_security-relevant