When the filtering engine is enabled, every incoming request is intercepted and analysed before it is processed. The analysis begins with a series of built-in checks designed to validate the request format. These checks can be controlled using configuration directives. In the second stage, the request goes through a series of user-defined filters that are matched against the request. Whenever there is a positive match, certain actions are taken.
The most simplest form of filtering is, well, simple. It looks like this:
SecFilter KEYWORD
For each simple filter like this, ModSecurity will look for the
keyword in the request. The search is pretty broad; it will be applied
to the first line of the request (the one that looks like this
GET /index.php?parameter=value HTTP/1.0
). In case of
POST
requests, the body of the request will be
searched too (provided the request body buffering is enabled, of
course).
All pattern matches are case insensitive by default.
Filters are not applied to raw request data, but on a normalised copy instead. We do this because attackers can (and do) apply various evasion techniques to avoid detection. For example, you might want to setup a filter that detects shell command execution:
SecFilter /bin/sh
But the attacker may use a string /bin/./sh
(which has the same meaning) in order to avoid the filter.
ModSecurity automatically applies the following transformations:
On Windows only, convert \
to
/
Reduce /./
to /
Reduce //
to /
Decode URL-encoded characters
You can choose whether to enable or disable the following checks:
Verify URL encoding
Allow only bytes from a certain range to be used
Null byte attacks try to confuse C/C++ based software and trick it
into thinking that a string ends before it actually does. This type of
an attack is typically rejected with a proper
SecFilterByteRange
filter. However, if you do not do
this a null byte can interfere with ModSecurity processing. To fight
this, ModSecurity looks for null bytes during the decoding phase and
converts them into spaces. So, where before this filter:
SecFilter hidden
would not detect the word hidden in this request:
GET /one/two/three?p=visible%00hidden HTTP/1.0
it now works as expected.
The simplest method of filtering I discussed earlier is actually slightly more complex. Its full syntax is as follows:
SecFilter KEYWORD [ACTIONS]
First of all, the keyword is not a simple text. It is an regular expression. A regular expression is a mini programming language designed to pattern matching in text. To make most out of this (now) powerful tool you need to understand regular expressions well. I recommend that you start with one of the following resources:
Perl-compatible regular expressions man page, http://www.pcre.org/pcre.txt
Perl Regular Expressions, http://www.perldoc.com/perl5.6/pod/perlre.html
Mastering Regular Expressions, http://www.oreilly.com/catalog/regex/
Google search on regular expressions, http://www.google.com/search?q=regular%20expressions
Wikipedia entry, http://en.wikipedia.org/wiki/Regular_expression
POSIX regular expressions, http://www.wellho.net/regex/posix.html
Two different regular expression engines are used in Apache 1.x and Apache 2.x. The Apache 1.x regular expression engine is POSIX compliant. The Apache 2.x regular engine is PCRE compliant. As a rule of thumb, regular expressions that work in Apache 1.x will work in Apache 2.x, but not the other way round. If you need to write rules that work on both major branches you will have to test them thoroughly. Since 1.9.2 it is possible to compile ModSecurity for Apache 1.x to use PCRE as an regular expression library.
The second parameter is an action list definition, which specifies what will happen if the filter matches. Actions are explained later in this manual.
If exclamation mark is the first character of a regular expression, the filter will treat that regular expression as inverted. For example, the following:
SecFilter !php
will reject all requests that do not contain the word php.
While SecFilter
allows you to start quickly,
you will soon discover that the search it performs is too broad, and
doesn't work very well. Another directive:
SecFilterSelective LOCATION KEYWORD [ACTIONS]
allows you to choose exactly where you want the search to be
performed. The KEYWORD
and the
ACTIONS
bits are the same as in
SecFilter
. The LOCATION
bit
requires further explanation.
The LOCATION
parameter consist of a series of
location identifiers separated with a pipe.
Examine the following example:
SecFilterSelective "REMOTE_ADDR|REMOTE_HOST" KEYWORD
It will apply the regular expression only the IP address of the client and the host name. The list of possible location identifiers includes all CGI variables, and some more. Here is the full list:
REMOTE_ADDR
REMOTE_HOST
REMOTE_USER
REMOTE_IDENT
REQUEST_METHOD
SCRIPT_FILENAME
PATH_INFO
QUERY_STRING
AUTH_TYPE
DOCUMENT_ROOT
SERVER_ADMIN
SERVER_NAME
SERVER_ADDR
SERVER_PORT
SERVER_PROTOCOL
SERVER_SOFTWARE
TIME_YEAR
TIME_MON
TIME_DAY
TIME_HOUR
TIME_MIN
TIME_SEC
TIME_WDAY
TIME
API_VERSION
THE_REQUEST
REQUEST_URI
REQUEST_FILENAME
IS_SUBREQ
There are some special locations:
POST_PAYLOAD
– filter the body of the
POST
request
ARGS
- filter arguments, the same as
QUERY_STRING|POST_PAYLOAD
ARGS_NAMES
– variable/parameter names
only
ARGS_VALUES
– variable/parameter values
only
COOKIES_NAMES
- cookie names only
COOKIES_VALUES
- cookie values only
SCRIPT_UID
SCRIPT_GID
SCRIPT_USERNAME
SCRIPT_GROUPNAME
SCRIPT_MODE
ARGS_COUNT
COOKIES_COUNT
HEADERS
HEADERS_COUNT
HEADERS_NAMES
HEADERS_VALUES
FILES_COUNT
FILES_NAMES
FILES_SIZES
And even more special:
HTTP_header – search request header
"header"
(HEADER_header
also works as
of 1.9)
ENV_variable
– search environment variable
variable
ARG_variable
– search request
variable/parameter variable
COOKIE_name
- search cookie with name
name
FILE_NAME_variable
- search the filename of
the file uploaded under the name variable
.
FILE_SIZE_variable
- search the size of the
file uploaded under the name variable
A limited number of output-specific variables are also available for Apache 2 (only when output buffering is enabled):
OUTPUT - the complete response body
OUTPUT_STATUS - response status code
The ARG_variable
location names support
inverted usage when used together with the ARG
location. For example:
SecFilterSelective "ARGS|!ARG_param" KEYWORD
will search all arguments except the one named param.
ModSecurity provides full support for Cookies. By default cookies
will be treated as if they were in version 0 format (Netscape-style
cookies). However, version 1 cookies (as defined in RFC 2965) are also
supported. To enable version 1 cookie support use the
SecFilterCookieFormat
directive:
# enable version 1 (RFC 2965) cookies SecFilterCookieFormat 1
By default, ModSecurity will not try to normalise cookie names and
values. However, since some applications and platforms (e.g. PHP) do
encode cookie content you can choose to apply normalisation techniques
to cookies. This is done using the
SecFilterNormalizeCookies
directive.
SecFilterNormalizeCookies On
Prior to version 1.8.7 ModSecurity supported the
SecFilterCheckCookieFormat
directive. Due to recent
changes in 1.8.7 this directive is now deprecated. It can still be
used in the configuration but it does not do anything. The directive
will be completely removed in the 1.9.x branch.
ModSecurity supports output filtering in the version for Apache 2. It is disabled by default so you need to enable it first:
SecFilterScanOutput On
After that, simply add selective filters using a special variable
OUTPUT
:
SecFilterSelective OUTPUT "credit card numbers"
Those who have perhaps followed my columns at http://www.webkreator.com/php/ know that I am somewhat obsessed with the inability of PHP to prevent fatal errors. I have gone to great lengths to prevent fatal errors from spilling to end users (see http://www.webkreator.com/php/configuration/handling-fatal-and-parse-errors.html) but now, finally, I don't have to worry any more about that. The following will catch PHP output error in the response body, replace the response with an error, and execute a custom PHP script (so that the application administrator can be notified):
SecFilterSelective OUTPUT "Fatal error:" deny,status:500 ErrorDocument 500 /php-fatal-error.html
You should note that although you can mix output filters with input filters, they are not executed at the same time. Input filters are executed before a request is processed by Apache, while the output filters are executed after Apache completes request processing.
Actions skipnext
and chain
do not work with output filters.
Output filtering is only useful for plain text and HTML output.
Applying regular expressions to binary content (for example images) will
only slow down the server. By default ModSecurity will scan output in
responses that have no content type, or whose content type is
text/plan
or text/html
. You can
change this using the SecFilterOutputMimeTypes
directive:
SecFilterOutputMimeTypes "(null) text/html text/plain"
Configured as in example above ModSecurity will apply output filters to plain text files, HTML files, and files where the mime type is not specified "(null)".
Using output buffering will make ModSecurity keep the whole of the page output in memory, no matter how large it is. The memory consumption is over twice the size of the page length.
While output monitoring is a useful feature in some circumstances you should be aware that it isn't foolproof. If an attacker is in a full control of request processing she can evade output monitoring in two ways:
Use a Content-Type that is not being monitored. (For performance reasons it is not feasible to monitor all content types.)
Encode the output in some way. Any simple encoding is likely to be enough to fool monitoring.
As of 1.9 another output variable is supported -
OUTPUT_STATUS
. This variable contains the status code
of the response.