Logging Remote Port Numbers in the Apache HTTP Server for Carrier Grade NAT

Logging Remote Port Numbers in the Apache HTTP Server

Problem statement

Carrier Grade NAT (CGN) comes into use. For instance, a Japanese mobile carrier "au" announced that it will provide its network to consumers with CGN [1]. CGN is one of key technologies against IPv4 address exhaustion and for IPv6 transition. Logging remote port numbers is urgent task in Carrier Grade NAT (CGN). This is because, unlike normal NATs that have been used in home gateways, CGN shares one IPv4 address among several consumers. This causes huge problem with logging, especially in forensic cases; for example, when we need to trace malicious access and to identify the user. Thus, remote port number logging is essential for all HTTP servers from now on.

The Apache HTTP server is one of the most deployed implementations. However, the default configuration of logging in Apache does not store port numbers of remote hosts. Here, this article provides a configuration example to enable remote port number logging.

A configuration example

In the configuration file httpd.conf, or sometimes renamed to apache2.conf etc., by default, you can find two log format configurations. Note that you may find more configurations that are created by OS/distribution package management systems.

    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

These log formats are used in the file or virtual host configurations like:

CustomLog path/to/log/dir/access_log common

or

CustomLog path/to/log/dir/access_log combined

The log format named "common" stores minimum information on clients such as remote host's IP address, and request/response information. The log format named "combined" stores more information such as browser's referrer, and user agent. Please refer to Ref. [2] for greater details. However, these formats do not store port numbers of remote hosts. In order to store port numbers of remote hosts, you can use the format style "%{remote}p". The following configurations add a port number after an IP address, extended from the default configurations.

    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "[%h]:%{remote}p %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "[%h]:%{remote}p %l %u %t \"%r\" %>s %b" common

You can change the format as you wish, but here we note that these extended configurations add brackets "[" and "]" on the side of an IP address represented by "%h" in order to distinguish the seperator (:) between the IP address and port number from IPv6 address seperators (e.g., [2001:db8::d09:f00d]:12345). One more thing to be noted is that changes in log formats affects your log analysis tools, so please test it carefully and we recommend to create new file for new log format.

References

[1] IPv4枯渇に伴い、auのスマートフォンはプライベートアドレスで運用へ
[2] Apache Module mod_log_config