What Do The Fields In An Apache Access Log Mean?

Sample Log Entry

Here's a typical Apache access log entry:

127.0.0.1 - - [05/Feb/2012:17:11:55 +0000] "GET / HTTP/1.1" 200 140 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.5 Safari/535.19"

Breakdown of Log Fields

The log entry has several fields, each giving specific information about the request:

• IP Address: 127.0.0.1
• Identity: -
• User ID: -
• Timestamp: [05/Feb/2012:17:11:55 +0000]
• HTTP Request: "GET / HTTP/1.1"
• Status Code: 200
• Response Size: 140
• Referrer: "-"
• User Agent: "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.5 Safari/535.19"

Each of these fields has a purpose and gives useful information about the request made to the server. Understanding these fields helps you analyze server traffic, fix issues, and monitor server performance.

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %D" combined

IP Address

The IP address field shows the client's IP address that made the request to the server. This information helps track visitor locations, identify potential security threats, and analyze traffic patterns.

Identity and User ID

The identity and user ID fields often appear as hyphens (-) in most log entries. The identity field is set by identd, a protocol rarely used today due to security concerns. The user ID field shows the user name for authenticated requests, but it's usually empty for public websites.

Timestamp

The timestamp field displays the date and time when the server received the request. It's typically in the format [day/month/year:hour:minute:second zone]. The zone part indicates the time difference from Coordinated Universal Time (UTC).

HTTP Request

This field contains three parts:

1. Method: The HTTP method used (e.g., GET, POST, PUT).
2. Resource: The requested resource on the server (e.g., /index.html).
3. Protocol: The HTTP protocol version used by the client (e.g., HTTP/1.1).

Status Code

The status code is a three-digit number that indicates the request's outcome. Common status codes include:

• 200: Successful request
• 301/302: Redirects
• 404: Not Found
• 500: Internal Server Error

Response Size

This field shows the size of the server's response to the client in bytes. It doesn't include the response headers' size. A hyphen (-) means zero bytes were sent.

Referrer

The referrer field shows the URL of the page that linked to the requested resource. It helps track where visitors are coming from. A hyphen (-) indicates no referrer information was provided.

User Agent

The user agent string provides information about the client's browser, operating system, and device. This data helps understand your audience and optimize your website for different browsers and devices.

Common Log Format vs Combined Log Format

Apache supports two main log formats: Common Log Format (CLF) and Combined Log Format. Both formats include basic information, but the Combined Log Format provides more details.

Common Log Format includes:

• IP address
• Identity
• User ID
• Timestamp
• HTTP request
• Status code
• Response size

Combined Log Format adds two more fields:

• Referrer
• User agent

The Combined Log Format gives more information about user behavior and client details, making it the preferred option for most web administrators.

Customizing Log Formats

To change Apache log formats:

1. Open your Apache configuration file (usually httpd.conf or apache2.conf).

2. Find or add the LogFormat directive:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined

3. Change the format string using Apache's log format placeholders:

   • %h: Remote host
   • %l: Remote logname
   • %u: Remote user
   • %t: Time the request was received
   • %r: First line of request
   • %>s: Status
   • %b: Response size in bytes

4. Add or remove fields as needed. For example, to include the time taken to serve the request:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %D" combined

5. Save the configuration file and restart Apache to apply changes.

By customizing log formats, you can adjust the logged information to your needs, helping you monitor and analyze your web server better.

LogFormat "%h %l %u %t \"%r\" %>s %b %D %T" performance_log
CustomLog /var/log/apache2/performance.log performance_log

Key Metrics to Monitor

When analyzing Apache access logs, focus on these key metrics:

1. Traffic patterns: Look at visitor numbers, peak hours, and popular pages. This data helps you understand user behavior and improve your website's content and performance.

2. Error rates: Monitor HTTP status codes, especially 4xx and 5xx errors. High error rates may point to issues with your website or server setup.

3. Performance indicators: Check response times and page load speeds. Slow-loading pages can harm user experience and search engine rankings.

Tools for Log Analysis

To analyze Apache access logs, you can use these tools:

1. Built-in Apache tools:

   • Apache's mod_status module provides basic log analysis.
   • The 'apachectl status' command gives a real-time view of server performance.
   • Apache's 'rotatelogs' utility helps manage log files by rotating them based on size or time.

2. Third-party software options:

   • AWStats: A free, open-source log analyzer that creates visual reports.
   • GoAccess: A real-time web log analyzer with a terminal-based interface.
   • ELK Stack (Elasticsearch, Logstash, Kibana): A tool for collecting, processing, and visualizing log data.
   • Webalizer: A fast, free web server log file analysis program.

goaccess /var/log/apache2/access.log -c

These tools can help you get useful insights from your Apache access logs, allowing you to make data-driven choices to improve your website's performance and user experience.