Linux Log Management

Why Logs Matter

Logs are one of the first places you look when a Linux system or application behaves unexpectedly. They show service starts, authentication attempts, kernel events, web server requests, and application errors.

For DevOps engineers, log management is not only about reading files. It is also about knowing where logs live, how to filter them quickly, and how to prevent them from filling the disk.

The /var/log Directory

Traditional Linux log files are usually stored under /var/log.

Useful examples include:

• /var/log/syslog for general system messages on many Debian based systems
• /var/log/auth.log for authentication related events
• /var/log/kern.log for kernel messages
• /var/log/nginx/ for Nginx access and error logs
• /var/log/apache2/ for Apache logs

You can inspect the directory with:

ls /var/log
ls /var/log/nginx

Different distributions place logs differently, so learn the layout of the systems you operate.

Watching Logs Live with tail -f

One of the quickest ways to follow activity is tail -f.

tail -f /var/log/syslog
tail -f /var/log/nginx/access.log

This keeps the terminal open and streams new lines as they are written.

It is especially useful while restarting a service, testing a deployment, or reproducing an error.

Tip: Open one terminal to follow a log and another terminal to reproduce the issue. Seeing the event appear live often shortens troubleshooting time.

Filtering Logs with grep

Raw logs can be noisy. grep helps you focus on the lines you care about.

grep sshd /var/log/auth.log
grep ERROR /var/log/myapp.log
tail -f /var/log/syslog | grep nginx

You can combine tools to narrow large outputs into something meaningful. That is a core Linux troubleshooting skill.

Using journalctl for systemd Logs

On systemd-based systems, journalctl reads the system journal.

Show recent journal entries:

journalctl

Follow logs live:

journalctl -f

Show logs for a specific service:

journalctl -u nginx
journalctl -u sshd -f

Filter by time:

journalctl --since "1 hour ago"
journalctl --since "2026-07-14 09:00:00" --until "2026-07-14 10:00:00"

Filter by priority:

journalctl -p err
journalctl -p warning

journalctl is powerful because it organizes logs by service, time, boot, and priority rather than forcing you to search every file manually.

Note: Some systems use both traditional files in /var/log and the systemd journal. You should be comfortable with both approaches.

Managing Log Growth with logrotate

Without rotation, log files can grow until they waste disk space or even cause outages.

logrotate solves this by rotating old logs, compressing them, and deleting very old copies according to policy.

Typical configuration concepts include:

• rotate for how many archived files to keep
• compress for gzip compression of old logs
• daily, weekly, or monthly for rotation schedule
• missingok so missing logs do not cause failure
• notifempty so empty logs are not rotated

A simplified example config might look like:

cat /etc/logrotate.d/nginx

And you may see directives such as:

/var/log/nginx/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
}

This means Nginx logs are rotated daily, 14 old copies are kept, and archived logs are compressed.

A Practical Troubleshooting Flow

Suppose a web app returns errors after deployment. A sensible sequence is:

tail -f /var/log/nginx/error.log
journalctl -u myapp --since "15 minutes ago"
grep 500 /var/log/nginx/access.log

That helps you inspect the reverse proxy, the application service, and request outcomes together.

What You Should Remember

Linux log management starts with knowing where logs are stored and how to inspect them quickly. Use /var/log for traditional files, tail -f for live updates, grep for filtering, and journalctl for structured systemd logs.

Then use logrotate so logs stay useful instead of becoming a disk space problem.

Test Your Understanding

<MCQExercise exercise={{ id: "linux-log-management-q1", title: "Exercise 1: Following new log lines", description: "Which command is commonly used to watch a log file as new entries are appended?", options: [ { id: "a", text: "tail -f /var/log/syslog" }, { id: "b", text: "chmod +x /var/log/syslog" }, { id: "c", text: "df -h /var/log/syslog" }, { id: "d", text: "ssh-keygen /var/log/syslog" }, ], correctId: "a", explanation: "tail -f follows a file and prints new lines as they are written. It is one of the fastest ways to observe logs during troubleshooting.", wrongExplanations: { b: "Changing execute permission is unrelated to monitoring a log file.", c: "df -h reports filesystem usage, not live log output.", d: "ssh-keygen creates SSH keys and has nothing to do with logs.", }, }} />

<MCQExercise exercise={{ id: "linux-log-management-q2", title: "Exercise 2: Reading logs for one service", description: "Which command shows journal entries for the nginx systemd unit?", options: [ { id: "a", text: "journalctl -u nginx" }, { id: "b", text: "grep nginx /etc/hosts" }, { id: "c", text: "logrotate -u nginx" }, { id: "d", text: "tail -u nginx" }, ], correctId: "a", explanation: "journalctl -u nginx filters the systemd journal to entries from the nginx unit. This is often easier than searching broad system logs.", wrongExplanations: { b: "/etc/hosts is for hostname mapping, not service logs.", c: "logrotate manages file rotation and does not query service journal entries this way.", d: "tail does not have a -u unit filter option for systemd services.", }, }} />

<MCQExercise exercise={{ id: "linux-log-management-q3", title: "Exercise 3: Purpose of logrotate", description: "What problem does logrotate primarily help solve?", options: [ { id: "a", text: "It schedules SSH logins" }, { id: "b", text: "It prevents log files from growing without control by rotating, compressing, and retaining old copies" }, { id: "c", text: "It changes user passwords every week" }, { id: "d", text: "It compiles the Linux kernel" }, ], correctId: "b", explanation: "logrotate keeps logs manageable by archiving them on a schedule, compressing older files, and limiting retention. That helps preserve disk space while keeping useful history.", wrongExplanations: { a: "SSH logins are unrelated to log rotation.", c: "Password management is a separate administrative task.", d: "Kernel compilation has nothing to do with log retention.", }, }} />