Removing Log Files with a Cron Job in Linux

1. Introduction

Log file management is a critical aspect of maintaining a healthy Linux system. Over time, log files can accumulate, consuming valuable disk space and potentially leading to degraded system performance or even outages if left unchecked. Automating the cleanup of these logs not only helps us preserve disk space but also ensures that our system remains stable.

In this tutorial, we’ll explore how to automate the removal of log files using cron jobs and the find command. We’ll also take a closer look at the more advanced and flexible logrotate utility. By the end, we’ll have a practical understanding of how to keep our log directories tidy and our system running smoothly.

2. Using Cron and the find Command

Cron is the standard time-based job scheduler in Unix-like operating systems. It enables us to schedule scripts or commands to run automatically at specified intervals, whether these intervals are minutes, hours, days, or even months. This makes it an ideal tool for automating repetitive system maintenance tasks such as log file cleanup.

First, let’s review how to interact with cron. To list all cron jobs for the current user, we run the following command:

$ crontab -l

To edit our crontab (the list of our scheduled jobs), we’ll run:

$ crontab -e

Suppose we have an application that writes logs to /var/log/myapp, and we want to remove all .log files older than seven days automatically. The find command is well-suited for this purpose. Here’s how we can construct the command to identify and remove such files:

$ find /var/log/myapp -name "*.log" -type f -mtime +7 -exec rm {} \;

The -name “*.log” argument only targets files with .log extensions, and -type f ensures we only consider files (not directories). Next, the -mtime +7 argument selects files older than 7 days. Finally, -exec rm {} \; removes each file found.

To automate this cleanup, we’ll add this command to our crontab. For example, to run the cleanup every day at midnight, we’ll add the following line:

0 0 * * * find /var/log/myapp -name "*.log" -type f -mtime +7 -exec rm {} \;

This approach is straightforward and effective for removing logs. However, it lacks features such as log rotation, compression, and notification, which can be essential in more complex environments.

3. Managing Log Deletion with logrotate

While coupling cron with find is suitable for many scenarios, it may not be robust enough for all log management needs. For more advanced requirements, such as rotating, compressing, and retaining logs for a set period, Linux provides the logrotate utility. This utility is powerful and highly configurable, designed specifically for managing log files.

We can automatically rotate logs with logrotate (renaming and archiving them), compress old logs to save space, remove logs after a certain period, and even execute custom scripts after rotation. Most Linux distributions come with logrotate pre-installed and configured to manage system logs out of the box.

To manage logs for a custom application, we’ll create a dedicated configuration file in /etc/logrotate.d. For example, to manage /var/log/myapp/*.log files, we create /etc/logrotate.d/myapp with the following content:

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    delaycompress
    maxage 14
    postrotate
        systemctl reload myapp
    endscript
}

Each line contains a command, also known as a directive, which configures the logrotate utility per our needs.

Here’s what we specified above, with each of these directives:

• daily: Rotates the logs every day
• rotate 7: Keeps seven archived log files before deleting the oldest
• compress: Compresses old log files to save disk space
• missingok: Ignores errors if log files are missing
• notifempty: Skips rotation if the log file is empty
• delaycompress: Delays compression until the next rotation cycle
• maxage 14: Deletes log files older than 14 days
• postrotate / endscript: Runs the specified commands after rotation – in this case, reloading the application to ensure it continues logging to the correct file

To verify our configuration and see what actions logrotate would perform, we can run:

$ logrotate --debug /etc/logrotate.d/myapp

We can also force a rotation (useful for testing) with:

$ logrotate --force /etc/logrotate.d/myapp

This allows us to confirm our log rotation policy works as expected without waiting for the scheduled interval.

4. Conclusion

Automating log file cleanup in Linux is essential for maintaining system stability and preventing disk space exhaustion. Certainly, we can address basic needs with a simple cron job with the find command. On the other hand, logrotate offers a more comprehensive and flexible solution. This is especially useful for production environments where log retention, compression, and application reloads are necessary.

By leveraging these tools effectively, we can ensure that our logs are managed efficiently, our disk space is preserved, and our system remains reliable. For most scenarios, combining both approaches provides a robust foundation for log hygiene on Linux systems.