Logrotate: Choosing Between Size-Based and Time-Based Log Rotation

Managing log files effectively is crucial for ensuring a well-performing, reliable system. Logrotate, a popular log management tool, provides a flexible way to automatically rotate, compress, and remove old logs. Among its many configurations, two common approaches to trigger log rotation are size-based and time-based rotation.

In this blog, we will explore the differences between these methods, compare their use cases, and help you decide which approach (or combination) suits your needs best.

Table of Contents:

1. What is Logrotate?
2. Time-Based Log Rotation
3. Size-Based Log Rotation
4. Combining Time-Based and Size-Based Log Rotation
5. Key Differences between Time-Based and Size-Based Log Rotation
6. How to Choose the Right Log Rotation Method?

What is Logrotate?

Logrotate is a system utility for managing the size and lifecycle of log files. Over time, log files can grow exponentially, consuming disk space and potentially affecting system performance. Logrotate automates the process of rotating (renaming and starting new logs), compressing, and deleting old logs based on configurable criteria.

Key features of Logrotate:

• Rotate logs based on file size, time intervals, or a combination of both.
• Save disk space through compression of rotated logs.
• Manage old logs effectively with retention policies.
• Customize log rotation using flexible directives.

Time-Based Log Rotation

Time-based log rotation rotates logs based on a specified time interval, regardless of the file size. Common intervals include:

• daily: Rotates logs every day.
• weekly: Rotates logs every week.
• monthly: Rotates logs every month.

Example Configuration:

/var/log/myapp.log {
    daily
    rotate 7
    compress
}

• daily: Logs are rotated every day.
• rotate 7: Retain the last 7 rotated logs.
• compress: Compress old logs after rotation.

When to use Time-Based Log Rotation:

• Predictable Log Growth: If your logs grow at a steady pace, time-based rotation ensures logs are rotated consistently.
• Compliance or Reporting: Logs may need to align with specific reporting periods, such as daily or monthly audits.
• Simplicity: Time-based rotation is straightforward and works well for systems with moderate log activity.

Size-Based Log Rotation

Size-based rotation rotates logs when they exceed a defined size threshold, such as 100M (100 MB). This ensures that no log file grows beyond a manageable size.

Example Configuration:

/var/log/myapp.log {
    size 100M
    rotate 5
    compress
}

• size 100M: Rotates logs when they reach or exceed 100 MB.
• rotate 5: Retain the last 5 rotated logs.
• compress: Compress old logs after rotation.

When to use Size-Based Log Rotation:

• Unpredictable Log Growth: If log file sizes vary greatly, size-based rotation prevents massive logs that could consume disk space.
• Disk Space Management: Ensures logs are rotated before they grow too large, safeguarding disk space.

Combining Time-Based and Size-Based Log Rotation

For maximum flexibility, Logrotate allows combining both methods. This approach ensures logs are rotated at regular intervals while also preventing oversized files.

Example Configuration:

/var/log/myapp.log {
    daily
    maxsize 200M
    rotate 10
    compress
}

• daily: Rotates logs every day.
• maxsize 200M: Rotates logs earlier if they exceed 200 MB.
• rotate 10: Retains the last 10 rotated logs.

When to Combine Both Methods:

• Critical Systems: For applications generating critical logs (e.g., error logs), combining size and time-based rotation ensures no logs are missed or oversized.
• High Log Volume: Handles both predictable and unpredictable log growth.

Key Differences between Time-Based and Size-Based Log Rotation

How to Choose the Right Log Rotation Method?

Choose time-based rotation if:

• Logs grow at a steady, predictable rate.
• You want consistent log rotation schedules (e.g., daily, weekly).
• Disk space is not a critical concern, or logs are limited in other ways.

Choose size-based rotation if:

• Logs grow unpredictably or very quickly.
• Disk space is a major concern, and you need to control log file sizes.
• Applications generate logs with variable patterns (e.g., during high traffic).

Combine both if:

• Your system handles both predictable and unpredictable log growth.
• You want to avoid both oversized files and skipped rotations.

Conclusion

Effective log management is a vital aspect of system administration, and logrotate makes it easy to automate. Whether you choose time-based, size-based, or a combination of both log rotation methods depends on your system’s needs.

• Time-based log rotation works well for predictable environments and reporting needs.
• Size-based log rotation excels in managing disk space and handling unpredictable log growth.
• Combining both methods ensures comprehensive coverage for diverse scenarios.

With the right configuration, you can keep your logs manageable, your systems performing efficiently, and your disk space under control.