How to Manage Log Files with logrotate (Stop Logs Filling the Disk)

Every program that writes a log will happily write it forever. logrotate is the cron-driven tool that renames access.log to access.log.1 each night, compresses the older ones, deletes the oldest, and tells the program to start a fresh file. Ubuntu ships it configured for the system's own logs; you add a few lines for anything else — your scripts, a web app, and especially Docker, which has its own worse default.

ls /etc/logrotate.d/
cat /etc/logrotate.d/nginx

/var/log/nginx/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
postrotate
invoke-rc.d nginx rotate >/dev/null 2>&1
endscript
}

Read top to bottom: rotate daily, don't complain if missing, keep 14, gzip them (but not the newest one, so a program still writing to it isn't disturbed), skip empty files, create the new file with these permissions, and afterwards tell Nginx to reopen its log.

Say your backup script writes /var/log/backup.log and the disk alert writes /var/log/rclone.log:

sudo nano /etc/logrotate.d/myscripts

/var/log/backup.log /var/log/rclone.log {
weekly
rotate 8
compress
delaycompress
missingok
notifempty
copytruncate
}

copytruncate copies the log then empties the original in place — right for scripts that append with >> and don't know how to reopen a file. Without it, the script would keep writing into the renamed .1 file.

sudo logrotate -d /etc/logrotate.d/myscripts # dry run: prints what it would do
sudo logrotate -f /etc/logrotate.d/myscripts # do it now
ls -l /var/log/backup.log*

By default Docker's container logs are unlimited. A chatty container can write gigabytes to /var/lib/docker/containers//-json.log. Fix it for every container at once:

sudo nano /etc/docker/daemon.json

{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}

sudo systemctl restart docker

Existing containers keep their old settings until recreated (docker compose up -d --force-recreate). Check who's noisy first: sudo du -sh /var/lib/docker/containers//-json.log | sort -h | tail.

Per container instead, in compose:

logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"

journalctl logs aren't handled by logrotate. Cap them in /etc/systemd/journald.conf:

SystemMaxUse=500M

sudo systemctl restart systemd-journald. (Tutorial 14 covers vacuuming the existing one.)

For a log that grows unpredictably: replace daily/weekly with size 50M — rotate whenever it passes 50 MB, checked daily.

Rotation never happens. logrotate runs from /etc/cron.daily/logrotate (or a systemd timer): systemctl status logrotate.timer. Check /var/lib/logrotate/status for the last run per file.

"error: skipping because parent directory has insecure permissions". The folder holding the log is world-writable. chmod 755 it, or add su root adm to the block.

Program keeps writing to the .1 file. It wasn't told to reopen. Add copytruncate, or a postrotate that sends it a reload.

Disk still full. Something else: sudo ncdu /var/log and docker system df.

One file per app in /etc/logrotate.d/, copytruncate for simple scripts, -d to test, and a daemon.json that caps Docker's logs at 30 MB per container before they quietly eat the disk.