> ## Content Index
> Fetch the complete content index at: https://logbuch-changelog.nichedthemes.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# pegel.example.org was down for 2 hours 45 minutes
- URL: https://logbuch-changelog.nichedthemes.com/pegel-example-org-was-down-for-2-hours-45-minutes/
- Published: 2026-09-27T09:00:00.000Z
- Updated: 2026-09-27T09:00:00.000Z
- Description: A full disk took down the site, the downloads and the status page on 2026-09-25. The download log had not been rotated since a move a month earlier.
- Author: David Osei
- Tags: Pegel, Incidents, #incident, #resolved, #critical, #Import 2026-11-19 07:37

The site and the downloads were down from 21:10 to 23:55 CEST on 2026-09-25, and the status page until 00:40\. Nobody could install or upgrade Pegel, and the page that should have said so was down with them. The disk of the VPS was full: the access log of the downloads had not been rotated since I moved them on 2026-08-30, and it had grown to 41 GB.

## Timeline

- 2026-09-25 21:10 CEST The disk of the VPS fills up. The site and the downloads answer 500.
- 21:12 Pegel alerts for pegel.example.org and the status page.
- 23:35 Investigating: We read the alert and log in to the VPS.
- 23:41 Identified: `/var` is full. The download log is 41 GB.
- 23:55 Monitoring: The log is compressed and moved. The site and the downloads answer again.
- 2026-09-26 00:40 The status page is back, restored from the backup of 2026-09-25 03:00\. Its database broke while writing to the full disk.
- 01:30 Resolved: Every check green for 50 minutes.

## What went wrong

The alert came on time. We read it at 23:35, after an evening away from our laptops, because Pegel has no on-call rota, and with two people it will not get one. The move on 2026-08-30 copied the download server’s config but not its `logrotate` entry, and nothing told us the disk was filling.

```console
$ df -h /var
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        50G   50G     0 100% /var
```

## What changed

- The download log is rotated daily and kept for 14 days.
- The status page runs on the second VPS, so it cannot fall with the site.
- Pegel checks the disk of the VPS through its health URL and alerts at 80 %.