Diagnosing Recurring CPU Spikes on a Small VPS with atop¶

Monitoring fires a "CPU is running high" alert on a small (1–2 vCPU) VPS. It auto-resolves before you log in, then fires again hours or days later. top shows nothing because the spike is already over. This is the classic recurring, intermittent CPU spike — and the only way to catch it is per-process history, not live sampling.

This article is the generalised playbook behind a real multi-week investigation on a 1 vCPU WordPress droplet. The same approach applies to any small VPS where spikes come and go.

Step 1 — Install atop for per-process history¶

top/htop only show now. atop samples the whole process table on an interval and writes it to a daily log, so you can rewind to exactly when the spike happened and see which process caused it.

sudo apt install atop        # Debian/Ubuntu
sudo dnf install atop        # Fedora/RHEL

sudo systemctl enable --now atop.service        # continuous sampling
sudo systemctl enable --now atop-rotate.timer   # daily log rotation

A 600-second (10 min) sample interval costs ~0.4% CPU and ~12 MB RAM — cheap enough to leave running permanently. Logs land in /var/log/atop/atop_YYYYMMDD.

Don't troubleshoot blind. If a spike already happened and you have no atop/sar history, you cannot diagnose it after the fact — install atop first and wait for the next occurrence.

Step 2 — Find the spike, then the process behind it¶

The authoritative view is atopsar -O — top-3 processes by CPU per sample. This is what tells you the cause, and it sidesteps the column-parsing traps below.

# Top CPU consumers per 10-min sample for a given day
atopsar -O -r /var/log/atop/atop_20260628

# Sweep the last 7 days for any sample where a single process broke 25%
for f in $(ls -1 /var/log/atop/atop_* | tail -7); do
  atopsar -O -r "$f" 2>/dev/null | awk '
    /^[0-9]/ { for (i=1;i<=NF;i++) if ($i ~ /%$/) { v=$i; sub(/%/,"",v);
               if (v+0 > 25) print FILENAME, $1, $(i-1), $i } }' FILENAME="$f"
done

A clean result looks like one expected event per week (e.g. the Sunday clamscan) with overnight idle hours topping out at netdata ~4% / mysqld ~2%.

Watch out: aggregate-CPU parsing traps¶

If you parse atopsar -c (aggregate CPU) instead of -O, two artifacts will manufacture phantom 100% spikes:

%idle is not the last field. The header row ends in a literal _cpu_ label. Read %idle by name/position, not blindly as $NF — $NF lands on the label (0) and reports busy = 100% everywhere.
The 00:00:0X rollover sample reads ~100%. The first sample after atop rotates its log is an artifact, not a spike. Filter it out.
Header rows share the timestamp regex. Filter to data rows only (e.g. $2 == "all").

When in doubt, trust atopsar -O (real per-process numbers) over any aggregate-CPU math.

Step 3 — Match the process to a root cause¶

Recurring spikes on a small web VPS almost always trace to one of a short list. In the reference investigation, the box hit all of these in sequence:

Symptom in `atopsar -O`	Likely root cause	Fix
`clamscan` ~96% for ~20–50 min, weekly	Antivirus full scan — single-threaded, fills the only core	Expected. `nice`/`ionice` it; see ClamAV Safe Scheduling
`kworker`/`blk_mq_*` kernel stalls, doubling overnight	`snapd` loop-device (squashfs) auto-refresh saturating a CPU workqueue	Remove snapd if unused, or blacklist the loop modules
`php-fpm` + `mysqld` together, traffic-correlated evenings	WordPress WP-Cron stampede — `wp-cron.php` fires on every page hit, stacking `action_scheduler`/sync jobs during bursts	`DISABLE_WP_CRON` in `wp-config.php` + a system cron running `wp-cron.php` every 5 min
`apache2` burning CPU shuffling queued connections under a small `pm.max_children`	Apache prefork + mod_php can't decouple connections from PHP workers	Migrate to mpm_event + php-fpm over a Unix socket (`mod_proxy_fcgi`)
Everything serialises; latency scales linearly with concurrency	Single-vCPU ceiling — N PHP processes share 1 core, each gets 1/N	Vertical resize (add a vCPU). No config can beat a hardware ceiling

The key insight: most of these are masked by prefork + mod_php (every Apache child is a PHP worker, so saturation looks like "busy"). Switching to mpm_event makes the PHP-FPM pool size the new visible bottleneck — that's expected, and you then tune pm.max_children / MaxRequestWorkers to the box's RAM.

Step 4 — The DigitalOcean monitoring caveat¶

DO (and most hypervisor-level monitors) measure total CPU utilization, not contention. A polite nice -19 ionice -c3 scan and a runaway mining trojan look identical to the hypervisor: both are "100% CPU." On a 1 vCPU box at 3 AM there's nothing for a niced job to yield to, so it opportunistically pins the core and trips a >85% / 5m alert.

Fix the alert, not the workload: scope a relaxed per-droplet alert (e.g. >95% / 30m) for the box with the known weekly scan, and keep the sensitive fleet-wide alert for everything else. Full recipe in ClamAV Fleet Deployment → DigitalOcean Monitoring Caveat.

Step 5 — Automate the weekly check, then retire it¶

A weekly CPU-spike investigator script — scan the last 7 days of atop logs, flag samples over a threshold, list the top consumers, mail the report — turns "did it spike?" into a passive question. Drive it from cron (e.g. Mondays):

0 13 * * 1 root /usr/local/bin/cpu-investigator.sh

Retire the report once the cause is fixed and confirmed. A diagnostic that always says "all clear" becomes noise. If the script and cron are Ansible-managed, gate them behind a host variable rather than hand-editing the box (which drifts back on the next run):

# host_vars/<host>/vars.yml
cpu_investigator_enabled: false   # template deploys the cron commented-out

Leave atop itself sampling — the history is free insurance — and flip the toggle back to true if spikes ever return.

Key Takeaways¶

Install atop before you need it — recurring spikes can't be diagnosed from top after the fact.
atopsar -O names the culprit; aggregate-CPU parsing invents phantom 100% spikes.
Layered causes: snapd, WP-Cron, Apache MPM model, and finally the single-vCPU ceiling — fix in that order, resize last.
Hypervisor monitors can't tell polite CPU from a runaway — scope per-host alerts accordingly.
Retire diagnostics once resolved via a config toggle, not a hand-edit; keep the data collection running.