Alert Script Never Emails: set -e + && Short-Circuit Kills the Run That Matters¶
You write a "only bother me when something's wrong" cron script. It exits cleanly in testing. It never emails you — and the one run where it had something to report is exactly the run that failed silently.
This is a nasty class of bug because the failure is inverted: the script works perfectly on every quiet day and breaks only when it finally has news. If you test it on a quiet system and ship it, you've deployed a monitor that is guaranteed never to fire.
Symptoms¶
- Script exits
0and sends nothing when there's nothing to report. Looks correct. - Script exits
1and sends nothing when there is something to report. - No error in the mail log, because
sendmailwas never reached. bash -npasses — it's not a syntax error.
Root cause 1 — set -e treats a false && list as a failure¶
The idiomatic-looking early return:
set -euo pipefail
COUNT=$(...)
[ "$COUNT" -eq 0 ] && exit 0 # ← the bug
# ... build report, send mail ...
When COUNT is 0, the test succeeds, exit 0 runs, and the script exits quietly. Correct.
When COUNT is non-zero, the test fails. && short-circuits, so the whole list evaluates to non-zero. Under set -e, a compound list that evaluates false is a failed command, and bash aborts the script right there — before it ever builds the report.
set -e deliberately ignores failures on the left of && (that's what makes cmd && other safe), but it does not ignore the failure of the list as a whole when it's used as a standalone statement.
Fix — use an explicit if¶
Verbose, but it means what it says and set -e has nothing to trip on.
Other spellings that also work:
[ "$COUNT" -eq 0 ] && exit 0 || true # neutralize the list's exit status
(( COUNT == 0 )) && exit 0 || : # same idea
Prefer the if block. The || true versions work but read like an accident waiting to be "cleaned up" by a future you.
Root cause 2 — psql does not interpolate :vars inside -c¶
Frequently in the same script, because alert scripts are usually query-driven:
psql mydb -v thr=5 -c "SELECT count(*) FROM t WHERE n >= :thr;"
# ERROR: syntax error at or near ":"
-v variables are expanded by psql's interactive/file parser, not by the single-command -c path. The :thr is handed to the server literally and Postgres rejects it.
Fix — inline it in the shell, or use a file / stdin¶
THR=5
psql mydb -c "SELECT count(*) FROM t WHERE n >= ${THR};" # shell-inlined
psql mydb -v thr=5 -f query.sql # -f honors :vars
psql mydb -v thr=5 <<< "SELECT count(*) FROM t WHERE n >= :thr;" # stdin honors :vars
Only inline values you control
Shell-inlining a literal integer constant is fine. Never inline untrusted input into SQL this way — use -f/stdin with :vars, or a real parameterized client.
Diagnosis checklist¶
- Run it with the condition forced true. This is the whole lesson. Temporarily lower the threshold, or relax a filter, so the script must take the positive branch.
bash -x /path/to/script— watch for the run stopping immediately after the count.- Check the exit code explicitly:
./script; echo "exit=$?". An alert script that exits1on a day it had findings is the signature. - Stub the sender (
sed 's|/usr/sbin/sendmail -t|cat|') so you can see the rendered message without mailing yourself. - Only after the positive path renders, do a real send and confirm delivery in the mail log (
status=sent) and thatmailqis empty.
Takeaway¶
A conditional alert script is only proven when you've seen it fire. A clean exit on a quiet system proves nothing — it exercises the one branch that was always going to work. Force the positive path during install, every time.
The general form of the set -e trap: set -euo pipefail is good advice that assumes every non-zero exit is an error. In alert scripts, a non-zero exit is often the signal. Wherever a test's result is data rather than a failure, use if.