Ansible Check Mode False Positives in Verify/Assert Tasks¶

The Problem¶

ansible-playbook --check (dry-run mode) reports failures on verify and assert tasks that depend on handler-triggered side effects — even when the playbook is correct and would succeed on a real run.

Symptom: Running --check produces errors like:

TASK [Assert hardened settings are active] ***
fatal: [host]: FAILED! => {
    "assertion": "'permitrootlogin without-password' in sshd_effective.stdout",
    "msg": "One or more SSH hardening settings not effective"
}

But a real run (ansible-playbook without --check) succeeds cleanly.

Why It Happens¶

In check mode, Ansible simulates tasks but does not execute handlers. This means:

A config file task reports changed (it would deploy the file)
The handler (reload sshd, reload firewalld, etc.) is never fired
A subsequent verify task runs sshd -T or ufw status verbose against the current live state (pre-change)
The assert compares the current state against the expected post-change state and fails

The verify task is reading reality accurately — the change hasn't happened yet — but the failure is misleading. It suggests the playbook is broken when it's actually correct.

The Fix¶

Guard verify and assert tasks that depend on handler side effects with when: not ansible_check_mode:

- name: Verify effective SSH settings post-reload
  ansible.builtin.command:
    cmd: sshd -T
  register: sshd_effective
  changed_when: false
  when: not ansible_check_mode   # sshd hasn't reloaded in check mode

- name: Assert hardened settings are active
  ansible.builtin.assert:
    that:
      - "'permitrootlogin without-password' in sshd_effective.stdout"
      - "'x11forwarding no' in sshd_effective.stdout"
    fail_msg: "SSH hardening settings not effective — check for conflicting config"
  when: not ansible_check_mode   # result would be pre-change state

This skips the verify/assert during check mode (where they'd produce false failures) while keeping them active on real runs (where they catch actual misconfigurations).

When to Apply This Guard¶

Apply when: not ansible_check_mode to any task that:

Reads the active/effective state of a service after a config change (sshd -T, ufw status verbose, firewall-cmd --list-all, nginx -T)
Asserts that the post-change state matches expectations
Depends on a handler having fired first (service reload, daemon restart)

Don't apply it to tasks that check pre-existing state (e.g., verifying a file exists before modifying it) — those are valid in check mode.

Common Patterns¶

SSH daemon verify¶

- name: Verify effective sshd settings
  ansible.builtin.command: sshd -T
  register: sshd_out
  changed_when: false
  when: not ansible_check_mode

- name: Assert sshd hardening active
  ansible.builtin.assert:
    that:
      - "'maxauthtries 3' in sshd_out.stdout"
  when: not ansible_check_mode

UFW status verify¶

- name: Show UFW status
  ansible.builtin.command: ufw status verbose
  register: ufw_status
  changed_when: false
  when: not ansible_check_mode

- name: Confirm default deny incoming
  ansible.builtin.assert:
    that:
      - "'Default: deny (incoming)' in ufw_status.stdout"
  when: not ansible_check_mode

nginx config verify¶

- name: Test nginx config
  ansible.builtin.command: nginx -t
  changed_when: false
  when: not ansible_check_mode

Related pattern: `command` / `shell` skipped in check mode¶

The inverse problem shows up when a playbook registers output from ansible.builtin.command (or shell) and uses that output in a downstream conditional or set_fact. In check mode, command is skipped by default — the registered variable comes back with no stdout, and any in / containment check against it silently evaluates to False.

Saw this 2026-04-19 while writing fix_ebtables_usrmerge.yml. The play queried update-alternatives --display ebtables to detect whether a host ran the nft or legacy backend, then branched on that fact. Under --check, the query was skipped, the fact defaulted to legacy on every host, and the next task's existence check failed on the nft hosts (/usr/bin/ebtables-legacy not found). A real run was fine — but --check output looked like the playbook was broken.

Fix: force the detection task to run even in check mode, since it's a read-only query with no side effects.

- name: Query current ebtables alternative
  ansible.builtin.command: update-alternatives --display ebtables
  register: alt_query
  changed_when: false
  failed_when: false
  check_mode: false        # force execution in --check so downstream conditionals see real data

Apply this to any command/shell task whose output feeds a when:, set_fact, or similar logic. Only safe when the task is genuinely read-only.

Escalation: a hard error, not just a silent false-negative¶

Confirmed twice more on 2026-08-05, during a fleet-wide audit prompted by scoping a weekly --check --diff drift-detection job (see Fleet-Security-Hardening). Both were worse than the ebtables case above — instead of silently evaluating a containment check to False, the missing registered variable caused a hard error partway through the run:

roles/root_password: a command check (passwd -S root equivalent) fed a when: gating the real password-set task. Skipped under --check, the when: dereferenced an undefined .rc, erroring the task on every host in steady state (i.e. every host that already had a password set).
roles/swap: two bare command tasks — "Get existing swapfile size in bytes" (stat -c %s) and "Verify swap is active" (swapon --show) — fed downstream when:/assert logic the same way. The first hard-errored the when: clause's arithmetic comparison (swapfile_size_bytes.stdout | int) on any host where the swapfile already existed; the second left the final assert evaluating an empty string, failing it even though swap was genuinely active.

Takeaway: whether a missing check-mode value produces a silent wrong answer or a hard crash depends on how the downstream task consumes it — a plain in containment check tends to just evaluate False (ebtables case); an arithmetic comparison or filter on an undefined value (| int, .rc, .stdout indexing) tends to error outright. Either way, the fix is the same: check_mode: false on the upstream read-only task. Worth grepping a role for every bare command/shell task and checking whether its register'd variable feeds anything downstream before assuming --check is safe to rely on for drift detection — a "looks read-only" role can still fail this way on its very first full-fleet --check run if nobody's exercised that path before.

Trade-off¶

Guarding with when: not ansible_check_mode means check mode won't validate these assertions. The benefit — no false failures — outweighs the gap because:

Check mode is showing you what would change, not whether the result is valid
Real runs still assert and will catch actual misconfigurations
The alternative (failing check runs) erodes trust in --check output

If you need to verify the effective post-change state in check mode, consider splitting the playbook into a deploy pass and a separate verify-only playbook run without --check.