Skip to content

Forgejo Actions: Jobs Finish, Then Are Marked Failed ~12 Minutes Later (Runner Too Old for the Server)

The Problem

After a Forgejo server update, every CI run looks broken in the Actions tab — red, and "running" for ten-plus minutes — even though the job itself did everything right.

  • Every step of the job shows success (or fails for its own, real reason), and the runner's own log says Job succeeded (or Job failed) within seconds.
  • Forgejo keeps the run in running for roughly 11–14 minutes, then marks it failed.
  • The job's duration is far longer than its steps (e.g. 708 s for a job whose last step finished after 5 s), and two unrelated jobs stopped in the same second.
  • The server log has, at that moment, a line like stopTasks() [W] Cannot transfer logs of task N: dbfs open "actions_log/…": file does not exist.
  • Nothing is rejected on the wire: the runner's requests to the server are all 200 OK, so it is not a firewall, DNS or TLS problem.

Why It Happens

The runner talks to the server throughout a job (progress and log updates), and at the very end sends one last message with the final result. Here the progress updates arrived and were accepted, but the final result never registered. Forgejo's own cleanup routine treats a task that stops reporting as abandoned, force-stops it after about ten minutes, and records it as failed — which is why the failure time has nothing to do with when the job actually ended.

The runner in this case was v6.3.1 (a March 2025 build) talking to Forgejo 15.0.10 — several runner major versions behind the server. Before the server update, the same runner had reported 261 successful jobs and promptly reported failed ones, so the pairing worked until the server changed.

What is and isn't proven: the server update was the only change on the box, and upgrading the runner fixed it completely — strong evidence of a runner/server compatibility gap. The server was not rolled back to bisect, so treat "15.0.10 introduced it" as the likely trigger, not a confirmed one.

Diagnose (all read-only)

1. Compare the pairing.

forgejo-runner --version
docker exec forgejo forgejo --version      # adjust the container name

2. Look at recent job outcomes in the server's database (SQLite install shown; the container name and data path are from a standard Docker setup — adjust to yours). Status codes: 1 success, 2 failure, 3 cancelled, 5 waiting, 6 running.

docker exec -i forgejo sqlite3 -readonly -header -column /data/gitea/gitea.db <<'EOF'
select t.id, j.name as job, t.status,
       datetime(t.started,'unixepoch','localtime') as started,
       t.stopped - t.started as secs
from action_task t
join action_run_job j on j.id = t.job_id
order by t.id desc limit 15;
EOF

The signature of this fault: status = 2 with secs around 700–860 on jobs whose steps took seconds. Fast failures (tens of seconds) that were reported correctly are ordinary, real failures — don't confuse the two.

3. Check step timings to confirm the steps themselves finished early:

docker exec -i forgejo sqlite3 -readonly -header -column /data/gitea/gitea.db <<'EOF'
select task_id, "index" as i, name, status,
       datetime(started,'unixepoch','localtime') as started,
       datetime(stopped,'unixepoch','localtime') as stopped
from action_task_step where task_id = 123 order by "index";   -- put your task id here
EOF

4. See the reaper and the runner's own verdict.

docker logs forgejo 2>&1 | grep 'stopTasks()'            # the force-stop lines
journalctl -u forgejo-runner --since "-1h"                 # runner side
zstd -dc <data>/gitea/data/actions_log/<owner>/<repo>/<NN>/<task-id>.log.zst | tail   # job log ends "Job succeeded/failed"

If the job log ends with a verdict but the database says the task was stopped ~12 minutes later by the reaper, this is your problem.

Fix: upgrade the runner

  1. Make sure nothing is running (select count(*) from action_run_job where status in (5,6); should be 0).
  2. Download the current runner from the Forgejo runner releases — the binary and its .asc signature — into a scratch folder, not the live path, and verify the signature against the signing-key fingerprint published in the Forgejo docs (gpg --verify file.asc file, expect "Good signature from Forgejo").
  3. Back up the old binary, config.yaml and the .runner registration file.
  4. Swap it while the service is stopped, and restore the SELinux label if you run an enforcing system (the binary should stay bin_t):
systemctl stop forgejo-runner
install -o root -g root -m 755 forgejo-runner-<ver>-linux-amd64 /usr/local/bin/forgejo-runner
restorecon -v /usr/local/bin/forgejo-runner
systemctl start forgejo-runner
journalctl -u forgejo-runner -n 5      # expect: version <new>, "declared successfully"

The registration and labels carry over; no config change was needed going from v6.3.1 to v13.1.0.

Check the release notes first — a jump of several major versions can break workflows. For v13, the ones that matter: workflow syntax errors now fail the job (they used to warn), add-path / set-output / set-env are gone, Docker 25 or newer is required, the container.network_mode config key was removed (use network), and from v13.1.0 job commands no longer get a TTY (a step running docker run -it fails — drop the -t).

Verify

Re-run any job (Actions tab → Re-run, no code change needed). A healthy run finishes in seconds with the true result and no new stopTasks() lines in the server log. Run the database query again: the new row's secs should match the real duration.

Rollback

Stop the service, install the backed-up old binary the same way, restorecon, start. Seconds; no re-registration.