Skip to content

2026-10-24

Silent agent failures are the default, not the exception

Why an AI agent that stops running rarely pages anyone, and the dead-man's switch pattern that fixes it.

A crashed web server usually tells you. A request fails, a health check flips, a user writes in. An agent that was supposed to triage the inbox at 7am does none of that. It simply does not run. The queue grows, and the only symptom is tomorrow's complaint.

Success is indistinguishable from absence

Most agent logs are full of the last good run. Nothing writes a new line when the process is gone, the cron was disabled, the n8n credential expired, or the Vercel project was paused. Monitoring the model provider does not help either: their status page is green because nobody called them.

The reliable signal is the opposite of a metric. You decide how often the job must prove it is alive, and you treat a missing proof as the incident. That is a dead-man's switch. Factories have used the idea for a century. Cron monitors productized it for shell scripts. Agents need the same wire.

Where to put the ping

  • At the end of an n8n, Make, or Zapier scenario, after the work has actually happened.
  • In the finally path of an agent loop, with /fail on the error path so a crash is not waiting on the whole interval.
  • In a Claude Code Stop hook if the thing you care about is that a session finished.
  • As the last step of a GitHub Actions job, using if: always() so failures ping /fail.

Grace periods exist because networks jitter. A five-minute job with a one-minute grace will page you at six minutes, not at five minutes and one millisecond. Start the grace at something you would still consider late, not at zero.

Recovery is part of the alert

A down alert that never clears trains people to ignore the channel. Nightshift sends a second message when a success ping arrives after a down state, so the channel shows both the incident and the end of it.