Heartbeats and yarrix run

A heartbeat answers "did the job run?". Instead of Yarrix looking at the job, the job tells Yarrix. If a ping does not arrive on schedule, Yarrix alerts. This is the right tool for cron jobs, backups, and anything that fails by not happening.

Declare one#

In checks/*.yaml (or under heartbeats: in a host file):

heartbeats:
  - id: nightly-report
    schedule: "0 6 * * *"        # when the job should run: a cron expression, or an interval such as 6h
    tz: UTC                      # the zone a cron expression is read in (ignored for an interval)
    grace: 15m                   # how late a ping may be before it alerts (default 10m)
    max_duration: 30m            # optional: a started run that takes longer alerts
    severity: warn               # optional: info, warn or critical
    description: Builds the nightly sales report   # optional, plain text, up to 2000 characters
    owner: data-team             # optional, who to ask, up to 120 characters
    runbook: https://wiki.example.net/runbooks/nightly-report   # optional, http or https only
    tags: [nightly, reports]     # optional, matched by `when: { tag: ... }` in alert rules
    keywords:                    # optional: judge the job's output, see below
      fail_if_present: ["ERROR", "Traceback"]
      fail_if_absent: ["report written"]

Schedules: an interval or cron#

schedule takes two forms:

• A duration such as 6h, 30m or 2d (one number and ms, s, m, h or d; at least 1 second). It means "at least every 6 hours, counted from the last successful ping". A heartbeat created or re-enabled with no ping yet counts from then. tz is ignored. Example: with schedule: 6h and grace: 10m, a last success 6 h 5 m ago is late, and missed once 6 h 10 m have passed.
• A five-field cron expression such as 0 6 * * *, read in tz (an IANA zone name, default UTC). The job is due at each matching time.

The web app shows an interval as "every 6h" and a cron expression as written.

Describe it: description, owner, runbook, tags#

These four fields are for the person who gets the alert at 3am. They are not secret and are returned by GET /api/v1/heartbeats, yarrix_get and shown in the web app: the owner and tags in each row of the list, and all four fields in the heartbeat's drawer.

• description: what the job is. owner: who to ask. Both plain text.
• runbook: a link to what to do. Only http:// and https:// are accepted; anything else (such as javascript:) is a config error. The web app shows it as a link that opens in a new tab with rel="noopener noreferrer".
• tags: a heartbeat's tags take part in alert routing, exactly like an app's. A rule with when: { tag: nightly } matches a heartbeat tagged nightly (alert-routing.md).
• When a heartbeat has an owner or runbook, its alerts carry them: the owner and runbook fields of the payload, an Owner and a Runbook row in the Slack, Discord, ntfy and email renderings, and {{owner}} and {{runbook}} in a template destination (alert-delivery.md).

Judge the output: keywords#

A job can exit 0 and still have failed. keywords turns a success ping into a failure from what the job printed:

keywords:
  fail_if_present: ["ERROR", "Traceback"]   # any of these in the output: the ping is a failure
  fail_if_absent: ["report written"]        # all of these must appear in the output
  case_sensitive: false                     # default false

• The rules apply only to pings that would count as success: /ping/<token> and /ping/<token>/0. /fail, a non-zero exit, /start and /log are unchanged. Keywords never turn a failure into a success.
• A match records the ping as a failure and the state goes failed, with the reason output contains "ERROR" or output is missing "report written". It is routed like any other failure, and the next clean success clears it.
• With fail_if_absent, a success ping that carries no body (a plain GET) fails with no output to check. Send the output: POST it, or use yarrix run.
• Matching is plain substring matching, not a regular expression. At most 16 strings, each 1 to 128 characters; more is a config error that names the file and the path.
• The rules look at the raw body the hub received, before secret masking, so masking can neither hide nor create a match. Only the masked body is stored.
• yarrix run sends only the last 10 KB of the job's output, so keywords see that tail.

Yarrix generates the ping address and keeps it stable. To use a address of your own choosing, set ping_token (16 to 64 characters of A-Z a-z 0-9 _ -) and reference it as a secret (${env:...}).

Find the ping URL#

An admin sees it in the heartbeat's detail (the web app, or GET /api/v1/heartbeats/<id>, which returns ping_url). It looks like https://yarrix.example.net/ping/<token>. The token is the credential: anyone with the URL can mark the job healthy. The list endpoints, the MCP server and the CLI never include it.

Report from the job#

Using only curl:

URL=https://yarrix.example.net/ping/<token>

curl -fsS -m 10 --retry 3 "$URL/start"        # optional: the job began (needed for max_duration)
/usr/local/bin/nightly-report
curl -fsS -m 10 --retry 3 "$URL"              # success

A POST body (up to 10 KB) is stored and shown on the heartbeat. Send the tail of the job's output on failure:

nightly-report 2>&1 | tail -c 2000 | curl -fsS -m 10 --data-binary @- "$URL/fail"

Do not send secrets in the body. The hub masks common secret shapes before storing (Authorization headers, bearer tokens, password=-style assignments, cloud keys, private key blocks), but that is a safety net, not a guarantee. A ping URL accepts about 10 requests a minute; more get 429. An unknown token is a 404.

yarrix run: wrap a command#

The agent binary has a wrapper so you do not write the curl yourself:

yarrix run <token-or-url> -- /usr/local/bin/nightly-report --full

It sends start, runs the command with its output passed straight through, then sends the exit code with the last 10 KB of output. It exits with the command's own exit code, so cron and shell && chains behave as before. If the hub is unreachable, the job is not affected: pings are retried a few times and then dropped silently. Give a bare token with --hub (or YARRIX_HUB; the older YARRIX_HUB_URL still works), or give the full URL. For a hub with a private certificate authority, add --ca-file PEM (or set YARRIX_CA_FILE):

0 6 * * *  yarrix run https://yarrix.example.net/ping/<token> -- /usr/local/bin/nightly-report

yarrix ping <token-or-url> [start|fail|log|<0-255>] [--data TEXT|-] sends one ping without running anything. The binary also answers to the name hc-run (ln -s yarrix hc-run) for scripts written for that tool's wrapper, taking the same arguments after the URL.

(yarrix run and yarrix ping work in the same agent binary you use for monitoring; a host that only runs jobs does not need to be enrolled to use them.)

States and alerts#

late raises a warning, and missed and failed a critical alert, unless the heartbeat sets severity, which then applies to all three. The incident is routed like any other (alert-routing.md). Only a success ping clears failed; the next on-time success resolves the incident. A backup can use a heartbeat as its run report (backups.md).