Backups · Yarrix

Yarrix does not take backups. It watches the jobs you already have and, where it can, checks the result at the destination instead of trusting the job's own "success".

Restore tests are not built yet. Check B9 always reads skip with "restore tests arrive in a later release". A backup Yarrix calls healthy has been seen to run, to finish, and to look right at the destination; it has not been proven restorable. Test a restore yourself now and then.

How a backup is reported#

A backup is declared under backups: in a host file. A run is reported in one of three ways; pick whichever your job can do:

Way	How
A heartbeat	Set `heartbeat: <id>` on the backup and ping that heartbeat from the job (heartbeats.md). `yarrix run <token> -- <command>` pings start and the exit code for you.
The report command	`yarrix report <backup> start`, then `success` or `fail` (optionally `--bytes`, `--files`, `--exit`, `--message`). Needs the report token minted for that backup (`--token`, or see `yarrix report --help`).
The report API	`POST /api/v1/backups/{id}/report` with a report token. An admin signed in to the web app makes the token for one backup from the backup's drawer, under Report token (`POST /api/v1/backups/{id}/report-token`). It is shown once, and making a new one stops the old one at once: a job still using it gets 401.

After the destination changes or a problem is fixed, Re-verify in the backup's drawer (operator or admin) runs the checks that read the destination, the log and the copy again for the newest run, without waiting for the next one (POST /api/v1/backups/{id}/verify).

The nine checks#

Check	Question	Needs
B1	Did the job exit successfully?	A report, a `yarrix run` wrapper or a heartbeat exit code
B2	Does the job's output contain error lines?	The wrapper's output tail, or `log:` read by an agent
B3	Is something at the destination newer than the run's start? (read from the destination's own timestamps)	A destination the hub or an agent can reach
B4	Is the size plausible?	`size_band` and/or `max_drop_pct`
B5	Is the newest copy intact?	`integrity:`
B6	Is the number of copies within `retention` min and max?	`retention:`
B7	Is the destination on the right disk, with room?	`dest_mount` and/or `dest_min_free`
B8	Did a run succeed on schedule (within `grace`)?	`schedule:`
B9	Can it be restored?	Not built: always `skip`

A check you did not configure is skip. A check you configured but that cannot be observed (a tool missing, no access to the destination) is unknown, and an unknown stops the run from counting as a success: silence is never reported as healthy.

Types, and which checks apply#

type: is a label that only selects the pattern pack B2 uses. It does not change how the job runs or how the others are judged.

`type`	B2 pattern pack	Integrity methods that fit
`rsync`	rsync	`checksum_sample`, `named`
`restic`	restic	`restic_check`
`borg`	borg	`named` (for example a `borg check` wrapper)
`tar`	tar	`tar_list`, `tar_entries`, `zip_test`
`sqlite`	sqlite	`sqlite_integrity`
`rclone`	rclone	`checksum_sample`, `named`
`ccc`	Carbon Copy Cloner	`named`
`time_machine`	Time Machine	`tmutil_verify`
anything else	generic	any

The "integrity methods that fit" column is a suggestion from what each method does, not a restriction: the hub accepts any method for any type.

B1, B3, B4, B6, B7 and B8 work the same whatever the type; the sections below describe the fields.

Destination fields (B3, B4, B6, B7)#

dest_min_free and size_band / max_drop_pct are in the config schema.

Field	Meaning
`dest_mount`	B7. `true`: the destination must not sit on the root filesystem. A path: that must be the mount point it sits on.
`size_method`	B4 size source: `report` (bytes the job reported), `artifact` (size of the newest file at the destination) or `du` (bounded du of the destination). Default: report when bytes were reported, else artifact.
`retention.glob`	Which names under the destination count as snapshots for B6 (default `*`).
`allow_paths` (agent.yaml)	Directories the agent may read. Default-deny: a path outside it, or one that reaches outside it through a symlink, is refused and the check is `unknown`.

Log patterns (B2)#

B2 looks for error lines in the job's output. Two sources, either or both:

the tail a yarrix run wrapper sends with its closing ping; no setup needed;
log: { path: /var/log/job.log } (or command: <name> for a named command in agent.yaml, with host:), read through the agent. Only lines written since the last read are scanned, so an old error does not fail every later run. The offset is stored per backup and follows a rotated file (new inode, or a smaller size).

Each backup type has a pattern pack (packages/backup-patterns/*.yaml: rsync, restic, borg, tar, sqlite, rclone, ccc, time_machine, generic). A line is an error when one of the pack's errors expressions matches (case-insensitive) and none of its ignore expressions does. error_patterns: [regex, ...] adds your own. The failing line, with secrets masked, is the reason shown.

Limits: the first read of a log takes its newest 256 KB. After that the agent pages forward from the saved offset; a page that is clean but leaves more waiting is unknown (never pass), and the next run continues from where the page ended. B2 over SSH is not built; the log source needs the agent. A missing, unreadable or out-of-allow_paths log is unknown.

Integrity (B5)#

integrity: takes a method name, or an object. One check per run, on the host that holds the copy.

Method	Checks
`sqlite_integrity`	`PRAGMA quick_check` on the newest file under `dest.path` (`mode: full` for `integrity_check`).
`tar_list`	The newest archive reads end to end.
`zip_test`	The newest zip passes `unzip -t`.
`tar_entries`	Every name in `entries: [...]` is in the newest archive. Needs GNU tar or bsdtar. A corrupt archive fails; a member name containing a newline is a failure, never an entry.
`checksum_sample`	Hashes `sample` random files (seeded by `seed`) at the source and the destination and compares. Assumes the destination mirrors the source layout and the files are not changing.
`restic_check`	`restic check`, reading `sample` percent of the data. The password comes from the host's environment (`RESTIC_PASSWORD*`), or `password_file` over SSH.
`tmutil_verify` (alias `tmutil_verifychecksums`)	`tmutil verifychecksums` on the latest backup. Needs root; without it the result is `unknown`.
`named`	Runs `command: <name>` from `agent.yaml`. Exit 0 passes; any other exit fails with its first output line.

glob narrows which file is picked. Over SSH the host's own tools are used; a missing tool is unknown: tool missing. Windows destinations are unknown. An empty or missing destination is a failure.

The hub's own database backup (yarrix-db) keeps its in-process integrity_check.

Restore tests (B9)#

Always skip: "restore tests arrive in a later release". It is stored on every run and shown as restore_test, and it does not hold a run back from success.