Disaster recovery

Rebuild a VAC instance from a backup after host loss — and why the master key comes first.

If the host dies, recovering VAC comes down to three artifacts: the master key, the control-plane database, and (optionally) the certificate store. Get those back and your apps' configuration, secrets, and history are intact.

The master key comes first

No master key, no secrets

VAC_MASTER_KEY decrypts every secret VAC holds — app env vars, SSH deploy keys, TOTP secrets, notification URLs, and backup credentials. If you lose it, those values are unrecoverable, even with a full database backup. Everything else can be rebuilt; the key cannot.

The key lives in /opt/vac/.env. Back that file up off the host, somewhere separate from your other backups (a password manager or secrets vault is ideal). This is the single most important recovery step.

Taking a full snapshot

The uninstall script doubles as a snapshot tool — its --backup mode writes three timestamped artifacts before doing anything else:

bashroot@your-vps:~#

sudo vac uninstall --backup /mnt/backups --yes

It produces:

a Postgres dump — all apps, deployments, services, and encrypted secrets,
a caddy_data tarball — issued Let's Encrypt certificates and the ACME account,
an install-directory tarball — including .env with the master key.

You can run the backup without the destructive flags to snapshot a live box; combine with --purge/--apps only when you actually intend to tear down.

Restoring on a fresh box

Install VAC normally on the new host (curl -sSL get.vac.vojir.io | sudo sh).
Restore the environment — replace /opt/vac/.env with your backup's, so VAC_MASTER_KEY and VAC_DB_PASSWORD match the dumped database.
Restore the database — stop the API, load the Postgres dump into vac-db, and bring it back up:

bashroot@your-vps:~#

vac down
docker exec -i vac-db psql -U vac -d vac < vac-db-backup.sql
vac up

Restore certificates (optional) — extract the caddy_data tarball into the caddy_data volume to skip re-issuance and avoid Let's Encrypt rate limits. If you skip this, Caddy simply re-issues certs on demand as hostnames are hit again — fine for a handful of domains.

Once the control plane is back with the original key and database, your apps' configuration and secrets are exactly as they were. Trigger a deploy per app to bring the workloads back up.

Recovering app data

The steps above recover VAC's state, not your apps' data. For that, restore from the per-service backups you configured — load the latest dump back into the relevant service after it's running.

A recovery drill is worth doing

Test the restore, not just the backup

A backup you've never restored is a hope, not a plan. Spin up a throwaway VPS, restore into it, and confirm you can log in and your apps deploy. Do it once now so you're not learning the procedure during an actual outage.

Operations

Backups

Set up the per-service backups you'll restore from.

Reference

The vac CLI

The host commands used above: up, down, uninstall.