flowchart TB accTitle: NutsNews VPS Restore accDescr { This is the restore runbook for encrypted VPS restic backups. } A["NutsNews VPS Restore"] --> B["NUTSNEWS_VPS_RESTORE.md"] B --> C["Auto-generated placeholder diagram"]

NutsNews VPS Restore

This is the restore runbook for encrypted VPS restic backups.

The short version: restore to a staging directory first, inspect it, then copy only what you actually need. Restores are surgery, not confetti.

Easy Summary

Backups are useful only if they restore. Otherwise they are emotional support files with timestamps.

To restore, you need:

restic
rclone
the nutsnews-onedrive rclone config
the restic repository password
a trusted machine or replacement VPS

The OneDrive folder is not directly readable because restic encrypted everything before rclone uploaded it. That is good. It also means losing the restic password is bad. Keep the password somewhere outside the VPS.

Intermediate Summary

The restic repository is:

rclone:nutsnews-onedrive:nutsnews-backups/vps

The restore flow is:

Prepare restic and rclone on a trusted host.
Set RCLONE_CONFIG, RESTIC_REPOSITORY, and RESTIC_PASSWORD_FILE.
List snapshots.
Restore the chosen snapshot to a staging directory.
Inspect restored files.
Copy selected data/config into place.
Restart services.
Verify the Ops Portal and backup verification workflow.

Do not blindly restore /etc over a running server like a digital leaf blower. Stage first.

Expert Summary

The VPS backup contains both runtime data and host restore clues:

/opt/nutsnews
/etc/nutsnews
NutsNews systemd units and timers
selected service configs managed by Ansible

The best restore shape is still GitOps-first:

rebuild or repair the host with Ansible
restore data and private config from restic
reconcile any restored config drift back into the infra repo
run validation

Restic is the data recovery path, not a replacement for source-controlled infrastructure.

Restore Flow

sequenceDiagram
  participant Human as Maintainer
  participant Host as Trusted restore host
  participant Restic as restic
  participant Rclone as rclone
  participant OneDrive as OneDrive ciphertext repo
  participant VPS as VPS

  Human->>Host: Provide rclone config and restic password file
  Host->>Restic: restic snapshots
  Restic->>Rclone: read repository through nutsnews-onedrive
  Rclone->>OneDrive: fetch encrypted repo data
  OneDrive-->>Rclone: encrypted blobs
  Rclone-->>Restic: ciphertext
  Restic-->>Host: decrypted file tree in staging
  Human->>Host: Inspect restored files
  Human->>VPS: Copy selected data/config
  Human->>VPS: Restart and verify services

Prepare A Restore Environment

Install restic and rclone on a trusted host.

Create local files:

/path/to/rclone.conf
/path/to/restic-password

Then export:

export RCLONE_CONFIG=/path/to/rclone.conf
export RESTIC_REPOSITORY=rclone:nutsnews-onedrive:nutsnews-backups/vps
export RESTIC_PASSWORD_FILE=/path/to/restic-password

Confirm access:

restic snapshots
restic ls latest
restic check --read-data-subset=5%

Restore To Staging

Use a staging directory. Always.

sudo install -m 0700 -d /tmp/nutsnews-restore
sudo -E restic restore latest --target /tmp/nutsnews-restore

Inspect:

sudo find /tmp/nutsnews-restore -maxdepth 3 -type d | sort
sudo ls -la /tmp/nutsnews-restore/opt/nutsnews
sudo ls -la /tmp/nutsnews-restore/etc/nutsnews
sudo find /tmp/nutsnews-restore/etc/systemd/system -maxdepth 1 -name 'nutsnews-*' -print

Restore Selected Data

For app/runtime data:

sudo rsync -aHAX --numeric-ids \
  /tmp/nutsnews-restore/opt/nutsnews/data/ \
  /opt/nutsnews/data/

For private config:

sudo rsync -aHAX --numeric-ids \
  /tmp/nutsnews-restore/etc/nutsnews/ \
  /etc/nutsnews/
sudo chmod 0700 /etc/nutsnews
sudo find /etc/nutsnews -type f -exec chmod 0600 {} \;

Use judgment. If the infra repo now manages a config file differently, prefer the repo and only restore the secret/data portion that cannot be recreated.

Restore Test Procedure

Run this after initial setup and periodically after meaningful changes:

The scheduled nutsnews-restic-verify.timer is useful routine evidence, but it is not this restore test. It checks repository readability and the latest snapshot from the VPS. A restore test still needs a trusted restore host, a staging directory, and human inspection of restored files. The recurring full restore drill is tracked separately in infra issue #24.

Run the Run VPS Backup workflow.
Run the Verify VPS Backup workflow.
On a trusted restore host, run restic snapshots.
Restore latest to /tmp/nutsnews-restore.
Confirm these paths exist:
- /tmp/nutsnews-restore/opt/nutsnews
- /tmp/nutsnews-restore/etc/nutsnews
- /tmp/nutsnews-restore/etc/systemd/system
Confirm /opt/nutsnews/data is present if app data exists in production.
Run restic check --read-data-subset=5%.
Record the snapshot ID, date, and result.
Delete the staging directory when finished.

Restore tests are the adult in the room. They are also how we find out that the backup is a backup instead of a very organized wish.

After Restore

Reload systemd if unit files changed:

sudo systemctl daemon-reload

Check services:

systemctl status nutsnews-ops-portal-collector.timer --no-pager
systemctl status nutsnews-restic-backup.timer --no-pager

Check the portal:

curl -fsS http://127.0.0.1:8080/healthz
curl -fsS http://127.0.0.1:8080/data/status.json

Then run:

Run VPS Backup
Verify VPS Backup

Primary diagram