feat(jobs): run the thumbnail migration at startup, by default
A migration nobody triggers never finishes. Scheduled ticks deliberately never pass `repair`, so a deployment whose operator never opens the admin panel re-imported the same sidecars forever and never drained the directory — and relying on operators to edit `.env` has the same failure mode one level up. `OXICLOUD_STARTUP_JOBS` dispatches named jobs once, in the background, after the scheduler is ready. Entries use the syntax operators already type at the trigger URL (`name?repair=true`), so the value is literally the request they would otherwise make by hand. It defaults to both migration jobs in repair mode, so an untouched deployment migrates and drains itself. That is a destructive default and a real exception to no-silent-auto-repair, so the guard it rests on had to get stronger: `verify_and_unlink` now compares CONTENT, not length. A blob of the right size and the wrong bytes used to pass — a key-mapping bug handing back another file's preview at the same length would have deleted the original and kept the impostor, and thumbnails cluster tightly enough in size for that to be a real coincidence. The readback streams from the backend with no cache in front, so it proves durability rather than that a write was acknowledged. Deletion of `.thumbnails/` is attempted first and only falls back to renaming it `.thumbnails.migrated` when `remove_dir` refuses because a non-sidecar file is inside (Finder's `.DS_Store`). Either way the directory stops existing, which lets the read-path probe go back to a single `stat` on the root instead of walking the size directories. Validation is fail-fast: an unknown job name or flag panics at boot. A silently dropped `?repare=true` would leave the job in discovery-only mode while the operator believed the tier was draining, surfacing months later as "the migration never finished" with nothing pointing at the config line. Interrupted runs resume. Boot recovery flips abandoned rows to Paused with their cursor, so `run_or_resume` continues rather than rescanning — a long migration completes across however many restarts it takes. That is a scoped exception to "we do not auto-resume": here somebody did ask, in configuration, and not having to ask again is the point. `StartupJob` holds a `JobRunArgs` rather than re-listing its four fields, so a fifth flag cannot be added to the scheduler and silently ignored in configuration. Jobs named here are ordinary registered jobs — visible in the panel, triggerable by hand, same runs and findings. Their rows now carry a `startup` object so an operator can see that a job deletes on every boot rather than only when someone clicks Run. Adds docs/config/thumbnail-migration.md: what runs on first boot, how to snapshot database and storage together beforehand, and how to verify afterwards with satellites_consistency plus backend_consistency ?deep=true. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+30
@@ -55,6 +55,36 @@ OXICLOUD_SERVER_HOST=0.0.0.0
|
||||
# Recommended: 127.0.0.1:9090 with node_exporter-style scrapers.
|
||||
#OXICLOUD_METRICS_LISTEN=127.0.0.1:9090
|
||||
|
||||
# ── Startup jobs ──────────────────────────────────────────────────────
|
||||
# Background jobs dispatched once, after the scheduler is ready.
|
||||
# Comma-separated; each entry is a registered job name, optionally with
|
||||
# the same flags the admin trigger URL takes (force, deep, repair,
|
||||
# storage).
|
||||
#
|
||||
# DEFAULT (applied when this variable is unset):
|
||||
# thumb_derived_import?repair=true,thumb_attached_import?repair=true
|
||||
#
|
||||
# Those two migrate thumbnails out of the legacy .thumbnails/ directory
|
||||
# into blob storage and then delete the originals, so the migration
|
||||
# completes without anyone having to trigger it from the admin panel.
|
||||
# Each sidecar is read back through the normal stack before it is
|
||||
# unlinked, and every deletion is written to the audit log.
|
||||
#
|
||||
# Dispatch is non-blocking — startup never waits on a job. A run
|
||||
# interrupted by a restart resumes from its cursor on the next boot, so
|
||||
# a long migration finishes across restarts. Safe to leave at the
|
||||
# default: the jobs are idempotent, and once the directory is drained a
|
||||
# run does nothing at all.
|
||||
#
|
||||
# An unknown job name or flag is a FATAL error at boot, not a warning —
|
||||
# a silently ignored entry means a migration that never runs.
|
||||
#
|
||||
# To disable every startup job, set this to the empty value:
|
||||
#OXICLOUD_STARTUP_JOBS=
|
||||
#
|
||||
# To import without deleting (inspect first, delete later by hand):
|
||||
#OXICLOUD_STARTUP_JOBS=thumb_derived_import,thumb_attached_import
|
||||
|
||||
# ── Upload size caps ──────────────────────────────────────────────────
|
||||
# See docs/config/storage-fine-tuning.md for sizing guidance.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user