From Ed's local→S3 outage run, which paused correctly and then said:
event="job.run" job=backend_migration outcome="ok"
... "paused":true,"retryable":true
This is the State-vs-Outcome distinction Ed drew earlier, in a channel
the earlier fix did not touch. The admin panel now separates the two;
the scheduler's own log line only ever carried the outcome, so a
migration frozen on an unreachable backend read as a clean run at INFO.
`JobOutcome` has just `Ok` and `Err`, and a pause is carried as `Ok`
with `paused: true` in `extra` — correct in itself: the handler did its
job and stopped cleanly at a checkpoint. The persisted shape is
unchanged for that reason. But projecting it to `outcome="ok"` tells an
operator the opposite of what they need to know, which is that nothing
will progress until the backend returns and someone resumes.
Paused runs now log at WARN with `outcome="paused"`, a `retryable`
field, and a message saying so. `grep 'outcome="ok"'` no longer matches
a blocked migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`JobRunArgs` was a fixed struct — `force`, `deep`, `storage`, `repair` —
and six places hardcoded that same list: the engine's persist/restore,
the trigger endpoint's query type, the OXICLOUD_STARTUP_JOBS parser, the
frontend API wrapper, the panel's checkboxes, and `StartupTrigger` on
the wire.
Two costs. Adding a parameter meant editing all six, and forgetting one
dropped it silently — most damagingly in persist/restore, where a
resumed run lost it and a `?repair=true` migration came back as
discovery-only after a restart. And the panel offered the same knobs on
every job: only two jobs read `deep`, six read `repair`, so most of
those controls did nothing with no way to tell which.
Now `JobHandler::parameters()` returns `&'static [JobParam]` — name,
type (boolean/string/number), default, and the job's own description of
what it does. `JobRunArgs` holds a map keyed by those names.
Everything reads the declaration:
* `run_or_resume` iterates it to persist and restore, replacing
`const FLAGS` plus a `storage` special case. `storage` stops being
special — it was the one Option<String> among three bools.
* `dispatch` normalises every run against it, which is what makes "a
handler sees its declared parameters with their declared defaults"
true rather than usual. The periodic tick passes an empty
`JobRunArgs::default()`, so a `default: true` parameter would
otherwise read false on every scheduled run.
* The trigger endpoint takes free-form query params and rejects
undeclared ones with a 400 naming the real set, instead of ignoring
them.
* OXICLOUD_STARTUP_JOBS keeps raw pairs (config is parsed before the
registry exists) and validates at dispatch, where the error can name
the job's actual parameters. Still a boot panic, same as an unknown
job name — a typo'd `?repare=true` must not leave a migration
importing forever in discovery mode.
* `JobSummary.parameters` carries it to the panel, whose `supportsDeep`
was a hardcoded name allowlist (`consistency_batch ||
backend_consistency`). A job gaining a deep mode needed a frontend
release; one losing it left a button that silently did nothing. The
menu now renders from the declaration, so a newly-declared boolean
appears with no frontend change.
Three consistency tenants were hand-rolling persist-on-fresh /
restore-on-resume for their own flag, under the same `params` key the
engine already used. Deleted — they read `args.get_bool(…)` now.
Fresh runs also filter to the declaration. `consistency_batch` forwards
its args verbatim to sub-jobs, so a tenant's `params` row could grow
`deep` with no deep mode, and the run-detail view would claim a mode the
job never had.
Two things found while wiring it, both worth knowing:
`RecoverableAdapter` bridges the two traits, and `parameters` has to be
forwarded there or the registry sees `&[]`. Both traits have defaults,
so omitting it compiled cleanly — and the trigger endpoint then rejected
`?repair=true` on the very jobs that declare it, with
OXICLOUD_STARTUP_JOBS panicking at boot. Now covered by
`adapter_forwards_job_metadata_from_inner_handler`.
`TriggerJobQuery` was briefly a newtype over the map. `serde_urlencoded`
cannot deserialize a newtype struct at the top level, so axum's `Query`
rejected EVERY trigger with a 400 — even one with no query string —
before the handler ran. It reads exactly like the new validation
rejecting something, which sent the first diagnosis to the wrong layer.
Now covered by `trigger_query_extracts_from_every_url_shape`.
Wire names are a compatibility surface: `params` rows are keyed by them
and the panel switches on them, so a rename breaks existing run history
the same way renaming a `Mutates` variant does. The JSON shape is pinned
in `snapshot_carries_job_metadata`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Edouard Vanbelle