fix(jobs): a resumed run must not inherit the last attempt's error

From Ed's completed migration, which reported success while still
carrying the reason it had stopped hours earlier:

    "status": "Completed",
    "error_message": "target backend init: Transient Backend:
                      Cannot access bucket 'test-oxicloud': …"

The resume UPDATE flipped `status` to Running and refreshed
`last_progress_at` but left `error_message` alone, so a message
describing why the LAST attempt stopped survived every subsequent
segment and outlived the condition entirely. The run recovered; the row
still said otherwise.

Ed placed it exactly: the same stale-state shape as the read-only banner
that kept showing after its migration was over. State that describes a
past condition has to be cleared by whatever ends that condition, not
left for a later writer to overwrite by luck.

Cleared on resume rather than on completion, because resume is the point
the condition demonstrably no longer holds — and it also fixes the
intermediate reads, where a Running row would otherwise show an error
for work that is actively progressing.

Comment lives in Rust, not in the SQL string: the query text goes over
the wire on every execution and ends up in pg_stat_statements, where
prose is noise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Edouard Vanbelle
2026-09-07 23:58:02 +02:00
parent 57952b2fdc
commit bea9e51128
@@ -874,10 +874,18 @@ impl PgJobStoreProvider {
// In practice this is a rare edge case that // In practice this is a rare edge case that
// ONLY hits if two admin triggers land in // ONLY hits if two admin triggers land in
// the same microsecond. // the same microsecond.
//
// `error_message` is cleared here: it records why
// the LAST attempt stopped, so carrying it past a
// resume leaves a Completed run still displaying a
// transient error it recovered from — a failure
// that did not happen. Same stale-state shape as
// the read-only banner outliving its migration.
let row: Option<(DateTime<Utc>, Option<Vec<u8>>)> = sqlx::query_as( let row: Option<(DateTime<Utc>, Option<Vec<u8>>)> = sqlx::query_as(
r#" r#"
UPDATE jobs.recoverable_runs UPDATE jobs.recoverable_runs
SET status = 'Running', SET status = 'Running',
error_message = NULL,
last_progress_at = NOW() last_progress_at = NOW()
WHERE id = $1 WHERE id = $1
RETURNING started_at, cursor RETURNING started_at, cursor