A migration nobody triggers never finishes. Scheduled ticks deliberately
never pass `repair`, so a deployment whose operator never opens the
admin panel re-imported the same sidecars forever and never drained the
directory — and relying on operators to edit `.env` has the same failure
mode one level up.
`OXICLOUD_STARTUP_JOBS` dispatches named jobs once, in the background,
after the scheduler is ready. Entries use the syntax operators already
type at the trigger URL (`name?repair=true`), so the value is literally
the request they would otherwise make by hand. It defaults to both
migration jobs in repair mode, so an untouched deployment migrates and
drains itself.
That is a destructive default and a real exception to
no-silent-auto-repair, so the guard it rests on had to get stronger:
`verify_and_unlink` now compares CONTENT, not length. A blob of the
right size and the wrong bytes used to pass — a key-mapping bug handing
back another file's preview at the same length would have deleted the
original and kept the impostor, and thumbnails cluster tightly enough in
size for that to be a real coincidence. The readback streams from the
backend with no cache in front, so it proves durability rather than that
a write was acknowledged.
Deletion of `.thumbnails/` is attempted first and only falls back to
renaming it `.thumbnails.migrated` when `remove_dir` refuses because a
non-sidecar file is inside (Finder's `.DS_Store`). Either way the
directory stops existing, which lets the read-path probe go back to a
single `stat` on the root instead of walking the size directories.
Validation is fail-fast: an unknown job name or flag panics at boot. A
silently dropped `?repare=true` would leave the job in discovery-only
mode while the operator believed the tier was draining, surfacing months
later as "the migration never finished" with nothing pointing at the
config line.
Interrupted runs resume. Boot recovery flips abandoned rows to Paused
with their cursor, so `run_or_resume` continues rather than rescanning —
a long migration completes across however many restarts it takes. That
is a scoped exception to "we do not auto-resume": here somebody did ask,
in configuration, and not having to ask again is the point.
`StartupJob` holds a `JobRunArgs` rather than re-listing its four
fields, so a fifth flag cannot be added to the scheduler and silently
ignored in configuration.
Jobs named here are ordinary registered jobs — visible in the panel,
triggerable by hand, same runs and findings. Their rows now carry a
`startup` object so an operator can see that a job deletes on every boot
rather than only when someone clicks Run.
Adds docs/config/thumbnail-migration.md: what runs on first boot, how to
snapshot database and storage together beforehand, and how to verify
afterwards with satellites_consistency plus backend_consistency
?deep=true.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The admin panel had no repair toggle wired to anything but a hardcoded
name list naming the two refcount tenants, so `thumb_derived_import` and
`thumb_attached_import` could not be run in repair mode from the UI at
all despite supporting it. And nothing in the job list said what any
given job does or whether clicking Run on production writes anything.
Three defaulted methods on `JobHandler` and `RecoverableJobHandler`:
fn description(&self) -> &'static str
fn mutates(&self) -> Mutates // Never | Always | OnRepairOnly
fn repair_description(&self) -> Option<&'static str>
`RecoverableAdapter` forwards them — the registry only holds
`dyn JobHandler`, so a tenant's metadata is invisible otherwise, and
falling back to the defaults would report every recoverable job as
read-only, including the ones that delete files.
Three values rather than a boolean because a job can be read-only by
default and destructive under `?repair=true`; a boolean answers wrongly
for one of its two modes, and `false` on something that unlinks files is
the dangerous direction to be wrong in. `repair_description` returning
`Option` collapses "does it repair" and "what does repair do" into one
method: presence gates the toggle, content is the confirmation text —
which the frontend cannot invent, since correcting a counter and
deleting sidecars are not the same warning.
`OnRepairOnly` with no `repair_description` is rejected at registration:
it claims to mutate only under a flag it does not support.
All 17 registered jobs declare all three. The panel now renders the
description under each name, badges read-only jobs, confirms before a
plain run of a mutating one, and offers the repair variant off the
backend flag instead of the name list.
Descriptions are English in the trait, next to the behaviour: one in
`locales/*.json` rots invisibly the moment a job changes, and a
translator cannot know what `manifests_consistency` reconciles. i18n can
layer on later keyed by job name with these as the fallback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
```
oxi.UPLOAD_BATCH_BYTES // get current value
oxi.UPLOAD_BATCH_BYTES=1024*1024 // change values
```
values are stored in localstorage, default is 8*1024*1024
add /api/folders/{id}/ancestors
this API to iterate parent up to the drive root or the shared folder
this will help UI to build the breadcrumb in 1 API call
and to identify the root element (is it a drive users has access to or
a shared folder ?)
ui: now only 1 API call is now required to build the breadcrumb
restore the legacy display with new element uploaded, when swimlane is in place
as the sort is done by server side just add new element in a "new elements" swimlane.
if user continue to scroll down in cursor/pages and item is found from server,
UI remove it from the new element and restore the position
other option: is user refresh it's page, server will restore the natural order
Round 2 of benchmark-gated optimizations (benches/ROUND2.md; every change
gated by a before/after in examples/bench_round2.rs — an AFTER that did
not beat its BEFORE was to be rolled back; none needed it):
- Range requests (REST/DAV/shares) answered from the moka content cache
for sub-10MB files: PG resolve + open/seek/read -> Bytes::slice.
256KiB seeks: 1,730/s -> 3.7M/s (p50 552us -> 0.15us).
- Streaming folder/share ZIPs via tokio duplex: TTFB no longer scales
with archive size (326ms -> 0.4ms on 192MiB corpus; total also faster).
Content-Length dropped (size unknown up front).
- NC chunked-upload per-PUT gate: O(k) directory scan+stat -> in-RAM
per-session counter (lazy rebuild on cold start). 1,000-chunk upload
gate cost: 33.1s -> 0.09s cumulative.
- Delta download + commit-verify now use the CDC path's
buffered(read_prefetch) read-ahead: 64-chunk drain at 5ms open
latency 440ms -> 51ms; order preserved.
- CDC ingest settles batches on a spawned task (depth-1 pipeline) so
the source stream keeps flowing during PG pin + backend writes;
rollback ledger shared + lock-serialized so compensation stays exact
on cancellation. 512MiB paced ingest: 60-69 -> 74-75 MB/s.
OXICLOUD_INGEST_OVERLAP=0 restores inline settling (ops/bench hatch).
- Frontend: instant-upload BLAKE3 hashing moved off the main thread to
a bounded Web Worker pool (File handles by reference); vitest gate
asserts the pool beats sequential (first gate draft posting buffers
was 2.6x slower and was rewritten — copies dominated).
Validation: cargo fmt + clippy -D warnings clean; 514 unit + 544
integration tests green; 270 frontend tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
Xalares