feat(job-registry): remplace /api/admin/internal/trigger-*
remplace /api/admin/internal/trigger-* to /api/admin/jobs/{...}/trigger
remove OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS
This commit is contained in:
Edouard Vanbelle
@@ -360,11 +360,11 @@ grace window has zero effect on live access decisions — an expired grant is
|
||||
invisible to `check(...)` even during the grace period. Cleanup only affects
|
||||
storage bloat and the `list_grants_*` history surface.
|
||||
|
||||
The daemon runs inside the same process (`tokio::spawn` at startup, same
|
||||
lifecycle as trash-cleanup / storage-usage sweep), so no external scheduler
|
||||
is needed. An admin-triggered `POST /api/admin/internal/trigger-grant-cleanup`
|
||||
lets operators force a purge in test or incident scenarios; the internal-
|
||||
endpoints gate (`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS`) applies.
|
||||
The daemon runs inside the same process, registered with the periodic-job
|
||||
scheduler (`docs/plan/job-registry.md`) on a 24-hour tick. An admin-
|
||||
triggered `POST /api/admin/jobs/grant_cleanup/trigger?force=true` lets
|
||||
operators force a purge in test or incident scenarios — `force=true`
|
||||
collapses the grace window to zero for that call only.
|
||||
|
||||
The [Share Integration](/architecture/share-integration) doc's reverse
|
||||
trigger takes it from there: when the daemon deletes the last `role_grants`
|
||||
|
||||
@@ -73,7 +73,6 @@ Most runtime variables use the `OXICLOUD_` prefix. A few build-time or allocator
|
||||
| `OXICLOUD_ENABLE_SEARCH` | `true` | Full-text and metadata search |
|
||||
| `OXICLOUD_ENABLE_MUSIC` | `true` | Music playlists and audio metadata |
|
||||
| `OXICLOUD_EXPOSE_SYSTEM_USERS` | `true` | Expose other OxiCloud users as a read-only address book at `GET /api/address-books` |
|
||||
| `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` | `false` | Expose `POST /api/admin/internal/trigger-sweep`, `POST /api/admin/internal/trigger-gc`, and `POST /api/admin/internal/trigger-grant-cleanup` — test-only synchronous triggers for the storage-usage reconciliation sweep, blob garbage collector, and expired-grant purge respectively. Used by the API test suite to assert convergence deterministically without waiting out the periodic tickers. Leave **off** in production: the routes return 404 even to an admin token when disabled. |
|
||||
| `OXICLOUD_GRANT_CLEANUP_ENABLED` | `true` | Background daemon that deletes expired rows from `storage.role_grants`. The authorization engine already filters expired grants out of every permission check at read time (`expires_at IS NULL OR expires_at > NOW()`), so leaving expired rows in place is a hygiene issue — not a security one. This daemon garbage-collects them daily. Set to `false` to keep every expired grant row forever (uncommon; a fresh install rarely wants this). |
|
||||
| `OXICLOUD_GRANT_CLEANUP_GRACE_DAYS` | `15` | Days past a grant's `expires_at` before the row is eligible for deletion. The grace window preserves the audit / support answer to "what happened to my access?" for a couple of weeks past expiration. Values below 1 are legal but discouraged — the recommendation is **≥ 15 days**. Values above the actual grant TTL used by clients waste index space; a few weeks is the sweet spot. |
|
||||
| `OXICLOUD_GRANT_CLEANUP_INTERVAL_HOURS` | `24` | How often the grant-cleanup daemon fires. Clamped to a minimum of 1 hour. Adjusting this doesn't change what gets deleted — only how promptly. Daily is fine for any realistic grant volume. |
|
||||
|
||||
@@ -426,7 +426,8 @@ pub trait CheckStore: Send + Sync {
|
||||
///
|
||||
/// See `BlobConsistencyCheck` for the canonical impl to copy-adapt.
|
||||
pub trait StatefulAdapter: Send + Sync {
|
||||
/// Subsystem slug — appears in `POST /api/admin/internal/consistency/{name}`
|
||||
/// Subsystem slug — appears in the JobRegistry-registered
|
||||
/// `job_name` (`consistency_<subsystem>`, e.g. `consistency_blobs`)
|
||||
/// and in audit log `event` values. Lowercase snake_case, unique
|
||||
/// per adapter. Convention: `"blobs"`, `"thumbnails"`, `"trash"`,
|
||||
/// `"folder_tree"`, `"used_bytes"`.
|
||||
@@ -483,26 +484,36 @@ impl ConsistencyRegistry {
|
||||
|
||||
## Admin surface
|
||||
|
||||
```
|
||||
POST /api/admin/internal/consistency/{name}
|
||||
→ 202 { run_id } (starts a new run)
|
||||
Consistency runs are ordinary `RecoverableJob`s (see
|
||||
`docs/plan/job-registry.md` Part 2), so most operator actions reach
|
||||
them through the shared scheduler surface:
|
||||
|
||||
POST /api/admin/internal/consistency/runs/{id}/cancel
|
||||
→ 200 { status: "CancelRequested" }
|
||||
```
|
||||
GET /api/admin/jobs
|
||||
→ summary list — consistency runs appear as
|
||||
`job_name = "consistency_<name>"`
|
||||
|
||||
POST /api/admin/jobs/consistency_{name}/trigger
|
||||
→ 200 { ok, outcome: { run_id, status } }
|
||||
(starts a new run or resumes the latest Paused one — see
|
||||
Part 2's `run_or_resume`)
|
||||
|
||||
POST /api/admin/jobs/consistency_{name}/cancel
|
||||
→ 200 { run_id, status: "CancelRequested" }
|
||||
(cooperative — check finishes its current batch and returns Paused)
|
||||
|
||||
POST /api/admin/internal/consistency/runs/{id}/resume
|
||||
→ 202 { run_id } (picks up cursor)
|
||||
GET /api/admin/jobs/consistency_{name}/runs?status=<status>
|
||||
→ 200 [{ id, status, scanned_count, last_progress_at, … }]
|
||||
|
||||
GET /api/admin/internal/consistency/runs?check=<name>&status=<status>
|
||||
→ 200 [{ id, check_name, status, scanned_count, last_progress_at, … }]
|
||||
|
||||
GET /api/admin/internal/consistency/runs/{id}
|
||||
GET /api/admin/jobs/consistency_{name}/runs/{id}
|
||||
→ 200 { run: {...}, findings: [...paginated] }
|
||||
```
|
||||
|
||||
Gated by `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` — same admin-guard
|
||||
middleware as `trigger-sweep`, `trigger-gc`, `trigger-grant-cleanup`.
|
||||
Findings enrichment on `runs/{id}` is consistency-specific — read
|
||||
from `admin.consistency_findings` and joined into the response.
|
||||
Everything else is generic Part 2 behaviour.
|
||||
|
||||
Production surface — always on, audit-logged. No feature-flag gate.
|
||||
|
||||
## Approach
|
||||
|
||||
@@ -630,8 +641,9 @@ Ship this PR without the actual checks. Grep `TODO(consistency)` = punch list.
|
||||
- `list_runs(filter)` — SELECT with filters + paginate.
|
||||
- `get_run(id)` — SELECT run + paginated findings.
|
||||
|
||||
Same admin-guard + `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` gate as
|
||||
existing internal endpoints.
|
||||
Same admin-guard as the JobRegistry surface (`trigger_job`,
|
||||
`list_jobs`). Production surface — always on, audit-logged, no
|
||||
feature-flag gate.
|
||||
|
||||
### 6. Boot-time crashed-run recovery
|
||||
|
||||
@@ -726,10 +738,10 @@ count decreases by one per PR.
|
||||
|
||||
## Reused existing utilities
|
||||
|
||||
- **Admin-guard + gate pattern** at
|
||||
`src/interfaces/api/handlers/admin_handler.rs::internal_trigger_gc` —
|
||||
same shape for the new endpoints.
|
||||
- **`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` gate** — same env var.
|
||||
- **Admin-guard + audit-log pattern** at
|
||||
`src/interfaces/api/handlers/admin_handler.rs::trigger_job` —
|
||||
same shape for the new endpoints (production surface, always-on,
|
||||
audit-logged; no feature-flag gate).
|
||||
- **Dedup GC's orphan-detection logic** (`dedup_service.rs`) — the
|
||||
algorithmic template for `BlobConsistencyCheck`'s orphan phase.
|
||||
Reference impl, not a callsite — the check needs its own two-pass
|
||||
@@ -762,8 +774,8 @@ count decreases by one per PR.
|
||||
7. **Grace-window sanity**: run against a fresh 10 s window; upload a
|
||||
file mid-scan; confirm the young blob does NOT surface as
|
||||
`MissingInStorage` (grace window covers it).
|
||||
8. **Env-flag off**: `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=false`
|
||||
→ endpoints return 404, no leakage in the audit channel.
|
||||
8. **AuthZ gate**: non-admin caller hits `POST /api/admin/jobs/consistency_blobs/trigger`
|
||||
→ 403 from the admin middleware, audit line records the rejection.
|
||||
|
||||
## Out of scope
|
||||
|
||||
|
||||
+16
-11
@@ -769,21 +769,26 @@ accepts an optional `?force=<bool>` query param that maps to
|
||||
mutations belong on the audit stream. Success/failure outcome fires
|
||||
its own `oxicloud::scheduler` line via the existing supervisor path.
|
||||
|
||||
**Legacy shim retirement** (Stage 2 — follow-up PR after this one):
|
||||
**Legacy shim retirement** (Stage 2 — landed):
|
||||
|
||||
The three existing internal endpoints map 1:1 to the new surface:
|
||||
The three legacy internal endpoints have been retired in favour of
|
||||
the JobRegistry surface. Kept here for archaeology / URL migration
|
||||
reference for any external tool that still expects the old paths:
|
||||
|
||||
| Legacy | Replacement |
|
||||
| Legacy (retired) | Replacement |
|
||||
|---|---|
|
||||
| `POST /admin/internal/trigger-sweep` | `POST /admin/jobs/storage_reconcile/trigger` |
|
||||
| `POST /admin/internal/trigger-gc?force=X` | `POST /admin/jobs/dedup_gc/trigger?force=X` |
|
||||
| `POST /admin/internal/trigger-grant-cleanup?force=X` | `POST /admin/jobs/grant_cleanup/trigger?force=X` |
|
||||
| `POST /admin/internal/trigger-sweep` | `POST /admin/jobs/storage_reconcile/trigger` |
|
||||
| `POST /admin/internal/trigger-gc?force=X` | `POST /admin/jobs/dedup_gc/trigger?force=X` |
|
||||
| `POST /admin/internal/trigger-grant-cleanup?force=X` | `POST /admin/jobs/grant_cleanup/trigger?force=X` |
|
||||
|
||||
Rewritten as thin forwards to the new endpoints with a `Deprecation:
|
||||
true` response header while Hurl suites migrate to the new paths. Once
|
||||
all callers cut over, the shims are deleted AND the
|
||||
`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` env var is removed — its
|
||||
sole purpose was gating those shims.
|
||||
The `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` env var was removed
|
||||
alongside — its sole purpose was gating those shims.
|
||||
|
||||
Response shape also changed: the old endpoints returned custom fields
|
||||
(`grants_deleted`, `blobs_deleted`, `bytes_freed`, `forced`); the new
|
||||
endpoint returns a uniform `{ ok, outcome: JobOutcome }` envelope with
|
||||
job-specific fields under `outcome.extra`. Any external caller reading
|
||||
the old fields needs updating.
|
||||
|
||||
### Config surface — env vars
|
||||
|
||||
|
||||
Reference in New Issue
Block a user