feat(job-registry): remplace /api/admin/internal/trigger-*
remplace /api/admin/internal/trigger-* to /api/admin/jobs/{...}/trigger
remove OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS
This commit is contained in:
@@ -360,11 +360,11 @@ grace window has zero effect on live access decisions — an expired grant is
|
||||
invisible to `check(...)` even during the grace period. Cleanup only affects
|
||||
storage bloat and the `list_grants_*` history surface.
|
||||
|
||||
The daemon runs inside the same process (`tokio::spawn` at startup, same
|
||||
lifecycle as trash-cleanup / storage-usage sweep), so no external scheduler
|
||||
is needed. An admin-triggered `POST /api/admin/internal/trigger-grant-cleanup`
|
||||
lets operators force a purge in test or incident scenarios; the internal-
|
||||
endpoints gate (`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS`) applies.
|
||||
The daemon runs inside the same process, registered with the periodic-job
|
||||
scheduler (`docs/plan/job-registry.md`) on a 24-hour tick. An admin-
|
||||
triggered `POST /api/admin/jobs/grant_cleanup/trigger?force=true` lets
|
||||
operators force a purge in test or incident scenarios — `force=true`
|
||||
collapses the grace window to zero for that call only.
|
||||
|
||||
The [Share Integration](/architecture/share-integration) doc's reverse
|
||||
trigger takes it from there: when the daemon deletes the last `role_grants`
|
||||
|
||||
@@ -73,7 +73,6 @@ Most runtime variables use the `OXICLOUD_` prefix. A few build-time or allocator
|
||||
| `OXICLOUD_ENABLE_SEARCH` | `true` | Full-text and metadata search |
|
||||
| `OXICLOUD_ENABLE_MUSIC` | `true` | Music playlists and audio metadata |
|
||||
| `OXICLOUD_EXPOSE_SYSTEM_USERS` | `true` | Expose other OxiCloud users as a read-only address book at `GET /api/address-books` |
|
||||
| `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` | `false` | Expose `POST /api/admin/internal/trigger-sweep`, `POST /api/admin/internal/trigger-gc`, and `POST /api/admin/internal/trigger-grant-cleanup` — test-only synchronous triggers for the storage-usage reconciliation sweep, blob garbage collector, and expired-grant purge respectively. Used by the API test suite to assert convergence deterministically without waiting out the periodic tickers. Leave **off** in production: the routes return 404 even to an admin token when disabled. |
|
||||
| `OXICLOUD_GRANT_CLEANUP_ENABLED` | `true` | Background daemon that deletes expired rows from `storage.role_grants`. The authorization engine already filters expired grants out of every permission check at read time (`expires_at IS NULL OR expires_at > NOW()`), so leaving expired rows in place is a hygiene issue — not a security one. This daemon garbage-collects them daily. Set to `false` to keep every expired grant row forever (uncommon; a fresh install rarely wants this). |
|
||||
| `OXICLOUD_GRANT_CLEANUP_GRACE_DAYS` | `15` | Days past a grant's `expires_at` before the row is eligible for deletion. The grace window preserves the audit / support answer to "what happened to my access?" for a couple of weeks past expiration. Values below 1 are legal but discouraged — the recommendation is **≥ 15 days**. Values above the actual grant TTL used by clients waste index space; a few weeks is the sweet spot. |
|
||||
| `OXICLOUD_GRANT_CLEANUP_INTERVAL_HOURS` | `24` | How often the grant-cleanup daemon fires. Clamped to a minimum of 1 hour. Adjusting this doesn't change what gets deleted — only how promptly. Daily is fine for any realistic grant volume. |
|
||||
|
||||
@@ -426,7 +426,8 @@ pub trait CheckStore: Send + Sync {
|
||||
///
|
||||
/// See `BlobConsistencyCheck` for the canonical impl to copy-adapt.
|
||||
pub trait StatefulAdapter: Send + Sync {
|
||||
/// Subsystem slug — appears in `POST /api/admin/internal/consistency/{name}`
|
||||
/// Subsystem slug — appears in the JobRegistry-registered
|
||||
/// `job_name` (`consistency_<subsystem>`, e.g. `consistency_blobs`)
|
||||
/// and in audit log `event` values. Lowercase snake_case, unique
|
||||
/// per adapter. Convention: `"blobs"`, `"thumbnails"`, `"trash"`,
|
||||
/// `"folder_tree"`, `"used_bytes"`.
|
||||
@@ -483,26 +484,36 @@ impl ConsistencyRegistry {
|
||||
|
||||
## Admin surface
|
||||
|
||||
```
|
||||
POST /api/admin/internal/consistency/{name}
|
||||
→ 202 { run_id } (starts a new run)
|
||||
Consistency runs are ordinary `RecoverableJob`s (see
|
||||
`docs/plan/job-registry.md` Part 2), so most operator actions reach
|
||||
them through the shared scheduler surface:
|
||||
|
||||
POST /api/admin/internal/consistency/runs/{id}/cancel
|
||||
→ 200 { status: "CancelRequested" }
|
||||
```
|
||||
GET /api/admin/jobs
|
||||
→ summary list — consistency runs appear as
|
||||
`job_name = "consistency_<name>"`
|
||||
|
||||
POST /api/admin/jobs/consistency_{name}/trigger
|
||||
→ 200 { ok, outcome: { run_id, status } }
|
||||
(starts a new run or resumes the latest Paused one — see
|
||||
Part 2's `run_or_resume`)
|
||||
|
||||
POST /api/admin/jobs/consistency_{name}/cancel
|
||||
→ 200 { run_id, status: "CancelRequested" }
|
||||
(cooperative — check finishes its current batch and returns Paused)
|
||||
|
||||
POST /api/admin/internal/consistency/runs/{id}/resume
|
||||
→ 202 { run_id } (picks up cursor)
|
||||
GET /api/admin/jobs/consistency_{name}/runs?status=<status>
|
||||
→ 200 [{ id, status, scanned_count, last_progress_at, … }]
|
||||
|
||||
GET /api/admin/internal/consistency/runs?check=<name>&status=<status>
|
||||
→ 200 [{ id, check_name, status, scanned_count, last_progress_at, … }]
|
||||
|
||||
GET /api/admin/internal/consistency/runs/{id}
|
||||
GET /api/admin/jobs/consistency_{name}/runs/{id}
|
||||
→ 200 { run: {...}, findings: [...paginated] }
|
||||
```
|
||||
|
||||
Gated by `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` — same admin-guard
|
||||
middleware as `trigger-sweep`, `trigger-gc`, `trigger-grant-cleanup`.
|
||||
Findings enrichment on `runs/{id}` is consistency-specific — read
|
||||
from `admin.consistency_findings` and joined into the response.
|
||||
Everything else is generic Part 2 behaviour.
|
||||
|
||||
Production surface — always on, audit-logged. No feature-flag gate.
|
||||
|
||||
## Approach
|
||||
|
||||
@@ -630,8 +641,9 @@ Ship this PR without the actual checks. Grep `TODO(consistency)` = punch list.
|
||||
- `list_runs(filter)` — SELECT with filters + paginate.
|
||||
- `get_run(id)` — SELECT run + paginated findings.
|
||||
|
||||
Same admin-guard + `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` gate as
|
||||
existing internal endpoints.
|
||||
Same admin-guard as the JobRegistry surface (`trigger_job`,
|
||||
`list_jobs`). Production surface — always on, audit-logged, no
|
||||
feature-flag gate.
|
||||
|
||||
### 6. Boot-time crashed-run recovery
|
||||
|
||||
@@ -726,10 +738,10 @@ count decreases by one per PR.
|
||||
|
||||
## Reused existing utilities
|
||||
|
||||
- **Admin-guard + gate pattern** at
|
||||
`src/interfaces/api/handlers/admin_handler.rs::internal_trigger_gc` —
|
||||
same shape for the new endpoints.
|
||||
- **`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` gate** — same env var.
|
||||
- **Admin-guard + audit-log pattern** at
|
||||
`src/interfaces/api/handlers/admin_handler.rs::trigger_job` —
|
||||
same shape for the new endpoints (production surface, always-on,
|
||||
audit-logged; no feature-flag gate).
|
||||
- **Dedup GC's orphan-detection logic** (`dedup_service.rs`) — the
|
||||
algorithmic template for `BlobConsistencyCheck`'s orphan phase.
|
||||
Reference impl, not a callsite — the check needs its own two-pass
|
||||
@@ -762,8 +774,8 @@ count decreases by one per PR.
|
||||
7. **Grace-window sanity**: run against a fresh 10 s window; upload a
|
||||
file mid-scan; confirm the young blob does NOT surface as
|
||||
`MissingInStorage` (grace window covers it).
|
||||
8. **Env-flag off**: `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=false`
|
||||
→ endpoints return 404, no leakage in the audit channel.
|
||||
8. **AuthZ gate**: non-admin caller hits `POST /api/admin/jobs/consistency_blobs/trigger`
|
||||
→ 403 from the admin middleware, audit line records the rejection.
|
||||
|
||||
## Out of scope
|
||||
|
||||
|
||||
+16
-11
@@ -769,21 +769,26 @@ accepts an optional `?force=<bool>` query param that maps to
|
||||
mutations belong on the audit stream. Success/failure outcome fires
|
||||
its own `oxicloud::scheduler` line via the existing supervisor path.
|
||||
|
||||
**Legacy shim retirement** (Stage 2 — follow-up PR after this one):
|
||||
**Legacy shim retirement** (Stage 2 — landed):
|
||||
|
||||
The three existing internal endpoints map 1:1 to the new surface:
|
||||
The three legacy internal endpoints have been retired in favour of
|
||||
the JobRegistry surface. Kept here for archaeology / URL migration
|
||||
reference for any external tool that still expects the old paths:
|
||||
|
||||
| Legacy | Replacement |
|
||||
| Legacy (retired) | Replacement |
|
||||
|---|---|
|
||||
| `POST /admin/internal/trigger-sweep` | `POST /admin/jobs/storage_reconcile/trigger` |
|
||||
| `POST /admin/internal/trigger-gc?force=X` | `POST /admin/jobs/dedup_gc/trigger?force=X` |
|
||||
| `POST /admin/internal/trigger-grant-cleanup?force=X` | `POST /admin/jobs/grant_cleanup/trigger?force=X` |
|
||||
| `POST /admin/internal/trigger-sweep` | `POST /admin/jobs/storage_reconcile/trigger` |
|
||||
| `POST /admin/internal/trigger-gc?force=X` | `POST /admin/jobs/dedup_gc/trigger?force=X` |
|
||||
| `POST /admin/internal/trigger-grant-cleanup?force=X` | `POST /admin/jobs/grant_cleanup/trigger?force=X` |
|
||||
|
||||
Rewritten as thin forwards to the new endpoints with a `Deprecation:
|
||||
true` response header while Hurl suites migrate to the new paths. Once
|
||||
all callers cut over, the shims are deleted AND the
|
||||
`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` env var is removed — its
|
||||
sole purpose was gating those shims.
|
||||
The `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` env var was removed
|
||||
alongside — its sole purpose was gating those shims.
|
||||
|
||||
Response shape also changed: the old endpoints returned custom fields
|
||||
(`grants_deleted`, `blobs_deleted`, `bytes_freed`, `forced`); the new
|
||||
endpoint returns a uniform `{ ok, outcome: JobOutcome }` envelope with
|
||||
job-specific fields under `outcome.extra`. Any external caller reading
|
||||
the old fields needs updating.
|
||||
|
||||
### Config surface — env vars
|
||||
|
||||
|
||||
-13
@@ -72,19 +72,6 @@ OXICLOUD_SERVER_HOST=127.0.0.1
|
||||
# higher = less background DB work. Minimum enforced: 30s.
|
||||
#OXICLOUD_STORAGE_USAGE_RECONCILE_SECS=600
|
||||
|
||||
# Test-only sweep triggers under /api/admin/internal/*.
|
||||
# When true, exposes:
|
||||
# POST /api/admin/internal/trigger-sweep — run the storage-usage
|
||||
# reconciliation synchronously
|
||||
# POST /api/admin/internal/trigger-gc — run the blob garbage collector
|
||||
# synchronously
|
||||
# Used by the Hurl / integration suites to assert post-delete quota
|
||||
# convergence without waiting out the periodic ticker (default 600 s).
|
||||
# These endpoints short-circuit operator-visible background cadence, so
|
||||
# leave OFF in production — when disabled, the routes return 404 even
|
||||
# to an admin token. Default: false.
|
||||
#OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=false
|
||||
|
||||
# Native WebDAV URL segment that returns the drive listing. Sanitized
|
||||
# by trimming leading/trailing `/` so `/@drive/`, `@drive`, and
|
||||
# `@drive/` are equivalent. Three deployment modes:
|
||||
|
||||
@@ -65,8 +65,8 @@ impl StorageUsageService {
|
||||
/// `GET /api/drives` therefore lags by up to the cache TTL (30 s),
|
||||
/// which matches the sibling caches' accepted UX phantom for
|
||||
/// drive-name staleness. Tests / operators that need immediate
|
||||
/// freshness call `POST /api/admin/internal/trigger-sweep`, which
|
||||
/// runs `update_all_drives_storage_usage` → this method.
|
||||
/// freshness call `POST /api/admin/jobs/storage_reconcile/trigger`,
|
||||
/// which runs `update_all_drives_storage_usage` → this method.
|
||||
///
|
||||
/// Security posture unaffected: `check_drive_quota` reads
|
||||
/// directly from SQL, bypassing the cache entirely, so quota
|
||||
|
||||
@@ -1078,15 +1078,6 @@ pub struct FeaturesConfig {
|
||||
/// trash/search). OFF by default — opt-in per deployment.
|
||||
/// Env: `OXICLOUD_ENABLE_EXTERNAL_MOUNTS`.
|
||||
pub enable_external_mounts: bool,
|
||||
/// Expose `/api/admin/internal/*` test-only endpoints that trigger
|
||||
/// background sweeps on demand (storage-usage reconciliation, blob
|
||||
/// GC). Intended for Hurl / integration tests that need to wait
|
||||
/// for these maintenance jobs deterministically rather than
|
||||
/// polling the cached value. Off by default — these endpoints
|
||||
/// short-circuit the operator-visible cadence, so production
|
||||
/// deployments don't want them reachable. Env:
|
||||
/// `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS`.
|
||||
pub enable_admin_internal_endpoints: bool,
|
||||
/// Native WebDAV path segment that lists the caller's drives.
|
||||
///
|
||||
/// * Default `"@drive"` — bare `/webdav/` addresses the caller's
|
||||
@@ -1163,10 +1154,6 @@ impl Default for FeaturesConfig {
|
||||
expose_system_users: true, // Expose OxiCloud users as address book by default
|
||||
enable_video_thumbnails: true, // Video thumbs via ffmpeg (if detected)
|
||||
enable_external_mounts: false, // External mounts — opt-in, off by default
|
||||
// Test-only sweep triggers — strictly opt-in. Production
|
||||
// deployments do NOT need this; the periodic ticker handles
|
||||
// reconciliation transparently.
|
||||
enable_admin_internal_endpoints: false,
|
||||
// Back-compat with pre-multi-drive clients — bare `/webdav/`
|
||||
// maps to the caller's default drive; drive listing is
|
||||
// reachable at `/webdav/@drive/`.
|
||||
@@ -1867,16 +1854,6 @@ impl AppConfig {
|
||||
config.features.enable_video_thumbnails = val;
|
||||
}
|
||||
|
||||
// `/api/admin/internal/*` test-only triggers. Disabled by
|
||||
// default; production deployments never need this. The Hurl
|
||||
// suite flips it on via `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`.
|
||||
if let Ok(enable_internal) =
|
||||
env::var("OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS").map(|v| v.parse::<bool>())
|
||||
&& let Ok(val) = enable_internal
|
||||
{
|
||||
config.features.enable_admin_internal_endpoints = val;
|
||||
}
|
||||
|
||||
// Grant-cleanup daemon. Purges rows from `storage.role_grants`
|
||||
// whose `expires_at` is more than `grace_days` in the past.
|
||||
// See `GrantCleanupConfig` for defaults + rationale.
|
||||
|
||||
+6
-5
@@ -2309,11 +2309,12 @@ pub struct AppState {
|
||||
pub places_service: Option<Arc<PlacesService>>,
|
||||
pub people_service: Option<Arc<PeopleService>>,
|
||||
pub storage_usage_service: Option<Arc<StorageUsageService>>,
|
||||
/// Handle to the background daemon that purges expired
|
||||
/// `storage.role_grants` rows. `None` when the daemon is disabled
|
||||
/// via `OXICLOUD_GRANT_CLEANUP_ENABLED=false`. The admin
|
||||
/// `POST /api/admin/internal/trigger-grant-cleanup` handler uses
|
||||
/// this to invoke the purge on demand (test-only).
|
||||
/// Handle to the service that purges expired `storage.role_grants`
|
||||
/// rows. Registered with the periodic-job scheduler on the
|
||||
/// configured cadence; `None` when disabled via
|
||||
/// `OXICLOUD_GRANT_CLEANUP_ENABLED=false`. Exposed on `AppState`
|
||||
/// so the admin trigger endpoint can invoke `purge(Some(0))` for
|
||||
/// the `?force=true` grace-override path.
|
||||
pub grant_cleanup_service: Option<
|
||||
Arc<crate::infrastructure::services::grant_cleanup_service::GrantCleanupService>,
|
||||
>,
|
||||
|
||||
@@ -117,11 +117,7 @@ async fn run(registry: Arc<JobRegistry>) {
|
||||
/// `args` is passed through to `JobHandler::run`. The supervisor's
|
||||
/// periodic ticks pass `JobRunArgs::default()`; the admin trigger
|
||||
/// endpoint forwards parsed query params such as `?force=true`.
|
||||
pub(super) async fn dispatch(
|
||||
name: &str,
|
||||
entry: Arc<JobEntry>,
|
||||
args: &JobRunArgs,
|
||||
) -> JobOutcome {
|
||||
pub(super) async fn dispatch(name: &str, entry: Arc<JobEntry>, args: &JobRunArgs) -> JobOutcome {
|
||||
// Try to acquire the single-permit gate. `try_acquire` is
|
||||
// non-blocking — if held, we know the previous run is still
|
||||
// executing and skip this tick.
|
||||
@@ -372,9 +368,10 @@ mod tests {
|
||||
// Kick off dispatch 1 in the background — it holds the permit
|
||||
// for ~200 ms.
|
||||
let entry_bg = entry.clone();
|
||||
let bg = tokio::spawn(async move {
|
||||
dispatch("overrun", entry_bg, &JobRunArgs::default()).await
|
||||
});
|
||||
let bg =
|
||||
tokio::spawn(
|
||||
async move { dispatch("overrun", entry_bg, &JobRunArgs::default()).await },
|
||||
);
|
||||
|
||||
// Give dispatch 1 time to grab the permit.
|
||||
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||
|
||||
@@ -216,11 +216,7 @@ impl JobRegistry {
|
||||
/// `args` is forwarded to `JobHandler::run`. Admin trigger routes
|
||||
/// use `JobRunArgs { force: query.force }`; programmatic callers
|
||||
/// that just want a plain run pass `JobRunArgs::default()`.
|
||||
pub async fn trigger(
|
||||
self: &Arc<Self>,
|
||||
name: &str,
|
||||
args: &JobRunArgs,
|
||||
) -> Option<JobOutcome> {
|
||||
pub async fn trigger(self: &Arc<Self>, name: &str, args: &JobRunArgs) -> Option<JobOutcome> {
|
||||
let entry = self.get(name).await?;
|
||||
Some(super::engine::dispatch(name, entry, args).await)
|
||||
}
|
||||
@@ -367,10 +363,6 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn trigger_returns_none_for_unknown_job() {
|
||||
let reg = Arc::new(JobRegistry::new());
|
||||
assert!(
|
||||
reg.trigger("nope", &JobRunArgs::default())
|
||||
.await
|
||||
.is_none()
|
||||
);
|
||||
assert!(reg.trigger("nope", &JobRunArgs::default()).await.is_none());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2471,15 +2471,15 @@ impl DedupService {
|
||||
.await
|
||||
}
|
||||
|
||||
/// Test-only variant that bypasses the orphan grace window — used by
|
||||
/// `POST /api/admin/internal/trigger-gc?force=true` so the
|
||||
/// Test-only variant that bypasses the orphan grace window — used
|
||||
/// by `POST /api/admin/jobs/dedup_gc/trigger?force=true` (via the
|
||||
/// `JobRunArgs.force` dispatch in `JobHandler::run`) so the
|
||||
/// integration suite can reap just-orphaned blobs synchronously
|
||||
/// (waiting out the production 1 h grace inside a test run is a
|
||||
/// non-starter). Drops the same rows the regular sweep would, just
|
||||
/// without the time floor. Unsafe under concurrent uploads because
|
||||
/// it reopens the TOCTOU window the grace closes — only the
|
||||
/// admin-internal route, itself gated by
|
||||
/// `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS`, may reach here.
|
||||
/// admin-triggered `?force=true` path reaches here.
|
||||
pub async fn garbage_collect_force(&self) -> Result<(u64, u64), DomainError> {
|
||||
self.garbage_collect_with_grace(0).await
|
||||
}
|
||||
@@ -3150,8 +3150,8 @@ impl crate::infrastructure::scheduler::JobHandler for DedupService {
|
||||
/// cleanup already reaped everything.
|
||||
///
|
||||
/// `args.force = true` skips the orphan grace window
|
||||
/// (`garbage_collect_force` — grace_secs = 0), matching the legacy
|
||||
/// `POST /admin/internal/trigger-gc?force=true` semantics. Unsafe
|
||||
/// (`garbage_collect_force` — grace_secs = 0). Same semantic as
|
||||
/// `POST /api/admin/jobs/dedup_gc/trigger?force=true`. Unsafe
|
||||
/// under concurrent uploads: only reachable through the admin
|
||||
/// endpoint and only intentionally used by tests + operator
|
||||
/// diagnostic sessions.
|
||||
|
||||
@@ -118,9 +118,9 @@ impl JobHandler for GrantCleanupService {
|
||||
/// listings can see it without a second lookup.
|
||||
///
|
||||
/// `args.force = true` collapses the grace window to zero for
|
||||
/// this run only — matches the legacy
|
||||
/// `POST /admin/internal/trigger-grant-cleanup?force=true` shape.
|
||||
/// The configured `self.grace_days` is not mutated.
|
||||
/// this run only — same semantic as
|
||||
/// `POST /api/admin/jobs/grant_cleanup/trigger?force=true`. The
|
||||
/// configured `self.grace_days` is not mutated.
|
||||
async fn run(&self, args: &JobRunArgs) -> JobOutcome {
|
||||
let grace_override = if args.force { Some(0) } else { None };
|
||||
let effective_grace = grace_override.unwrap_or(self.grace_days);
|
||||
|
||||
@@ -24,7 +24,6 @@ use crate::application::dtos::settings_dto::{
|
||||
use crate::application::dtos::user_dto::{AdminUserSummaryDto, UserDto};
|
||||
use crate::application::ports::authorization_ports::AuthorizationEngine;
|
||||
use crate::application::ports::plugin_ports::{LogQuery, PluginManagementPort, PluginMgmtError};
|
||||
use crate::application::ports::storage_ports::StorageUsagePort;
|
||||
use crate::common::di::AppState;
|
||||
use crate::domain::repositories::drive_repository::DriveRepository;
|
||||
use crate::domain::services::authorization::{Resource, Subject};
|
||||
@@ -144,21 +143,10 @@ pub fn admin_routes() -> Router<Arc<AppState>> {
|
||||
// when `OXICLOUD_SMTP_MOCK` is off, so production deployments
|
||||
// can route the path freely without leaking inboxes.
|
||||
.route("/smtp/test/captured", get(get_captured_email))
|
||||
// Test-only sweep triggers. Routes are always registered; the
|
||||
// handlers themselves short-circuit to 404 when
|
||||
// `features.enable_admin_internal_endpoints` is off — matches
|
||||
// the `/smtp/test/captured` convention so production
|
||||
// deployments don't need a different route table.
|
||||
.route("/internal/trigger-sweep", post(internal_trigger_sweep))
|
||||
.route("/internal/trigger-gc", post(internal_trigger_gc))
|
||||
.route(
|
||||
"/internal/trigger-grant-cleanup",
|
||||
post(internal_trigger_grant_cleanup),
|
||||
)
|
||||
// JobRegistry admin surface — production, always-on,
|
||||
// audit-logged. See `docs/plan/job-registry.md` §Cross-cutting.
|
||||
// The `/internal/trigger-*` shims above will be retired in a
|
||||
// follow-up PR (deprecated forwards to these endpoints).
|
||||
// Retired the `/internal/trigger-sweep|gc|grant-cleanup` shims
|
||||
// that used to sit here (Stage 2 of the job-registry rollout).
|
||||
.route("/jobs", get(list_jobs))
|
||||
.route("/jobs/{name}/trigger", post(trigger_job))
|
||||
// Drives — admin-wide view (distinct from `/api/drives` which
|
||||
@@ -2065,246 +2053,6 @@ pub async fn delete_drive_admin(
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
// Test-only sweep triggers (`/api/admin/internal/*`)
|
||||
//
|
||||
// Wraps the periodic background jobs (storage-usage reconciliation,
|
||||
// blob garbage collection) behind admin-gated synchronous endpoints
|
||||
// so Hurl / integration tests can wait for them deterministically
|
||||
// rather than polling the cached value. Disabled at the handler edge
|
||||
// when `features.enable_admin_internal_endpoints == false` — match
|
||||
// the `/smtp/test/captured` convention so production deployments
|
||||
// don't need a different route table.
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
/// Refusal when the test-only endpoints are disabled. Returns 404
|
||||
/// rather than 403 to avoid leaking the route's existence (and the
|
||||
/// corresponding config flag) to an unauthenticated probe — the
|
||||
/// legitimate test runner sets the env explicitly.
|
||||
fn internal_endpoints_disabled() -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
(
|
||||
StatusCode::NOT_FOUND,
|
||||
Json(serde_json::json!({ "error": "endpoint not available" })),
|
||||
)
|
||||
.into_response()
|
||||
}
|
||||
|
||||
/// `POST /api/admin/internal/trigger-sweep` — run the storage-usage
|
||||
/// reconciliation sweep synchronously.
|
||||
///
|
||||
/// Test-only. Recomputes `users.storage_used_bytes` and
|
||||
/// `drives.used_bytes` from `SUM(size) WHERE NOT is_trashed`, in the
|
||||
/// same set-based UPDATEs the periodic ticker runs. Used by Hurl
|
||||
/// suites that need to assert post-delete quota convergence without
|
||||
/// waiting out the sweep interval (default 600 s).
|
||||
#[utoipa::path(
|
||||
post,
|
||||
path = "/api/admin/internal/trigger-sweep",
|
||||
responses(
|
||||
(status = 200, description = "Sweep ran"),
|
||||
(status = 401, description = "Unauthorized"),
|
||||
(status = 403, description = "Admin required"),
|
||||
(status = 404, description = "Endpoint disabled (set OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true)"),
|
||||
),
|
||||
security(("bearerAuth" = [])),
|
||||
tag = "admin"
|
||||
)]
|
||||
pub async fn internal_trigger_sweep(
|
||||
State(state): State<Arc<AppState>>,
|
||||
) -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
if !state.core.config.features.enable_admin_internal_endpoints {
|
||||
return internal_endpoints_disabled();
|
||||
}
|
||||
let svc = match state.storage_usage_service.as_ref() {
|
||||
Some(s) => s,
|
||||
None => {
|
||||
return (
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
Json(serde_json::json!({
|
||||
"error": "storage_usage_service not available",
|
||||
})),
|
||||
)
|
||||
.into_response();
|
||||
}
|
||||
};
|
||||
// Order matches the periodic ticker (`start_reconciliation_job`):
|
||||
// drive sweep first because the user sweep reads `drives.used_bytes`
|
||||
// (sum-of-personal-drives — `docs/plan/drive.md` §7). Running them
|
||||
// in the other order makes the user counter freeze on the previous
|
||||
// tick's drive numbers — invisible in steady state but breaks any
|
||||
// Hurl that trashes + sweeps within one call.
|
||||
if let Err(e) = svc.update_all_drives_storage_usage().await {
|
||||
return AppError::internal_error(format!("drive sweep failed: {e}")).into_response();
|
||||
}
|
||||
if let Err(e) = svc.update_all_users_storage_usage().await {
|
||||
return AppError::internal_error(format!("user sweep failed: {e}")).into_response();
|
||||
}
|
||||
(
|
||||
StatusCode::OK,
|
||||
Json(serde_json::json!({ "ok": true, "ran": ["drives", "users"] })),
|
||||
)
|
||||
.into_response()
|
||||
}
|
||||
|
||||
/// Query parameters for `POST /api/admin/internal/trigger-gc`.
|
||||
///
|
||||
/// `force=true` bypasses the orphan-grace window so the sweep reaps
|
||||
/// just-orphaned blobs in the same call. Without this, a blob orphaned
|
||||
/// less than `GC_ORPHAN_GRACE_SECS` (1 h) ago survives the sweep — the
|
||||
/// grace exists so a concurrent uploader pinning a just-orphaned chunk
|
||||
/// can't race the row-delete → file-unlink gap. Integration tests
|
||||
/// don't have concurrent uploaders, so the test runner sets
|
||||
/// `force=true` to make the sweep deterministic within a test's
|
||||
/// runtime.
|
||||
#[derive(Debug, serde::Deserialize, Default)]
|
||||
pub struct InternalTriggerGcQuery {
|
||||
#[serde(default)]
|
||||
pub force: bool,
|
||||
}
|
||||
|
||||
/// `POST /api/admin/internal/trigger-gc` — run the blob garbage
|
||||
/// collector synchronously.
|
||||
///
|
||||
/// Test-only. Drops `file_blobs` rows with `ref_count = 0` (subject
|
||||
/// to the orphan-grace window) and their on-disk content. Same call
|
||||
/// as the inline post-purge GC and the periodic blob-GC sweep — just
|
||||
/// exposed under an admin route so Hurl can wait for it
|
||||
/// deterministically. Add `?force=true` to bypass the grace window —
|
||||
/// see [`InternalTriggerGcQuery`].
|
||||
#[utoipa::path(
|
||||
post,
|
||||
path = "/api/admin/internal/trigger-gc",
|
||||
params(("force" = Option<bool>, Query, description = "Bypass the orphan-grace window (test-only)")),
|
||||
responses(
|
||||
(status = 200, description = "GC ran"),
|
||||
(status = 401, description = "Unauthorized"),
|
||||
(status = 403, description = "Admin required"),
|
||||
(status = 404, description = "Endpoint disabled (set OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true)"),
|
||||
),
|
||||
security(("bearerAuth" = [])),
|
||||
tag = "admin"
|
||||
)]
|
||||
pub async fn internal_trigger_gc(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Query(query): Query<InternalTriggerGcQuery>,
|
||||
) -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
if !state.core.config.features.enable_admin_internal_endpoints {
|
||||
return internal_endpoints_disabled();
|
||||
}
|
||||
let result = if query.force {
|
||||
state.core.dedup_service.garbage_collect_force().await
|
||||
} else {
|
||||
state.core.dedup_service.garbage_collect().await
|
||||
};
|
||||
match result {
|
||||
Ok((blobs_deleted, bytes_freed)) => (
|
||||
StatusCode::OK,
|
||||
Json(serde_json::json!({
|
||||
"ok": true,
|
||||
"blobs_deleted": blobs_deleted,
|
||||
"bytes_freed": bytes_freed,
|
||||
"forced": query.force,
|
||||
})),
|
||||
)
|
||||
.into_response(),
|
||||
Err(e) => AppError::internal_error(format!("gc failed: {e}")).into_response(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Query parameters for `POST /api/admin/internal/trigger-grant-cleanup`.
|
||||
///
|
||||
/// `force=true` sets the grace window to `0` for this call — deletes
|
||||
/// every row whose `expires_at` is in the past, right now. Enables
|
||||
/// Hurl regressions to plant a past-dated grant and immediately
|
||||
/// observe it purged, without waiting the configured
|
||||
/// `OXICLOUD_GRANT_CLEANUP_GRACE_DAYS` out.
|
||||
///
|
||||
/// Without `force`, the daemon's configured grace applies — the same
|
||||
/// SQL the daily loop runs.
|
||||
#[derive(Debug, serde::Deserialize, Default)]
|
||||
pub struct InternalTriggerGrantCleanupQuery {
|
||||
#[serde(default)]
|
||||
pub force: bool,
|
||||
}
|
||||
|
||||
/// `POST /api/admin/internal/trigger-grant-cleanup` — run the expired-
|
||||
/// grant purge synchronously.
|
||||
///
|
||||
/// Test-only. Deletes rows from `storage.role_grants` whose
|
||||
/// `expires_at` is more than `grace_days` in the past (or immediately,
|
||||
/// with `?force=true`). Same SQL as the periodic `GrantCleanupService`
|
||||
/// daemon — exposed under an admin route so Hurl can wait for it
|
||||
/// deterministically.
|
||||
///
|
||||
/// Response fields:
|
||||
/// `grants_deleted` — count of rows removed by this invocation
|
||||
/// `grace_days` — the grace window that was applied (0 when
|
||||
/// `?force=true`, otherwise the config value)
|
||||
/// `forced` — echoes the query param
|
||||
#[utoipa::path(
|
||||
post,
|
||||
path = "/api/admin/internal/trigger-grant-cleanup",
|
||||
params(("force" = Option<bool>, Query, description = "Force grace = 0 for this run (test-only)")),
|
||||
responses(
|
||||
(status = 200, description = "Purge ran"),
|
||||
(status = 401, description = "Unauthorized"),
|
||||
(status = 403, description = "Admin required"),
|
||||
(status = 404, description = "Endpoint disabled (set OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true)"),
|
||||
(status = 503, description = "Grant-cleanup daemon disabled (OXICLOUD_GRANT_CLEANUP_ENABLED=false)"),
|
||||
),
|
||||
security(("bearerAuth" = [])),
|
||||
tag = "admin"
|
||||
)]
|
||||
pub async fn internal_trigger_grant_cleanup(
|
||||
State(state): State<Arc<AppState>>,
|
||||
Query(query): Query<InternalTriggerGrantCleanupQuery>,
|
||||
) -> axum::response::Response {
|
||||
use axum::response::IntoResponse;
|
||||
if !state.core.config.features.enable_admin_internal_endpoints {
|
||||
return internal_endpoints_disabled();
|
||||
}
|
||||
// Daemon may be disabled by config even when the internal-endpoint
|
||||
// gate is on. Return 503 (rather than 404 or 500) so integration
|
||||
// tests can distinguish "surface not exposed" from "surface
|
||||
// exposed but backing service off".
|
||||
let svc = match state.grant_cleanup_service.as_ref() {
|
||||
Some(s) => s,
|
||||
None => {
|
||||
return (
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
Json(serde_json::json!({
|
||||
"error": "grant_cleanup_service not available (disabled by OXICLOUD_GRANT_CLEANUP_ENABLED=false)",
|
||||
})),
|
||||
)
|
||||
.into_response();
|
||||
}
|
||||
};
|
||||
// `force=true` collapses the grace window to zero for this run
|
||||
// only — the daemon's configured grace is untouched. Mirrors the
|
||||
// `trigger-gc?force=true` shape.
|
||||
let grace_override = if query.force { Some(0) } else { None };
|
||||
let grants_deleted = match svc.purge(grace_override).await {
|
||||
Ok(n) => n,
|
||||
Err(e) => {
|
||||
return AppError::internal_error(format!("grant cleanup failed: {e}")).into_response();
|
||||
}
|
||||
};
|
||||
let grace_days = grace_override.unwrap_or_else(|| svc.grace_days());
|
||||
(
|
||||
StatusCode::OK,
|
||||
Json(serde_json::json!({
|
||||
"ok": true,
|
||||
"grants_deleted": grants_deleted,
|
||||
"grace_days": grace_days,
|
||||
"forced": query.force,
|
||||
})),
|
||||
)
|
||||
.into_response()
|
||||
}
|
||||
|
||||
// ─────────────────────────────────────────────────────
|
||||
// JobRegistry admin surface (`/api/admin/jobs/*`)
|
||||
// ─────────────────────────────────────────────────────
|
||||
@@ -2312,9 +2060,8 @@ pub async fn internal_trigger_grant_cleanup(
|
||||
/// `GET /api/admin/jobs` — enumerate every registered job with its
|
||||
/// interval, next-run/last-run timestamps, and last outcome.
|
||||
///
|
||||
/// Production endpoint (no `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS`
|
||||
/// gate). Read-only, so no audit line — the standard admin-middleware
|
||||
/// auth check is enough.
|
||||
/// Production endpoint, always on. Read-only, so no audit line —
|
||||
/// the standard admin-middleware auth check is enough.
|
||||
#[utoipa::path(
|
||||
get,
|
||||
path = "/api/admin/jobs",
|
||||
|
||||
@@ -228,13 +228,12 @@ use crate::interfaces::api::handlers::file_handler::MoveFilePayload;
|
||||
handlers::admin_handler::complete_migration,
|
||||
handlers::admin_handler::verify_migration,
|
||||
handlers::admin_handler::generate_encryption_key,
|
||||
// Admin internal-trigger handlers — gated by
|
||||
// OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS (Off by default in
|
||||
// prod; on for the Hurl suite). Documented in OpenAPI so
|
||||
// integrators writing test harnesses can discover the surface.
|
||||
handlers::admin_handler::internal_trigger_sweep,
|
||||
handlers::admin_handler::internal_trigger_gc,
|
||||
handlers::admin_handler::internal_trigger_grant_cleanup,
|
||||
// JobRegistry admin surface — production, always-on,
|
||||
// audit-logged. Retired the `/internal/trigger-*` handlers in
|
||||
// favour of `/api/admin/jobs/{name}/trigger` uniform surface
|
||||
// (docs/plan/job-registry.md §Cross-cutting).
|
||||
handlers::admin_handler::list_jobs,
|
||||
handlers::admin_handler::trigger_job,
|
||||
// Grant / ReBAC handlers (free functions)
|
||||
handlers::grant_handler::create_grant,
|
||||
handlers::grant_handler::revoke_grant,
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
# same `v_dest_drive_id` variable, so (a) passing implies
|
||||
# file rows used the same value and (b) cross-checks it.
|
||||
#
|
||||
# Sweep convergence: `/api/admin/internal/trigger-sweep` is the
|
||||
# Sweep convergence: `/api/admin/jobs/storage_reconcile/trigger` is the
|
||||
# deterministic synchronisation point — without it the
|
||||
# fire-and-forget delta hook may not yet have updated the cached
|
||||
# `used_bytes` when we read it.
|
||||
@@ -136,7 +136,7 @@ file_id: jsonpath "$.id"
|
||||
# numbers, the late hook adds its delta on top, and used_bytes ends
|
||||
# up high by exactly one file's size. Symptom: expected 32, got 64.
|
||||
# Real fix is await'ing the hook inline server-side.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -171,7 +171,7 @@ HTTP 200
|
||||
[Captures]
|
||||
shared_file_id: jsonpath "$.successful[0].id"
|
||||
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -259,7 +259,7 @@ HTTP 201
|
||||
# the file's size into the cached counter. 200 ms is well above
|
||||
# the tokio task latency on any reasonable box; the deterministic
|
||||
# fix would be intra-transaction hooks, deferred until D7.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -348,7 +348,7 @@ jsonpath "$.name" == "dc-subtree-inner"
|
||||
# the Step 6 file copy (32) = 96. Anything other than (96, 96)
|
||||
# would mean the file INSERT in copy_folder_tree used the wrong
|
||||
# drive_id.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
# a file inside and watching the destination drive's
|
||||
# `used_bytes` jump by the descendant's size (not 0).
|
||||
#
|
||||
# Sweep convergence: `/api/admin/internal/trigger-sweep` is the
|
||||
# Sweep convergence: `/api/admin/jobs/storage_reconcile/trigger` is the
|
||||
# deterministic synchronisation point — it recomputes every
|
||||
# drive's cached `used_bytes` from `SUM(file.size) WHERE
|
||||
# drive_id = d.id`. If the file/folder move didn't update
|
||||
@@ -125,7 +125,7 @@ file_id: jsonpath "$.id"
|
||||
# Baseline used_bytes after the upload settles. Trigger-sweep is
|
||||
# the deterministic sync point — but only after the spawn'd hook
|
||||
# has had a chance to land (bug_trigger_sweep_vs_spawn_hook_race.md).
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -161,7 +161,7 @@ Content-Type: application/json
|
||||
|
||||
HTTP 200
|
||||
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -245,7 +245,7 @@ nested_file_id: jsonpath "$.id"
|
||||
# size. Symptom: expected 64, got 96 (one extra hook landed late).
|
||||
# Real fix is await'ing the hook inline server-side; until then this
|
||||
# delay deflakes the test.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -278,7 +278,7 @@ HTTP 200
|
||||
# shared: nested hello-copy.txt now charged here (32)
|
||||
# Anything other than (32, 32) means the descendant file's
|
||||
# drive_id wasn't cascaded by the trigger.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
|
||||
+13
-15
@@ -120,7 +120,7 @@ small_file_id: jsonpath "$.id"
|
||||
# Ed's 2026-07-17 design call: the sweep is the escape hatch
|
||||
# for tests / operators that need immediate cache freshness;
|
||||
# per-write invalidation would nuke the cache on every upload.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -157,7 +157,7 @@ HTTP 201
|
||||
# `used_bytes` climbs to 64 (32 + 32). Same trigger-sweep pattern
|
||||
# as the first assertion — the delta is fire-and-forget and the
|
||||
# listing cache lags until the sweep invalidates it.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -194,7 +194,7 @@ HTTP 507
|
||||
# consumed by the intervening GET which re-populated the cache
|
||||
# with the pre-refused-write value. Sweep + re-check for
|
||||
# determinism.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
@@ -238,7 +238,7 @@ HTTP 201
|
||||
|
||||
# Unlimited drive's `used_bytes` climbs to the file's exact size
|
||||
# (5 MiB = 5_242_880 bytes). Trigger-sweep pattern (see above).
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -266,9 +266,7 @@ jsonpath "$[?(@.id=='{{unlimited_drive_id}}')].used_bytes" == 5242880
|
||||
# tight drive; the 5 MiB is in the unlimited one).
|
||||
# b) Permanently delete via empty-trash.
|
||||
# c) Trigger the reconciliation sweep on demand —
|
||||
# `/api/admin/internal/trigger-sweep` is gated by
|
||||
# `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`
|
||||
# (set in `tests/common/server.env`).
|
||||
# `POST /api/admin/jobs/storage_reconcile/trigger`.
|
||||
# d) `GET /api/drives` now shows the corrected counter.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
DELETE {{base_url}}/api/files/{{small_file_id}}
|
||||
@@ -286,7 +284,7 @@ HTTP 200
|
||||
|
||||
# Sweep is fire-and-forget on a ticker (default 600 s). Run it now
|
||||
# so the assertion below is deterministic instead of polling.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
@@ -312,7 +310,7 @@ jsonpath "$[?(@.id=='{{unlimited_drive_id}}')].used_bytes" == 5242880
|
||||
# the route in production configs; here we just confirm a
|
||||
# non-admin caller is refused even when the feature is on.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{owner_token}}
|
||||
|
||||
HTTP 403
|
||||
@@ -322,14 +320,14 @@ HTTP 403
|
||||
# summary shape. Hard count is non-deterministic (depends on the
|
||||
# grace window vs the test's elapsed time), so we only check the
|
||||
# response shape.
|
||||
POST {{base_url}}/api/admin/internal/trigger-gc
|
||||
POST {{base_url}}/api/admin/jobs/dedup_gc/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.ok" == true
|
||||
jsonpath "$.blobs_deleted" exists
|
||||
jsonpath "$.bytes_freed" exists
|
||||
jsonpath "$.outcome.count" exists
|
||||
jsonpath "$.outcome.extra.bytes_reclaimed" exists
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
@@ -417,7 +415,7 @@ HTTP 200
|
||||
# operations above never wrote anything. Trigger-sweep so the
|
||||
# check reads live SQL (see the class doc on the earlier
|
||||
# sweep + GET pair for the design rationale).
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
@@ -572,7 +570,7 @@ HTTP 201
|
||||
soft_shrink_file_id: jsonpath "$.id"
|
||||
|
||||
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -656,7 +654,7 @@ Authorization: Bearer {{owner_token}}
|
||||
HTTP 200
|
||||
|
||||
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
|
||||
@@ -12,12 +12,10 @@
|
||||
# assertions here catch it before the daemon runs against real
|
||||
# data.
|
||||
#
|
||||
# Uses the `POST /api/admin/internal/trigger-grant-cleanup`
|
||||
# admin endpoint (gated by
|
||||
# `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`, on for the
|
||||
# api-test suite). `?force=true` collapses the grace window to
|
||||
# zero for the call so we can plant a past-dated grant and
|
||||
# immediately observe it purged, without waiting 15+ days.
|
||||
# Uses the `POST /api/admin/jobs/grant_cleanup/trigger` admin
|
||||
# endpoint (production surface, always on). `?force=true` collapses
|
||||
# the grace window to zero for the call so we can plant a past-dated
|
||||
# grant and immediately observe it purged, without waiting 15+ days.
|
||||
# =============================================================
|
||||
|
||||
|
||||
@@ -150,16 +148,17 @@ permanent_grant_id: jsonpath "$.grants[0].id"
|
||||
# Expect `grants_deleted >= 1` (the past-dated row),
|
||||
# `grace_days == 0`, `forced == true`.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/admin/internal/trigger-grant-cleanup?force=true
|
||||
POST {{base_url}}/api/admin/jobs/grant_cleanup/trigger?force=true
|
||||
Authorization: Bearer {{alice_token}}
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.ok" == true
|
||||
jsonpath "$.forced" == true
|
||||
jsonpath "$.grace_days" == 0
|
||||
jsonpath "$.outcome.outcome" == "ok"
|
||||
jsonpath "$.outcome.extra.forced" == true
|
||||
jsonpath "$.outcome.extra.grace_days" == 0
|
||||
# At least the expired-fixture row we just planted.
|
||||
jsonpath "$.grants_deleted" >= 1
|
||||
jsonpath "$.outcome.count" >= 1
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
@@ -194,28 +193,29 @@ jsonpath "$[?(@.id=='{{permanent_grant_id}}')].role" == "viewer"
|
||||
# false in SQL, so it's already correct; but a
|
||||
# mistyped predicate could regress).
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/admin/internal/trigger-grant-cleanup?force=true
|
||||
POST {{base_url}}/api/admin/jobs/grant_cleanup/trigger?force=true
|
||||
Authorization: Bearer {{alice_token}}
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.grants_deleted" == 0
|
||||
jsonpath "$.outcome.count" == 0
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 9 — Unforced trigger. Grace = configured value (15).
|
||||
# No new expired grants planted, so purge is a no-op.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/admin/internal/trigger-grant-cleanup
|
||||
POST {{base_url}}/api/admin/jobs/grant_cleanup/trigger
|
||||
Authorization: Bearer {{alice_token}}
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.ok" == true
|
||||
jsonpath "$.forced" == false
|
||||
jsonpath "$.outcome.outcome" == "ok"
|
||||
jsonpath "$.outcome.extra.forced" == false
|
||||
# Response echoes the configured grace (15 days by default).
|
||||
jsonpath "$.grace_days" == 15
|
||||
jsonpath "$.grants_deleted" == 0
|
||||
jsonpath "$.outcome.extra.grace_days" == 15
|
||||
jsonpath "$.outcome.count" == 0
|
||||
|
||||
|
||||
# Permanent grant still there after the unforced call.
|
||||
|
||||
@@ -265,34 +265,29 @@ log "API confirms trash is empty."
|
||||
# its `clear_trash_in` path, but that GC honours the 1-hour orphan-grace
|
||||
# window — a blob orphaned seconds ago survives the inline sweep. The
|
||||
# regular periodic sweep would catch it eventually, but tests need the
|
||||
# disk state to be quiescent NOW. The two admin-internal triggers below
|
||||
# (gated by `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`, set in
|
||||
# tests/common/server.env) make this deterministic:
|
||||
# disk state to be quiescent NOW. The two JobRegistry admin triggers
|
||||
# below (production surface, always on) make this deterministic:
|
||||
#
|
||||
# 1. trigger-sweep — reconciles users.storage_used_bytes and
|
||||
# drives.used_bytes from SUM(size) — keeps the
|
||||
# cached counters honest for any quota
|
||||
# assertions that follow.
|
||||
# 2. trigger-gc?force=true — same `garbage_collect()` as the inline
|
||||
# call, but `force=true` bypasses the orphan
|
||||
# grace so freshly-orphaned blobs ARE reaped.
|
||||
# Safe here because the test has no concurrent
|
||||
# uploaders to race the row-delete → unlink
|
||||
# window the grace normally protects.
|
||||
#
|
||||
# Without `force=true`, the test would have to wait an hour for the
|
||||
# probe blob's `orphaned_at` timestamp to age past the grace window —
|
||||
# why this script was disabled until the admin-internal triggers
|
||||
# landed (commit `74b33744`).
|
||||
# 1. storage_reconcile — reconciles users.storage_used_bytes and
|
||||
# drives.used_bytes from SUM(size) — keeps
|
||||
# the cached counters honest for any quota
|
||||
# assertions that follow.
|
||||
# 2. dedup_gc?force=true — same `garbage_collect()` as the inline
|
||||
# call, but `force=true` bypasses the
|
||||
# orphan grace so freshly-orphaned blobs
|
||||
# ARE reaped. Safe here because the test
|
||||
# has no concurrent uploaders to race the
|
||||
# row-delete → unlink window the grace
|
||||
# normally protects.
|
||||
|
||||
curl -sf -X POST -H "$AUTH" "$base_url/api/admin/internal/trigger-sweep" >/dev/null \
|
||||
|| fail "trigger-sweep failed (is OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true?)"
|
||||
curl -sf -X POST -H "$AUTH" "$base_url/api/admin/jobs/storage_reconcile/trigger" >/dev/null \
|
||||
|| fail "storage_reconcile trigger failed"
|
||||
log "Reconciliation sweep triggered."
|
||||
|
||||
GC_RESULT=$(curl -sf -X POST -H "$AUTH" "$base_url/api/admin/internal/trigger-gc?force=true")
|
||||
GC_RESULT=$(curl -sf -X POST -H "$AUTH" "$base_url/api/admin/jobs/dedup_gc/trigger?force=true")
|
||||
[[ -z "$GC_RESULT" ]] && fail "trigger-gc returned an empty body"
|
||||
GC_BLOBS=$(echo "$GC_RESULT" | jq -r '.blobs_deleted')
|
||||
GC_BYTES=$(echo "$GC_RESULT" | jq -r '.bytes_freed')
|
||||
GC_BLOBS=$(echo "$GC_RESULT" | jq -r '.outcome.count')
|
||||
GC_BYTES=$(echo "$GC_RESULT" | jq -r '.outcome.extra.bytes_reclaimed')
|
||||
log "GC reaped $GC_BLOBS blob(s), $GC_BYTES byte(s) freed."
|
||||
|
||||
# ── 4. Disk verification ──────────────────────────────────────────────────────
|
||||
|
||||
@@ -19,13 +19,11 @@
|
||||
# 4. Sweep self-heals — after trashing the personal file and
|
||||
# `trigger-sweep`, `/me.storage_used_bytes` returns to 0.
|
||||
#
|
||||
# `trigger-sweep` is the deterministic synchronisation point:
|
||||
# it runs the drive-side sweep then the user-side sweep
|
||||
# (`StorageUsageService::start_reconciliation_job`), so both
|
||||
# cached counters are authoritative ground-truth by the time
|
||||
# the assertion fires. Gated by
|
||||
# `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`
|
||||
# (set in `tests/common/server.env`).
|
||||
# `POST /api/admin/jobs/storage_reconcile/trigger` is the
|
||||
# deterministic synchronisation point: it runs the drive-side sweep
|
||||
# then the user-side sweep (both under the periodic scheduler), so
|
||||
# both cached counters are authoritative ground-truth by the time
|
||||
# the assertion fires. Production surface — always on, no gate.
|
||||
#
|
||||
# Self-contained: provisions `ue_owner` so it can run alongside
|
||||
# the rest of the suite.
|
||||
@@ -141,7 +139,7 @@ HTTP 201
|
||||
# acts as the synchronisation point for the user-envelope
|
||||
# assertion below — the sweep is the authoritative
|
||||
# ground-truth for both drive- and user-side counters.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
[Options]
|
||||
delay: 200ms
|
||||
@@ -161,7 +159,7 @@ jsonpath "$[?(@.id=='{{shared_drive_id}}')].used_bytes" == 32
|
||||
# If the delta path incorrectly fired the user counter, the sweep
|
||||
# would still correct it back to 0 (the new SQL excludes shared
|
||||
# drives) — this also validates the sweep formula.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
@@ -209,7 +207,7 @@ jsonpath "$.storage_used_bytes" == 32
|
||||
|
||||
# Confirm the sweep agrees with the delta — both code paths must
|
||||
# give the same number.
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
@@ -239,7 +237,7 @@ Authorization: Bearer {{owner_token}}
|
||||
|
||||
HTTP 200
|
||||
|
||||
POST {{base_url}}/api/admin/internal/trigger-sweep
|
||||
POST {{base_url}}/api/admin/jobs/storage_reconcile/trigger
|
||||
Authorization: Bearer {{admin_token}}
|
||||
|
||||
HTTP 200
|
||||
|
||||
@@ -28,12 +28,6 @@ OXICLOUD_OIDC_ENABLED=false
|
||||
|
||||
OXICLOUD_NEXTCLOUD_ENABLED=true
|
||||
|
||||
# Test-only sweep triggers (`/api/admin/internal/trigger-sweep`,
|
||||
# `/api/admin/internal/trigger-gc`). Off by default in production;
|
||||
# the Hurl suite needs them to assert post-delete quota convergence
|
||||
# without waiting out the 600 s reconciliation tick.
|
||||
OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true
|
||||
|
||||
RUST_LOG="warn,audit=info,sqlx::migrate=info"
|
||||
#RUST_LOG="warn,audit=info,oxicloud::quota=debug"
|
||||
#RUST_LOG=debug
|
||||
@@ -79,7 +73,6 @@ OXICLOUD_MAGIC_LINK_SEND_PER_IP_PER_HOUR=50
|
||||
# permits IP spoofing for tests
|
||||
OXICLOUD_TRUST_PROXY_CIDR=0.0.0.0/0
|
||||
|
||||
OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true
|
||||
|
||||
# /webdav/ will points directly to list of drives
|
||||
OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX=""
|
||||
|
||||
@@ -31,7 +31,6 @@ OXICLOUD_ENABLE_MUSIC=true
|
||||
OXICLOUD_EXPOSE_SYSTEM_USERS=true
|
||||
OXICLOUD_WOPI_ENABLED=false
|
||||
OXICLOUD_NEXTCLOUD_ENABLED=true
|
||||
OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true
|
||||
|
||||
RUST_LOG="warn,audit=info,oxicloud::infrastructure::services::oidc_service=info,oxicloud::application::services::auth_application_service=info"
|
||||
|
||||
|
||||
@@ -30,7 +30,6 @@ OXICLOUD_ENABLE_MUSIC=true
|
||||
OXICLOUD_EXPOSE_SYSTEM_USERS=true
|
||||
OXICLOUD_WOPI_ENABLED=false
|
||||
OXICLOUD_NEXTCLOUD_ENABLED=true
|
||||
OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true
|
||||
|
||||
RUST_LOG="warn,audit=info,oxicloud::infrastructure::services::oidc_service=info,oxicloud::application::services::auth_application_service=info"
|
||||
|
||||
|
||||
@@ -31,12 +31,6 @@ OXICLOUD_OIDC_ENABLED=false
|
||||
|
||||
OXICLOUD_NEXTCLOUD_ENABLED=true
|
||||
|
||||
# Test-only sweep triggers (`/api/admin/internal/trigger-sweep`,
|
||||
# `/api/admin/internal/trigger-gc`). Off by default in production;
|
||||
# the Hurl suite needs them to assert post-delete quota convergence
|
||||
# without waiting out the 600 s reconciliation tick.
|
||||
OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true
|
||||
|
||||
RUST_LOG="warn,audit=info,sqlx::migrate=info"
|
||||
#RUST_LOG="warn,audit=info,oxicloud::quota=debug"
|
||||
#RUST_LOG=debug
|
||||
@@ -107,4 +101,3 @@ OXICLOUD_MAGIC_LINK_SEND_PER_IP_PER_HOUR=50
|
||||
# permits IP spoofing for tests
|
||||
OXICLOUD_TRUST_PROXY_CIDR=0.0.0.0/0
|
||||
|
||||
OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true
|
||||
|
||||
Reference in New Issue
Block a user