Edouard Vanbelle
0ef4c624c5
initial test from Ed's nuc:
metric pctl baseline current delta status
-----------------------------------------------------------------------------------------------
folder_cascade.list_depth1 p50 0.3ms 0.3ms -6.3% ok
folder_cascade.list_depth1 p95 2.3ms 0.5ms -75.7% ok
folder_cascade.list_depth1 p99 4.7ms 2.5ms -48.1% ok
folder_cascade.list_depth4 p50 0.4ms 0.3ms -10.0% ok
folder_cascade.list_depth4 p95 0.9ms 0.6ms -31.2% ok
folder_cascade.list_depth4 p99 2.4ms 1.0ms -56.7% ok
folder_cascade.list_depth8 p50 0.3ms 0.3ms -8.8% ok
folder_cascade.list_depth8 p95 0.6ms 0.5ms -22.2% ok
folder_cascade.list_depth8 p99 1.9ms 0.5ms -71.5% ok
folder_cascade.list_depth_deep p50 0.3ms 0.3ms -5.0% ok
folder_cascade.list_depth_deep p95 0.6ms 0.5ms -18.6% ok
folder_cascade.list_depth_deep p99 2.0ms 0.7ms -67.2% ok
share_cascade_rebac.list_grants p50 0.4ms 0.3ms -27.6% ok
share_cascade_rebac.list_grants p95 1.2ms 0.5ms -57.0% ok
share_cascade_rebac.list_grants p99 1.7ms 1.1ms -36.7% ok
share_cascade_rebac.fetch_as_grantee_depth1 p50 0.5ms 0.5ms -11.1% ok
share_cascade_rebac.fetch_as_grantee_depth1 p95 1.1ms 0.7ms -41.2% ok
share_cascade_rebac.fetch_as_grantee_depth1 p99 3.0ms 1.3ms -58.3% ok
share_cascade_rebac.fetch_as_grantee_depth4 p50 0.5ms 0.5ms -13.7% ok
share_cascade_rebac.fetch_as_grantee_depth4 p95 1.4ms 0.7ms -50.5% ok
share_cascade_rebac.fetch_as_grantee_depth4 p99 2.2ms 1.1ms -49.3% ok
share_cascade_rebac.fetch_as_grantee_depth8 p50 0.5ms 0.4ms -16.0% ok
share_cascade_rebac.fetch_as_grantee_depth8 p95 0.9ms 0.7ms -26.1% ok
share_cascade_rebac.fetch_as_grantee_depth8 p99 1.5ms 0.9ms -40.1% ok
share_cascade_rebac.fetch_as_grantee_depth_deep p50 0.5ms 0.4ms -17.3% ok
share_cascade_rebac.fetch_as_grantee_depth_deep p95 1.0ms 0.7ms -31.2% ok
share_cascade_rebac.fetch_as_grantee_depth_deep p99 1.6ms 0.8ms -49.4% ok
subject_group_nested.fetch_as_member_depth1 p50 0.5ms 0.4ms -8.4% ok
subject_group_nested.fetch_as_member_depth1 p95 0.6ms 0.6ms -10.2% ok
subject_group_nested.fetch_as_member_depth1 p99 1.4ms 0.6ms -55.2% ok
subject_group_nested.fetch_as_member_depth4 p50 0.5ms 0.5ms -7.9% ok
subject_group_nested.fetch_as_member_depth4 p95 0.6ms 0.6ms -4.0% ok
subject_group_nested.fetch_as_member_depth4 p99 0.7ms 0.6ms -2.2% ok
subject_group_nested.fetch_as_member_depth8 p50 0.5ms 0.4ms -8.9% ok
subject_group_nested.fetch_as_member_depth8 p95 0.5ms 0.6ms +7.1% ok
subject_group_nested.fetch_as_member_depth8 p99 0.6ms 0.7ms +10.3% ok
subject_group_nested.fetch_as_member_depth_deep p50 0.5ms 0.4ms -7.6% ok
subject_group_nested.fetch_as_member_depth_deep p95 0.6ms 0.5ms -11.9% ok
subject_group_nested.fetch_as_member_depth_deep p99 0.6ms 0.7ms +8.9% ok
tests/load/
K6 load-test suite for OxiCloud. Detects performance regressions by comparing each run's p50/p95/p99 against a committed baseline.
Suites
- smoke —
just load-smoke. Single VU, single iteration of one scenario. Verifies the harness still builds and the server boots. ~1 minute. Run on every PR. No regression gate. - full —
just load. Runs every scenario underscenarios/against a seeded database. Compares results againstbaseline/load.json; exits non-zero on regression beyond the per-metric tolerance. Run nightly onmainand manually.
Scenarios
| File | What it measures |
|---|---|
scenarios/smoke.js |
Login + create folder + upload + list root + delete. Liveness only. |
scenarios/folder_cascade.js |
GET /contents, PUT /move, batch copy, DELETE on a depth-8 fanout-5 tree. |
scenarios/share_cascade_rebac.js |
POST /grants on a folder, then descendants fetched by the grantee. |
scenarios/subject_group_nested.js |
Grant via a 3-level nested group chain, then descendants fetched by a member. |
Add new scenarios as scenarios/<name>.js; register their metric names in
baseline/load.json (or baseline/smoke.json if you wire smoke gating).
Seeding
src/bin/load-seed.rs (invoked by run.sh) bulk-inserts fixtures directly
via sqlx: users, deep folder tree, files (all sharing one dedup'd blob),
nested subject groups, ReBAC grants. Only the resources each scenario
actively touches (the grant being created, the move target, etc.) go
through the REST API at run time — that is the measured hot path.
Baseline & regression detection
Baselines live under baseline/, split by which runner grades them:
| File | Used by | Regression-gated? |
|---|---|---|
baseline/load.json |
just load (run.sh) |
Yes |
baseline/smoke.json |
just load-smoke |
Not yet (see below) |
Both have the same shape — one entry per <scenario>.<op>:
{
"folder_cascade.list_depth1": { "p50": 0.97, "p95": 2.04, "p99": 4.82, "tolerance_pct": 10 }
}
K6 scenarios load the relevant file at startup and set thresholds from
it, so a regression fails the K6 run directly. compare.mjs also prints a
human-readable diff table after the run and exits non-zero if any metric
regresses beyond its tolerance.
The smoke scenario is currently not regression-gated — smoke.sh runs
the scenario but doesn't call compare.mjs. When you decide it should be,
mirror the run.sh pattern and point compare.mjs at
baseline/smoke.json.
Updating a baseline is deliberate. Run just load-baseline to rewrite
baseline/load.json from the latest run, then commit it as
chore(load): accept new baseline for <reason>. Never auto-update. For
smoke.json, pass explicit paths to bake-baseline.mjs.
Local workflow
just db # start the test postgres (port 5433)
just load-seed # seed alone (poking around in psql)
just load-smoke # fast liveness check
just load # full suite + regression diff
just load-baseline # rerun, accept current numbers as the new bar
CI
.github/workflows/load-smoke.yml— every PR. ~1 min. No regression gate..github/workflows/load-nightly.yml— cron daily +workflow_dispatch. Runs the full suite, uploads results as artifact, opens an issue on regression. Currently runs onubuntu-latest; replace with a stable self-hosted runner for trustworthy regression signal (shared GitHub runners produce noisy timings).
Why K6 and not Goose/drill
K6 is Go-based (JS scripting in goja), not Node. For scenario A, the
client adds ~50–200µs per request — negligible vs. multi-ms server work,
and regression deltas only need consistency. K6 also gives us
thresholds-as-DSL, native InfluxDB/Prometheus output, and faster
scenario iteration than a Rust tester would. Reassess when scenario B
(many concurrent users) demonstrates K6 saturation issues.