chore(load): start implementation of load tests
initial test from Ed's nuc:
metric pctl baseline current delta status
-----------------------------------------------------------------------------------------------
folder_cascade.list_depth1 p50 0.3ms 0.3ms -6.3% ok
folder_cascade.list_depth1 p95 2.3ms 0.5ms -75.7% ok
folder_cascade.list_depth1 p99 4.7ms 2.5ms -48.1% ok
folder_cascade.list_depth4 p50 0.4ms 0.3ms -10.0% ok
folder_cascade.list_depth4 p95 0.9ms 0.6ms -31.2% ok
folder_cascade.list_depth4 p99 2.4ms 1.0ms -56.7% ok
folder_cascade.list_depth8 p50 0.3ms 0.3ms -8.8% ok
folder_cascade.list_depth8 p95 0.6ms 0.5ms -22.2% ok
folder_cascade.list_depth8 p99 1.9ms 0.5ms -71.5% ok
folder_cascade.list_depth_deep p50 0.3ms 0.3ms -5.0% ok
folder_cascade.list_depth_deep p95 0.6ms 0.5ms -18.6% ok
folder_cascade.list_depth_deep p99 2.0ms 0.7ms -67.2% ok
share_cascade_rebac.list_grants p50 0.4ms 0.3ms -27.6% ok
share_cascade_rebac.list_grants p95 1.2ms 0.5ms -57.0% ok
share_cascade_rebac.list_grants p99 1.7ms 1.1ms -36.7% ok
share_cascade_rebac.fetch_as_grantee_depth1 p50 0.5ms 0.5ms -11.1% ok
share_cascade_rebac.fetch_as_grantee_depth1 p95 1.1ms 0.7ms -41.2% ok
share_cascade_rebac.fetch_as_grantee_depth1 p99 3.0ms 1.3ms -58.3% ok
share_cascade_rebac.fetch_as_grantee_depth4 p50 0.5ms 0.5ms -13.7% ok
share_cascade_rebac.fetch_as_grantee_depth4 p95 1.4ms 0.7ms -50.5% ok
share_cascade_rebac.fetch_as_grantee_depth4 p99 2.2ms 1.1ms -49.3% ok
share_cascade_rebac.fetch_as_grantee_depth8 p50 0.5ms 0.4ms -16.0% ok
share_cascade_rebac.fetch_as_grantee_depth8 p95 0.9ms 0.7ms -26.1% ok
share_cascade_rebac.fetch_as_grantee_depth8 p99 1.5ms 0.9ms -40.1% ok
share_cascade_rebac.fetch_as_grantee_depth_deep p50 0.5ms 0.4ms -17.3% ok
share_cascade_rebac.fetch_as_grantee_depth_deep p95 1.0ms 0.7ms -31.2% ok
share_cascade_rebac.fetch_as_grantee_depth_deep p99 1.6ms 0.8ms -49.4% ok
subject_group_nested.fetch_as_member_depth1 p50 0.5ms 0.4ms -8.4% ok
subject_group_nested.fetch_as_member_depth1 p95 0.6ms 0.6ms -10.2% ok
subject_group_nested.fetch_as_member_depth1 p99 1.4ms 0.6ms -55.2% ok
subject_group_nested.fetch_as_member_depth4 p50 0.5ms 0.5ms -7.9% ok
subject_group_nested.fetch_as_member_depth4 p95 0.6ms 0.6ms -4.0% ok
subject_group_nested.fetch_as_member_depth4 p99 0.7ms 0.6ms -2.2% ok
subject_group_nested.fetch_as_member_depth8 p50 0.5ms 0.4ms -8.9% ok
subject_group_nested.fetch_as_member_depth8 p95 0.5ms 0.6ms +7.1% ok
subject_group_nested.fetch_as_member_depth8 p99 0.6ms 0.7ms +10.3% ok
subject_group_nested.fetch_as_member_depth_deep p50 0.5ms 0.4ms -7.6% ok
subject_group_nested.fetch_as_member_depth_deep p95 0.6ms 0.5ms -11.9% ok
subject_group_nested.fetch_as_member_depth_deep p99 0.6ms 0.7ms +8.9% ok
This commit is contained in:
+91
-8
@@ -1,11 +1,94 @@
|
||||
iPurpose of this directory: implement load test and identify response time under:
|
||||
- heavy load
|
||||
- many content
|
||||
- many sub folders and sharing
|
||||
# tests/load/
|
||||
|
||||
Goal is to identify inflections and regression when a new feature is added
|
||||
K6 load-test suite for OxiCloud. Detects performance regressions by comparing
|
||||
each run's p50/p95/p99 against a committed baseline.
|
||||
|
||||
No accemtance criteria yet
|
||||
## Suites
|
||||
|
||||
Load test via k6 ?
|
||||
or drill (written in Rust) ?
|
||||
- **smoke** — `just load-smoke`. Single VU, single iteration of one scenario.
|
||||
Verifies the harness still builds and the server boots. ~1 minute. Run on
|
||||
every PR. No regression gate.
|
||||
- **full** — `just load`. Runs every scenario under `scenarios/` against a
|
||||
seeded database. Compares results against `baseline/load.json`; exits
|
||||
non-zero on regression beyond the per-metric tolerance. Run nightly on
|
||||
`main` and manually.
|
||||
|
||||
## Scenarios
|
||||
|
||||
| File | What it measures |
|
||||
| --------------------------------- | ------------------------------------------------------------------------------- |
|
||||
| `scenarios/smoke.js` | Login + create folder + upload + list root + delete. Liveness only. |
|
||||
| `scenarios/folder_cascade.js` | `GET /contents`, `PUT /move`, batch copy, `DELETE` on a depth-8 fanout-5 tree. |
|
||||
| `scenarios/share_cascade_rebac.js`| `POST /grants` on a folder, then descendants fetched by the grantee. |
|
||||
| `scenarios/subject_group_nested.js`| Grant via a 3-level nested group chain, then descendants fetched by a member. |
|
||||
|
||||
Add new scenarios as `scenarios/<name>.js`; register their metric names in
|
||||
`baseline/load.json` (or `baseline/smoke.json` if you wire smoke gating).
|
||||
|
||||
## Seeding
|
||||
|
||||
`src/bin/load-seed.rs` (invoked by `run.sh`) bulk-inserts fixtures directly
|
||||
via sqlx: users, deep folder tree, files (all sharing one dedup'd blob),
|
||||
nested subject groups, ReBAC grants. Only the resources each scenario
|
||||
actively touches (the grant being created, the move target, etc.) go
|
||||
through the REST API at run time — that is the measured hot path.
|
||||
|
||||
## Baseline & regression detection
|
||||
|
||||
Baselines live under `baseline/`, split by which runner grades them:
|
||||
|
||||
| File | Used by | Regression-gated? |
|
||||
| --------------------- | ------------------------ | ----------------- |
|
||||
| `baseline/load.json` | `just load` (`run.sh`) | Yes |
|
||||
| `baseline/smoke.json` | `just load-smoke` | Not yet (see below) |
|
||||
|
||||
Both have the same shape — one entry per `<scenario>.<op>`:
|
||||
|
||||
```json
|
||||
{
|
||||
"folder_cascade.list_depth1": { "p50": 0.97, "p95": 2.04, "p99": 4.82, "tolerance_pct": 10 }
|
||||
}
|
||||
```
|
||||
|
||||
K6 scenarios load the relevant file at startup and set `thresholds` from
|
||||
it, so a regression fails the K6 run directly. `compare.mjs` also prints a
|
||||
human-readable diff table after the run and exits non-zero if any metric
|
||||
regresses beyond its tolerance.
|
||||
|
||||
The smoke scenario is currently **not regression-gated** — `smoke.sh` runs
|
||||
the scenario but doesn't call `compare.mjs`. When you decide it should be,
|
||||
mirror the `run.sh` pattern and point `compare.mjs` at
|
||||
`baseline/smoke.json`.
|
||||
|
||||
**Updating a baseline is deliberate.** Run `just load-baseline` to rewrite
|
||||
`baseline/load.json` from the latest run, then commit it as
|
||||
`chore(load): accept new baseline for <reason>`. Never auto-update. For
|
||||
`smoke.json`, pass explicit paths to `bake-baseline.mjs`.
|
||||
|
||||
## Local workflow
|
||||
|
||||
```bash
|
||||
just db # start the test postgres (port 5433)
|
||||
just load-seed # seed alone (poking around in psql)
|
||||
just load-smoke # fast liveness check
|
||||
just load # full suite + regression diff
|
||||
just load-baseline # rerun, accept current numbers as the new bar
|
||||
```
|
||||
|
||||
## CI
|
||||
|
||||
- `.github/workflows/load-smoke.yml` — every PR. ~1 min. No regression gate.
|
||||
- `.github/workflows/load-nightly.yml` — cron daily + `workflow_dispatch`.
|
||||
Runs the full suite, uploads results as artifact, opens an issue on
|
||||
regression. Currently runs on `ubuntu-latest`; replace with a stable
|
||||
self-hosted runner for trustworthy regression signal (shared GitHub
|
||||
runners produce noisy timings).
|
||||
|
||||
## Why K6 and not Goose/drill
|
||||
|
||||
K6 is Go-based (JS scripting in `goja`), not Node. For scenario A, the
|
||||
client adds ~50–200µs per request — negligible vs. multi-ms server work,
|
||||
and regression deltas only need *consistency*. K6 also gives us
|
||||
thresholds-as-DSL, native InfluxDB/Prometheus output, and faster
|
||||
scenario iteration than a Rust tester would. Reassess when scenario B
|
||||
(many concurrent users) demonstrates K6 saturation issues.
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env node
|
||||
// bake-baseline.mjs — convert the latest k6 --summary-export into the
|
||||
// baseline shape that compare.mjs reads.
|
||||
//
|
||||
// Tolerance values from the existing baseline are preserved; only the p50/
|
||||
// p95/p99 numbers are overwritten. Run this after `just load` to lock in a
|
||||
// new accepted bar, then commit the result deliberately.
|
||||
//
|
||||
// Usage:
|
||||
// node bake-baseline.mjs # auto-pick latest summary, target baseline/load.json
|
||||
// node bake-baseline.mjs <summary> <baseline> # explicit paths (e.g. baseline/smoke.json)
|
||||
|
||||
import { readFileSync, writeFileSync, readdirSync, statSync } from 'node:fs';
|
||||
import { join, dirname, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { argv, exit } from 'node:process';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
|
||||
function latestSummary() {
|
||||
const dir = join(HERE, 'results');
|
||||
const candidates = readdirSync(dir)
|
||||
.filter((f) => f.startsWith('run-') && f.endsWith('.json'))
|
||||
.map((f) => ({ f, mtime: statSync(join(dir, f)).mtimeMs }))
|
||||
.sort((a, b) => b.mtime - a.mtime);
|
||||
if (candidates.length === 0) {
|
||||
console.error('bake-baseline: no run-*.json under tests/load/results/');
|
||||
exit(2);
|
||||
}
|
||||
return join(dir, candidates[0].f);
|
||||
}
|
||||
|
||||
const summaryPath = argv[2] ? resolve(argv[2]) : latestSummary();
|
||||
const baselinePath = argv[3] ? resolve(argv[3]) : join(HERE, 'baseline/load.json');
|
||||
|
||||
const summary = JSON.parse(readFileSync(summaryPath, 'utf8'));
|
||||
const baseline = JSON.parse(readFileSync(baselinePath, 'utf8'));
|
||||
|
||||
const metricKeyFor = (op) => `http_req_duration{op:${op}}`;
|
||||
const PERCENTILES = [
|
||||
{ baseline: 'p50', k6: 'med' },
|
||||
{ baseline: 'p95', k6: 'p(95)' },
|
||||
{ baseline: 'p99', k6: 'p(99)' },
|
||||
];
|
||||
|
||||
let updated = 0;
|
||||
let missing = 0;
|
||||
|
||||
for (const [op, entry] of Object.entries(baseline)) {
|
||||
if (op.startsWith('_')) continue;
|
||||
const metric = summary.metrics?.[metricKeyFor(op)];
|
||||
if (!metric) {
|
||||
console.warn(` missing: ${op} (no data in ${summaryPath})`);
|
||||
missing++;
|
||||
continue;
|
||||
}
|
||||
// k6 --summary-export puts percentile values directly on the metric object
|
||||
// (alongside `thresholds`), not inside a `.values` wrapper.
|
||||
let touched = false;
|
||||
for (const p of PERCENTILES) {
|
||||
const cur = metric[p.k6];
|
||||
if (cur === undefined) continue;
|
||||
entry[p.baseline] = Number(cur.toFixed(2));
|
||||
touched = true;
|
||||
}
|
||||
if (touched) updated++;
|
||||
}
|
||||
|
||||
writeFileSync(baselinePath, `${JSON.stringify(baseline, null, 2)}\n`);
|
||||
console.log(`bake-baseline: updated ${updated} metric(s), missing ${missing} in ${baselinePath}`);
|
||||
console.log(`Source: ${summaryPath}`);
|
||||
console.log('Commit deliberately: chore(load): accept new baseline for <reason>');
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"_comment": "Baseline for `just load` — the long scenarios. Each entry: p50/p95/p99 (ms) + tolerance_pct (regression threshold). Updated by `just load-baseline`. Keys must match the `op` tag the matching scenario emits — adding a scenario means adding entries here in the same PR. Smoke metrics live in smoke.json.",
|
||||
"folder_cascade.list_depth1": {
|
||||
"p50": 0.35,
|
||||
"p95": 2.25,
|
||||
"p99": 4.73,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"folder_cascade.list_depth4": {
|
||||
"p50": 0.36,
|
||||
"p95": 0.92,
|
||||
"p99": 2.39,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"folder_cascade.list_depth8": {
|
||||
"p50": 0.34,
|
||||
"p95": 0.63,
|
||||
"p99": 1.9,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"folder_cascade.list_depth_deep": {
|
||||
"p50": 0.3,
|
||||
"p95": 0.61,
|
||||
"p99": 2.03,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"share_cascade_rebac.list_grants": {
|
||||
"p50": 0.4,
|
||||
"p95": 1.16,
|
||||
"p99": 1.68,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"share_cascade_rebac.fetch_as_grantee_depth1": {
|
||||
"p50": 0.51,
|
||||
"p95": 1.14,
|
||||
"p99": 3.02,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"share_cascade_rebac.fetch_as_grantee_depth4": {
|
||||
"p50": 0.53,
|
||||
"p95": 1.43,
|
||||
"p99": 2.22,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"share_cascade_rebac.fetch_as_grantee_depth8": {
|
||||
"p50": 0.52,
|
||||
"p95": 0.89,
|
||||
"p99": 1.48,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"share_cascade_rebac.fetch_as_grantee_depth_deep": {
|
||||
"p50": 0.51,
|
||||
"p95": 1.04,
|
||||
"p99": 1.64,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"subject_group_nested.fetch_as_member_depth1": {
|
||||
"p50": 0.48,
|
||||
"p95": 0.63,
|
||||
"p99": 1.41,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"subject_group_nested.fetch_as_member_depth4": {
|
||||
"p50": 0.5,
|
||||
"p95": 0.58,
|
||||
"p99": 0.65,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"subject_group_nested.fetch_as_member_depth8": {
|
||||
"p50": 0.48,
|
||||
"p95": 0.53,
|
||||
"p99": 0.62,
|
||||
"tolerance_pct": 10
|
||||
},
|
||||
"subject_group_nested.fetch_as_member_depth_deep": {
|
||||
"p50": 0.45,
|
||||
"p95": 0.62,
|
||||
"p99": 0.63,
|
||||
"tolerance_pct": 10
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"_comment": "Baseline for `just load-smoke` — the smoke scenario. Placeholder values; smoke is not currently regression-gated by smoke.sh. When you decide it should be, point compare.mjs at this file from smoke.sh (mirror the run.sh pattern) and re-bake.",
|
||||
"smoke.login": {
|
||||
"p50": 50,
|
||||
"p95": 200,
|
||||
"p99": 400,
|
||||
"tolerance_pct": 15
|
||||
},
|
||||
"smoke.create_folder": {
|
||||
"p50": 20,
|
||||
"p95": 80,
|
||||
"p99": 150,
|
||||
"tolerance_pct": 15
|
||||
},
|
||||
"smoke.upload_tiny": {
|
||||
"p50": 30,
|
||||
"p95": 120,
|
||||
"p99": 250,
|
||||
"tolerance_pct": 15
|
||||
},
|
||||
"smoke.list_root": {
|
||||
"p50": 15,
|
||||
"p95": 60,
|
||||
"p99": 120,
|
||||
"tolerance_pct": 15
|
||||
},
|
||||
"smoke.delete_folder": {
|
||||
"p50": 20,
|
||||
"p95": 80,
|
||||
"p99": 150,
|
||||
"tolerance_pct": 15
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
#!/usr/bin/env node
|
||||
// compare.mjs — regression diff for k6 load-test runs.
|
||||
//
|
||||
// Reads a k6 --summary-export JSON and the committed baseline, diffs the
|
||||
// p50/p95/p99 of every baseline metric against the current run, prints a
|
||||
// human-readable table, and exits non-zero if any metric regresses beyond
|
||||
// its tolerance.
|
||||
//
|
||||
// Usage:
|
||||
// node compare.mjs <summary.json> <baseline.json>
|
||||
//
|
||||
// Exit codes:
|
||||
// 0 — every metric within tolerance.
|
||||
// 1 — one or more metrics regressed, or a baseline metric is absent from
|
||||
// the current summary (suite drift — either a scenario was deleted
|
||||
// or a tag was renamed without updating baseline.json).
|
||||
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { argv, exit } from 'node:process';
|
||||
|
||||
if (argv.length < 4) {
|
||||
console.error('usage: compare.mjs <summary.json> <baseline.json>');
|
||||
exit(2);
|
||||
}
|
||||
|
||||
const summaryPath = argv[2];
|
||||
const baselinePath = argv[3];
|
||||
|
||||
const summary = JSON.parse(readFileSync(summaryPath, 'utf8'));
|
||||
const baseline = JSON.parse(readFileSync(baselinePath, 'utf8'));
|
||||
|
||||
// k6 summary metrics are keyed verbatim with the tag: e.g.
|
||||
// "http_req_duration{op:smoke.login}".
|
||||
const metricKeyFor = (op) => `http_req_duration{op:${op}}`;
|
||||
|
||||
// Map k6 percentile field names → baseline field names.
|
||||
const PERCENTILES = [
|
||||
{ baseline: 'p50', k6: 'med' },
|
||||
{ baseline: 'p95', k6: 'p(95)' },
|
||||
{ baseline: 'p99', k6: 'p(99)' },
|
||||
];
|
||||
|
||||
const rows = [];
|
||||
let anyRegression = false;
|
||||
let anyMissing = false;
|
||||
|
||||
for (const [op, base] of Object.entries(baseline)) {
|
||||
if (op.startsWith('_')) continue; // skip _comment etc.
|
||||
const metric = summary.metrics?.[metricKeyFor(op)];
|
||||
if (!metric) {
|
||||
rows.push({ op, status: 'MISSING', detail: 'no data for this tag in current summary' });
|
||||
anyMissing = true;
|
||||
continue;
|
||||
}
|
||||
const tol = (base.tolerance_pct ?? 10) / 100;
|
||||
// Noise-floor guard: at sub-millisecond percentiles, a 10% tolerance is
|
||||
// smaller than a single context switch. Require the absolute delta to
|
||||
// exceed `min_delta_ms` too — otherwise the swing is below the measurement
|
||||
// floor and we don't flag it. Default 0.5ms is conservative for the load
|
||||
// suite's typical 0.3–3ms range. Override per metric in baseline.json.
|
||||
const minDelta = base.min_delta_ms ?? 0.5;
|
||||
|
||||
for (const p of PERCENTILES) {
|
||||
const baseVal = base[p.baseline];
|
||||
// k6 --summary-export puts percentile values directly on the metric, not
|
||||
// inside a `.values` wrapper.
|
||||
const curVal = metric[p.k6];
|
||||
if (baseVal === undefined || curVal === undefined) continue;
|
||||
|
||||
const deltaPct = ((curVal - baseVal) / baseVal) * 100;
|
||||
const deltaAbs = curVal - baseVal;
|
||||
const limitPct = baseVal * (1 + tol);
|
||||
// Regression only if BOTH the % rule AND the absolute floor are breached.
|
||||
const regressed = curVal > limitPct && deltaAbs > minDelta;
|
||||
if (regressed) anyRegression = true;
|
||||
rows.push({
|
||||
op,
|
||||
percentile: p.baseline,
|
||||
base: baseVal,
|
||||
cur: curVal,
|
||||
deltaPct,
|
||||
regressed,
|
||||
tol: base.tolerance_pct ?? 10,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// ── Render table ──────────────────────────────────────────────────────────
|
||||
|
||||
const ms = (v) => `${v.toFixed(1)}ms`;
|
||||
const pct = (v) => `${v >= 0 ? '+' : ''}${v.toFixed(1)}%`;
|
||||
|
||||
function pad(s, n) {
|
||||
if (s.length >= n) return s;
|
||||
return s + ' '.repeat(n - s.length);
|
||||
}
|
||||
|
||||
const colOp = Math.max(20, ...rows.map((r) => r.op.length));
|
||||
const colP = 5;
|
||||
const colVal = 11;
|
||||
|
||||
console.log();
|
||||
console.log(
|
||||
pad('metric', colOp + 1) +
|
||||
pad('pctl', colP + 1) +
|
||||
pad('baseline', colVal + 1) +
|
||||
pad('current', colVal + 1) +
|
||||
pad('delta', 9) +
|
||||
'status',
|
||||
);
|
||||
console.log('-'.repeat(colOp + colP + colVal * 2 + 9 + 12));
|
||||
|
||||
for (const r of rows) {
|
||||
if (r.status === 'MISSING') {
|
||||
console.log(`${pad(r.op, colOp + 1)}${pad('-', colP + 1)}${pad('-', colVal + 1)}${pad('-', colVal + 1)}${pad('-', 9)}MISSING — ${r.detail}`);
|
||||
continue;
|
||||
}
|
||||
const status = r.regressed ? `REGRESSION (tol ±${r.tol}%)` : 'ok';
|
||||
console.log(
|
||||
pad(r.op, colOp + 1) +
|
||||
pad(r.percentile, colP + 1) +
|
||||
pad(ms(r.base), colVal + 1) +
|
||||
pad(ms(r.cur), colVal + 1) +
|
||||
pad(pct(r.deltaPct), 9) +
|
||||
status,
|
||||
);
|
||||
}
|
||||
|
||||
console.log();
|
||||
if (anyRegression || anyMissing) {
|
||||
if (anyRegression) console.error('compare.mjs: one or more metrics regressed beyond tolerance.');
|
||||
if (anyMissing) console.error('compare.mjs: one or more baseline metrics are missing from the current summary.');
|
||||
exit(1);
|
||||
}
|
||||
console.log('compare.mjs: all metrics within tolerance.');
|
||||
exit(0);
|
||||
@@ -0,0 +1,38 @@
|
||||
// Login helper for OxiCloud k6 load tests.
|
||||
// Returns the JWT access token from POST /api/auth/login.
|
||||
|
||||
import { check, fail } from 'k6';
|
||||
import http from 'k6/http';
|
||||
import { BASE } from './http.js';
|
||||
|
||||
/**
|
||||
* Log in via the public auth endpoint and return the bearer token.
|
||||
*
|
||||
* `body` shape matches `auth_handler.rs::login` — `{username, password}`.
|
||||
* The response carries `access_token` plus refresh state we don't need here.
|
||||
*
|
||||
* The `op` tag should be scenario-qualified (e.g. 'smoke.login') so the
|
||||
* recorded metric matches the corresponding key in baseline/baseline.json.
|
||||
* Bare 'login' is fine for ad-hoc scripts that aren't regression-gated.
|
||||
*
|
||||
* @param {string} username
|
||||
* @param {string} password
|
||||
* @param {string} [op='login']
|
||||
* @returns {string}
|
||||
*/
|
||||
export function login(username, password, op = 'login') {
|
||||
const res = http.post(
|
||||
`${BASE}/api/auth/login`,
|
||||
JSON.stringify({ username, password }),
|
||||
{ headers: { 'Content-Type': 'application/json' }, tags: { op } },
|
||||
);
|
||||
|
||||
const ok = check(res, {
|
||||
'login 200': (r) => r.status === 200,
|
||||
'login has access_token': (r) => !!r.json('access_token'),
|
||||
});
|
||||
if (!ok) {
|
||||
fail(`login failed for ${username}: status=${res.status}, body=${res.body}`);
|
||||
}
|
||||
return res.json('access_token');
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
// Shared HTTP helpers for OxiCloud k6 load tests.
|
||||
// Centralises the base URL and adds the Bearer token + JSON Content-Type
|
||||
// headers expected by every protected endpoint.
|
||||
|
||||
import http from 'k6/http';
|
||||
|
||||
/**
|
||||
* Base URL of the OxiCloud server under test.
|
||||
* Read from K6_BASE_URL (set by run.sh), defaulting to the load-suite port.
|
||||
*/
|
||||
export const BASE = __ENV.K6_BASE_URL || 'http://localhost:8088';
|
||||
|
||||
/**
|
||||
* Build a request params object with auth + JSON headers and a `tag` so
|
||||
* the response's metrics are isolated under `<scenario>.<op>`.
|
||||
*
|
||||
* @param {string} token Bearer token from auth.login()
|
||||
* @param {string} op Metric tag, e.g. 'folder_cascade.list_depth8'
|
||||
*/
|
||||
export function jsonParams(token, op) {
|
||||
return {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
tags: { op },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Same as jsonParams but without a body content-type — for GETs and DELETEs.
|
||||
*/
|
||||
export function authParams(token, op) {
|
||||
return {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
tags: { op },
|
||||
};
|
||||
}
|
||||
|
||||
export const httpClient = http;
|
||||
@@ -0,0 +1,76 @@
|
||||
// Baseline-driven thresholds and shared helpers for k6 scenarios.
|
||||
//
|
||||
// Each scenario tags its requests with `op: '<scenario>.<op>'` (see http.js
|
||||
// `jsonParams` / `authParams`). The baseline file lists one entry per
|
||||
// `<scenario>.<op>` with p50/p95/p99 and a tolerance percentage; we convert
|
||||
// every relevant entry to a k6 threshold so the run fails directly on
|
||||
// regression — `compare.mjs` then produces the human-readable diff.
|
||||
|
||||
// Resolve relative to THIS file (lib/metrics.js), not the importer. Future
|
||||
// k6 versions will align open()'s path-resolution with ES module semantics;
|
||||
// using import.meta.resolve() future-proofs against the warning logged by
|
||||
// k6 ≥ 0.50.
|
||||
const LOAD_BASELINE_PATH = import.meta.resolve('../baseline/load.json');
|
||||
const SMOKE_BASELINE_PATH = import.meta.resolve('../baseline/smoke.json');
|
||||
const MANIFEST_PATH = import.meta.resolve('../results/seed-manifest.json');
|
||||
|
||||
// Eagerly load both baseline files at module init (open() is only allowed
|
||||
// in init context). Keys are disjoint by scenario prefix, so merging the
|
||||
// two maps is safe; `thresholdsFromBaseline(prefix)` filters from the union.
|
||||
const BASELINE = {
|
||||
...JSON.parse(open(LOAD_BASELINE_PATH)),
|
||||
...JSON.parse(open(SMOKE_BASELINE_PATH)),
|
||||
};
|
||||
|
||||
/**
|
||||
* Return the merged baseline (load + smoke) — convenient for tooling that
|
||||
* wants to inspect everything; scenarios should use `thresholdsFromBaseline`.
|
||||
*/
|
||||
export function loadBaseline() {
|
||||
return BASELINE;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a k6 `thresholds` object from the baseline files, filtered to the
|
||||
* given scenario prefix (e.g. 'folder_cascade').
|
||||
*
|
||||
* Result shape (k6 expects metric-name → threshold-expression-array):
|
||||
* {
|
||||
* 'http_req_duration{op:folder_cascade.list_depth8}': [
|
||||
* 'p(95)<49.5', // 45 * (1 + 10/100)
|
||||
* 'p(99)<88.0',
|
||||
* ],
|
||||
* }
|
||||
*
|
||||
* @param {string} scenarioPrefix
|
||||
*/
|
||||
export function thresholdsFromBaseline(scenarioPrefix) {
|
||||
const baseline = BASELINE;
|
||||
const thresholds = {};
|
||||
for (const [key, val] of Object.entries(baseline)) {
|
||||
if (key.startsWith('_')) continue; // skip _comment etc.
|
||||
if (!key.startsWith(`${scenarioPrefix}.`)) continue;
|
||||
const tol = (val.tolerance_pct || 10) / 100;
|
||||
// Mirror compare.mjs: use the larger of the %-based limit and the
|
||||
// absolute-floor limit (baseline + min_delta_ms). Below sub-millisecond
|
||||
// scale, the % rule alone fires on noise — the floor stops that.
|
||||
const minDelta = val.min_delta_ms ?? 0.5;
|
||||
const p95Limit = Math.max(val.p95 * (1 + tol), val.p95 + minDelta);
|
||||
const p99Limit = Math.max(val.p99 * (1 + tol), val.p99 + minDelta);
|
||||
thresholds[`http_req_duration{op:${key}}`] = [
|
||||
`p(95)<${p95Limit.toFixed(2)}`,
|
||||
`p(99)<${p99Limit.toFixed(2)}`,
|
||||
];
|
||||
}
|
||||
// `abortOnFail: false` (default) keeps the run going so we collect all
|
||||
// regressions in one pass; the non-zero exit at the end still fails CI.
|
||||
return thresholds;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the seed manifest written by `cargo run --bin load-seed`.
|
||||
*/
|
||||
export function loadManifest() {
|
||||
const raw = open(MANIFEST_PATH);
|
||||
return JSON.parse(raw);
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
#!/usr/bin/env node
|
||||
// merge-summaries.mjs — combine multiple k6 --summary-export outputs into
|
||||
// one. Used because k6 accepts a single script per invocation, but our
|
||||
// regression-diff tooling (compare.mjs, bake-baseline.mjs) expects one
|
||||
// summary per run.
|
||||
//
|
||||
// Per-scenario summaries are disjoint in their metric tags
|
||||
// (folder_cascade.* vs share_cascade_rebac.* vs subject_group_nested.*),
|
||||
// so merging is just a union of the `metrics` maps. The top-level
|
||||
// envelope (root_group, options, etc.) is taken from the first input.
|
||||
//
|
||||
// Usage:
|
||||
// node merge-summaries.mjs <in1.json> [in2.json …] <out.json>
|
||||
|
||||
import { readFileSync, writeFileSync } from 'node:fs';
|
||||
import { argv, exit } from 'node:process';
|
||||
|
||||
if (argv.length < 5) {
|
||||
console.error('usage: merge-summaries.mjs <in1.json> [in2.json …] <out.json>');
|
||||
exit(2);
|
||||
}
|
||||
|
||||
const inputs = argv.slice(2, -1);
|
||||
const output = argv[argv.length - 1];
|
||||
|
||||
const merged = JSON.parse(readFileSync(inputs[0], 'utf8'));
|
||||
merged.metrics = { ...merged.metrics };
|
||||
|
||||
for (const path of inputs.slice(1)) {
|
||||
const next = JSON.parse(readFileSync(path, 'utf8'));
|
||||
for (const [k, v] of Object.entries(next.metrics ?? {})) {
|
||||
// If two scenarios both report a global metric (e.g. http_req_duration
|
||||
// with no op tag), the second wins — these aren't gated by baseline.json,
|
||||
// so it's only the per-`op` metrics that need to be preserved precisely.
|
||||
merged.metrics[k] = v;
|
||||
}
|
||||
}
|
||||
|
||||
writeFileSync(output, `${JSON.stringify(merged, null, 2)}\n`);
|
||||
console.log(`merged ${inputs.length} summaries → ${output}`);
|
||||
Executable
+166
@@ -0,0 +1,166 @@
|
||||
#!/usr/bin/env bash
|
||||
# Full k6 load-test runner.
|
||||
# Starts postgres + OxiCloud server, seeds fixtures, runs k6 scenarios,
|
||||
# compares results against baseline/load.json, tears everything down.
|
||||
#
|
||||
# Usage (from repo root):
|
||||
# bash tests/load/run.sh
|
||||
#
|
||||
# Env overrides:
|
||||
# BUILD_TARGET=release # prefer release build for accurate timings
|
||||
# LOAD_DEPTH=8 # override seeder shape (otherwise read from test.env)
|
||||
# LOAD_FANOUT=3
|
||||
# LOAD_FILES_PER_LEAF=3
|
||||
# LOAD_EXTRA_USERS=20
|
||||
# LOAD_GROUP_DEPTH=3
|
||||
# LOAD_GROUP_FANOUT=5
|
||||
# K6_SUMMARY_OUT=path # explicit output path (default: tests/load/results/<ts>.json)
|
||||
#
|
||||
# Prerequisites: docker, cargo, k6 >= 0.46, node >= 18
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
COMMON="$REPO_ROOT/tests/common"
|
||||
LOAD_DIR="$REPO_ROOT/tests/load"
|
||||
|
||||
# shellcheck source=test.env
|
||||
source "$LOAD_DIR/test.env"
|
||||
|
||||
SERVER_PORT="${base_url##*:}"
|
||||
|
||||
log() { echo "[load] $*"; }
|
||||
die() { echo "[load] ERROR: $*" >&2; exit 1; }
|
||||
|
||||
wait_for_http() {
|
||||
local url="$1" timeout="${2:-120}"
|
||||
local deadline=$(( $(date +%s) + timeout ))
|
||||
until curl -sf "$url" >/dev/null 2>&1; do
|
||||
[[ $(date +%s) -ge $deadline ]] && die "Timeout waiting for $url"
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
command -v k6 >/dev/null 2>&1 || die "k6 is required (https://k6.io/docs/get-started/installation/)"
|
||||
command -v node >/dev/null 2>&1 || die "node >= 18 is required for compare.mjs"
|
||||
|
||||
SERVER_PID=""
|
||||
cleanup() {
|
||||
if [[ -n "$SERVER_PID" ]]; then
|
||||
log "Stopping OxiCloud server (pid $SERVER_PID)..."
|
||||
kill "$SERVER_PID" 2>/dev/null || true
|
||||
wait "$SERVER_PID" 2>/dev/null || true
|
||||
fi
|
||||
bash "$COMMON/stop-db.sh" || true
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
bash "$COMMON/spawn-db.sh"
|
||||
|
||||
# NOTE: do NOT run init-test-schema.sh here. The OxiCloud server applies
|
||||
# sqlx migrations on startup; applying them via raw psql first leaves the
|
||||
# server's _sqlx_migrations tracking table empty, which makes the second
|
||||
# pass try to re-ALTER tables that already have the column (e.g. migration
|
||||
# 20260507000000_session_family.sql), and the server panics on boot.
|
||||
|
||||
set -a
|
||||
# shellcheck source=../common/server.env
|
||||
source "$COMMON/server.env"
|
||||
OXICLOUD_SERVER_PORT=$SERVER_PORT
|
||||
OXICLOUD_STORAGE_PATH="$LOAD_DIR/storage"
|
||||
set +a
|
||||
|
||||
rm -rf "$OXICLOUD_STORAGE_PATH"
|
||||
mkdir -p "$OXICLOUD_STORAGE_PATH"
|
||||
|
||||
BUILD_TARGET="${BUILD_TARGET:-release}"
|
||||
# Respect CARGO_TARGET_DIR for self-hosted runners that bind-mount target/
|
||||
# outside the workspace (avoids actions/checkout EBUSY on the mount point).
|
||||
TARGET_DIR="${CARGO_TARGET_DIR:-$REPO_ROOT/target}"
|
||||
OXICLOUD_BIN="$TARGET_DIR/$BUILD_TARGET/oxicloud"
|
||||
SEED_BIN="$TARGET_DIR/$BUILD_TARGET/load-seed"
|
||||
|
||||
# Build both bins in one invocation. `load_seed_bin` is an empty marker
|
||||
# feature that gates the load-seed bin without affecting oxicloud's dep
|
||||
# graph, so cargo plans a single workspace build and oxicloud compiles
|
||||
# exactly once. (See Cargo.toml comments on `load_seed_bin`.)
|
||||
if [[ ! -x "$OXICLOUD_BIN" || ! -x "$SEED_BIN" ]]; then
|
||||
log "Building OxiCloud + load-seed ($BUILD_TARGET)..."
|
||||
if [[ "$BUILD_TARGET" == "release" ]]; then
|
||||
cargo build --release --features load_seed_bin --bin oxicloud --bin load-seed
|
||||
else
|
||||
cargo build --features load_seed_bin --bin oxicloud --bin load-seed
|
||||
fi
|
||||
fi
|
||||
|
||||
# Start the server FIRST so its sqlx::migrate! populates _sqlx_migrations
|
||||
# against the clean DB. The seeder then runs against the migrated schema
|
||||
# while the server is still alive (it issues plain INSERTs, no DDL).
|
||||
log "Starting OxiCloud server ($BUILD_TARGET) on port $SERVER_PORT..."
|
||||
"$OXICLOUD_BIN" &
|
||||
SERVER_PID=$!
|
||||
wait_for_http "$base_url/ready" 120
|
||||
log "Server ready."
|
||||
|
||||
log "Seeding fixtures..."
|
||||
DEPTH="${LOAD_DEPTH:-${load_depth:-5}}"
|
||||
FANOUT="${LOAD_FANOUT:-${load_fanout:-4}}"
|
||||
FILES_PER_LEAF="${LOAD_FILES_PER_LEAF:-${load_files_per_leaf:-3}}"
|
||||
EXTRA_USERS="${LOAD_EXTRA_USERS:-${load_extra_users:-20}}"
|
||||
GROUP_DEPTH="${LOAD_GROUP_DEPTH:-${load_group_depth:-3}}"
|
||||
GROUP_FANOUT="${LOAD_GROUP_FANOUT:-${load_group_fanout:-5}}"
|
||||
|
||||
mkdir -p "$LOAD_DIR/results"
|
||||
MANIFEST_PATH="$LOAD_DIR/results/seed-manifest.json"
|
||||
|
||||
"$SEED_BIN" \
|
||||
--depth "$DEPTH" \
|
||||
--fanout "$FANOUT" \
|
||||
--files-per-leaf "$FILES_PER_LEAF" \
|
||||
--extra-users "$EXTRA_USERS" \
|
||||
--group-depth "$GROUP_DEPTH" \
|
||||
--group-fanout "$GROUP_FANOUT" \
|
||||
--password "${password:-TestPassword1!}" \
|
||||
--manifest "$MANIFEST_PATH"
|
||||
|
||||
TS="$(date +%s)"
|
||||
SUMMARY_OUT="${K6_SUMMARY_OUT:-$LOAD_DIR/results/run-$TS.json}"
|
||||
|
||||
export K6_BASE_URL="$base_url"
|
||||
export K6_USERNAME="${username:-admin}"
|
||||
export K6_PASSWORD="${password:-TestPassword1!}"
|
||||
|
||||
# k6 only accepts one script per invocation, so each scenario runs separately
|
||||
# and we merge the summaries afterwards. Per-scenario summaries also make it
|
||||
# easier to attribute regressions when looking at raw artifacts in CI.
|
||||
SCENARIOS=(folder_cascade share_cascade_rebac subject_group_nested)
|
||||
PARTIAL_SUMMARIES=()
|
||||
K6_FAILED=0
|
||||
|
||||
for name in "${SCENARIOS[@]}"; do
|
||||
partial="$LOAD_DIR/results/run-$TS-$name.json"
|
||||
PARTIAL_SUMMARIES+=("$partial")
|
||||
log "Running k6 scenario: $name"
|
||||
# --summary-trend-stats forces p(99) into the summary export; k6's default
|
||||
# only includes avg/min/med/max/p(90)/p(95), so without it baseline.p99 can
|
||||
# never be baked or diffed.
|
||||
k6 run \
|
||||
--summary-export="$partial" \
|
||||
--summary-trend-stats="avg,min,med,max,p(90),p(95),p(99)" \
|
||||
--quiet \
|
||||
"$LOAD_DIR/scenarios/$name.js" \
|
||||
|| K6_FAILED=$?
|
||||
done
|
||||
|
||||
log "Merging summaries -> $SUMMARY_OUT"
|
||||
node "$LOAD_DIR/merge-summaries.mjs" "${PARTIAL_SUMMARIES[@]}" "$SUMMARY_OUT"
|
||||
|
||||
log "Comparing against baseline..."
|
||||
COMPARE_RC=0
|
||||
node "$LOAD_DIR/compare.mjs" "$SUMMARY_OUT" "$LOAD_DIR/baseline/load.json" || COMPARE_RC=$?
|
||||
|
||||
if [[ "$K6_FAILED" -ne 0 ]]; then
|
||||
log "k6 reported threshold failures (exit $K6_FAILED)."
|
||||
exit "$K6_FAILED"
|
||||
fi
|
||||
exit "$COMPARE_RC"
|
||||
@@ -0,0 +1,60 @@
|
||||
// folder_cascade.js — measures the read-path of `GET /api/folders/{id}/resources?resource_types=folder`
|
||||
// at four depths against a pre-seeded tree. Captures how listing cost scales
|
||||
// with ltree depth (cf. `idx_folders_lpath` GiST index). Mid-depth samples
|
||||
// (depth4, depth8) fall back to `deepest` when the seeded tree is shallower
|
||||
// than that depth — see load-seed.rs::build_subtree.
|
||||
|
||||
import { check } from 'k6';
|
||||
import http from 'k6/http';
|
||||
import { BASE, authParams } from '../lib/http.js';
|
||||
import { login } from '../lib/auth.js';
|
||||
import { thresholdsFromBaseline, loadManifest } from '../lib/metrics.js';
|
||||
|
||||
const manifest = loadManifest();
|
||||
|
||||
export const options = {
|
||||
vus: 1,
|
||||
// 100 iterations gives p99 statistical meaning: at N=100, p99 = position 99,
|
||||
// representing one bad sample out of a hundred — a real percentile rather
|
||||
// than the worst-of-the-batch. At N=25 (the typical k6 example default),
|
||||
// p99 ≈ max, dominated by single-sample kernel/scheduler noise.
|
||||
iterations: 100,
|
||||
thresholds: thresholdsFromBaseline('folder_cascade'),
|
||||
};
|
||||
|
||||
// One per-VU login. K6 calls setup() once across the whole test, default()
|
||||
// `iterations` times per VU. Logging in inside default() would dominate the
|
||||
// per-iter cost; we hand the token down through the `data` arg.
|
||||
export function setup() {
|
||||
const token = login(manifest.admin.username, manifest.admin.password);
|
||||
return { token };
|
||||
}
|
||||
|
||||
export default function (data) {
|
||||
const { token } = data;
|
||||
const t = manifest.shared_subtree;
|
||||
|
||||
const r1 = http.get(
|
||||
`${BASE}/api/folders/${t.root}/resources?resource_types=folder`,
|
||||
authParams(token, 'folder_cascade.list_depth1'),
|
||||
);
|
||||
check(r1, { 'list depth1 200': (r) => r.status === 200 });
|
||||
|
||||
const r4 = http.get(
|
||||
`${BASE}/api/folders/${t.depth4}/resources?resource_types=folder`,
|
||||
authParams(token, 'folder_cascade.list_depth4'),
|
||||
);
|
||||
check(r4, { 'list depth4 200': (r) => r.status === 200 });
|
||||
|
||||
const r8 = http.get(
|
||||
`${BASE}/api/folders/${t.depth8}/resources?resource_types=folder`,
|
||||
authParams(token, 'folder_cascade.list_depth8'),
|
||||
);
|
||||
check(r8, { 'list depth8 200': (r) => r.status === 200 });
|
||||
|
||||
const rD = http.get(
|
||||
`${BASE}/api/folders/${t.deepest}/resources?resource_types=folder`,
|
||||
authParams(token, 'folder_cascade.list_depth_deep'),
|
||||
);
|
||||
check(rD, { 'list deepest 200': (r) => r.status === 200 });
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
// share_cascade_rebac.js — measures ReBAC AuthZ cascade via a direct user
|
||||
// grant. The seeder grants `read` on shared_subtree.root (depth 0) to the
|
||||
// grantee user; this scenario times how long the grantee takes to fetch
|
||||
// folders at varying depths inside that subtree.
|
||||
//
|
||||
// The AuthZ recursive-CTE in `pg_acl_engine` walks up the ancestor chain
|
||||
// from the requested folder until it finds a matching grant (or runs out
|
||||
// of ancestors). The deeper the requested folder, the more ancestors the
|
||||
// CTE has to traverse before hitting the grant on the root:
|
||||
//
|
||||
// fetch_at_grant_root → 0 ancestors walked (grant is right here)
|
||||
// fetch_as_grantee_depth4 → 4 ancestors walked
|
||||
// fetch_as_grantee_depth8 → 8 ancestors walked
|
||||
// fetch_as_grantee_depth_deep → `load_depth` ancestors walked
|
||||
//
|
||||
// A regression in AuthZ cost should show up as the cascade-depth curve
|
||||
// flattening or steepening — that's the value of intermediate samples.
|
||||
|
||||
import { check } from 'k6';
|
||||
import http from 'k6/http';
|
||||
import { BASE, authParams } from '../lib/http.js';
|
||||
import { login } from '../lib/auth.js';
|
||||
import { thresholdsFromBaseline, loadManifest } from '../lib/metrics.js';
|
||||
|
||||
const manifest = loadManifest();
|
||||
|
||||
export const options = {
|
||||
vus: 1,
|
||||
// See folder_cascade.js for the iteration-count rationale: N=100 makes
|
||||
// p99 a real percentile instead of the worst-of-the-batch.
|
||||
iterations: 100,
|
||||
thresholds: thresholdsFromBaseline('share_cascade_rebac'),
|
||||
};
|
||||
|
||||
export function setup() {
|
||||
const adminToken = login(manifest.admin.username, manifest.admin.password);
|
||||
const granteeToken = login(manifest.grantee.username, manifest.grantee.password);
|
||||
return { adminToken, granteeToken };
|
||||
}
|
||||
|
||||
export default function (data) {
|
||||
const { adminToken, granteeToken } = data;
|
||||
const t = manifest.shared_subtree;
|
||||
|
||||
// List grants on the granted folder (admin only).
|
||||
const grantsRes = http.get(
|
||||
`${BASE}/api/grants?resource_type=folder&resource_id=${t.root}`,
|
||||
authParams(adminToken, 'share_cascade_rebac.list_grants'),
|
||||
);
|
||||
check(grantsRes, { 'list grants 200': (r) => r.status === 200 });
|
||||
|
||||
// Baseline: grantee fetches the folder the grant is directly on. AuthZ
|
||||
// finds the matching grant on the first row of the CTE; zero ancestors
|
||||
// walked. This metric measures the constant overhead of an authorized
|
||||
// request — moves to the right of this metric is "cascade cost."
|
||||
const d1 = http.get(
|
||||
`${BASE}/api/folders/${t.root}/resources?resource_types=folder`,
|
||||
authParams(granteeToken, 'share_cascade_rebac.fetch_as_grantee_depth1'),
|
||||
);
|
||||
check(d1, { 'fetch root 200': (r) => r.status === 200 });
|
||||
|
||||
// Grantee fetches a mid-tree folder. AuthZ walks 4 ancestors before
|
||||
// hitting the grant.
|
||||
const d4 = http.get(
|
||||
`${BASE}/api/folders/${t.depth4}/resources?resource_types=folder`,
|
||||
authParams(granteeToken, 'share_cascade_rebac.fetch_as_grantee_depth4'),
|
||||
);
|
||||
check(d4, { 'fetch depth4 200': (r) => r.status === 200 });
|
||||
|
||||
// Grantee fetches a folder 8 levels under the granted root. AuthZ
|
||||
// walks 8 ancestors.
|
||||
const d8 = http.get(
|
||||
`${BASE}/api/folders/${t.depth8}/resources?resource_types=folder`,
|
||||
authParams(granteeToken, 'share_cascade_rebac.fetch_as_grantee_depth8'),
|
||||
);
|
||||
check(d8, { 'fetch depth8 200': (r) => r.status === 200 });
|
||||
|
||||
// Grantee fetches the deepest descendant — full-length cascade.
|
||||
const dD = http.get(
|
||||
`${BASE}/api/folders/${t.deepest}/resources?resource_types=folder`,
|
||||
authParams(granteeToken, 'share_cascade_rebac.fetch_as_grantee_depth_deep'),
|
||||
);
|
||||
check(dD, { 'fetch deepest 200': (r) => r.status === 200 });
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
// smoke.js — fast PR-tier check. Verifies the load harness still builds and
|
||||
// the server boots, exercising one happy-path of every HTTP verb the long
|
||||
// scenarios use. No regression gate; run.sh's `smoke.sh` skips compare.mjs.
|
||||
|
||||
import { check, sleep } from 'k6';
|
||||
import http from 'k6/http';
|
||||
import { BASE, jsonParams, authParams } from '../lib/http.js';
|
||||
import { login } from '../lib/auth.js';
|
||||
import { thresholdsFromBaseline } from '../lib/metrics.js';
|
||||
|
||||
export const options = {
|
||||
vus: 1,
|
||||
iterations: 1,
|
||||
thresholds: thresholdsFromBaseline('smoke'),
|
||||
};
|
||||
|
||||
// Admin creds match tests/load/test.env defaults — overridable via env.
|
||||
const USERNAME = __ENV.K6_USERNAME || 'admin';
|
||||
const PASSWORD = __ENV.K6_PASSWORD || 'TestPassword1!';
|
||||
|
||||
export default function () {
|
||||
// 1. login
|
||||
const token = login(USERNAME, PASSWORD, 'smoke.login');
|
||||
|
||||
// 2. create a scratch folder at root
|
||||
const folderName = `smoke_${Date.now()}`;
|
||||
const createRes = http.post(
|
||||
`${BASE}/api/folders`,
|
||||
JSON.stringify({ name: folderName }),
|
||||
jsonParams(token, 'smoke.create_folder'),
|
||||
);
|
||||
check(createRes, { 'create folder 200/201': (r) => r.status === 200 || r.status === 201 });
|
||||
const folderId = createRes.json('id');
|
||||
|
||||
// 3. upload a tiny file via multipart
|
||||
const fileData = http.file('hello\n', 'smoke.txt', 'text/plain');
|
||||
const uploadRes = http.post(
|
||||
`${BASE}/api/files/upload`,
|
||||
{ folder_id: folderId, file: fileData },
|
||||
{ headers: { Authorization: `Bearer ${token}` }, tags: { op: 'smoke.upload_tiny' } },
|
||||
);
|
||||
const uploadOk = check(uploadRes, {
|
||||
'upload 200/201': (r) => r.status === 200 || r.status === 201,
|
||||
});
|
||||
if (!uploadOk) {
|
||||
console.error(
|
||||
`upload failed: status=${uploadRes.status}, body=${uploadRes.body}, headers=${JSON.stringify(uploadRes.headers)}`,
|
||||
);
|
||||
}
|
||||
|
||||
// 4. list root folders for this user
|
||||
const listRes = http.get(`${BASE}/api/folders`, authParams(token, 'smoke.list_root'));
|
||||
check(listRes, { 'list root 200': (r) => r.status === 200 });
|
||||
|
||||
// 5. delete the scratch folder
|
||||
const delRes = http.del(
|
||||
`${BASE}/api/folders/${folderId}`,
|
||||
null,
|
||||
authParams(token, 'smoke.delete_folder'),
|
||||
);
|
||||
check(delRes, { 'delete 200/204': (r) => r.status === 200 || r.status === 204 });
|
||||
|
||||
sleep(0.1);
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
// subject_group_nested.js — measures the worst-case AuthZ path: a user who
|
||||
// is a direct member of the *innermost* group of a depth-N nested chain, and
|
||||
// the grant is on the *outermost* group. Every AuthZ check has to expand the
|
||||
// chain transitively. Pairs with share_cascade_rebac.js to attribute regressions
|
||||
// to either the resource side (folder cascade) or the subject side (group
|
||||
// expansion).
|
||||
//
|
||||
// Same depth-gradient idea as share_cascade_rebac: measure AuthZ cost at
|
||||
// depth 1 / 4 / 8 / deep so we can see WHERE in the chain a regression
|
||||
// lands. The subject-side expansion (group_member → leaf → mid → root group
|
||||
// → grant) runs once per request regardless of folder depth, but the
|
||||
// resource-side cascade (folder ancestors walked to find the grant) grows
|
||||
// linearly with depth — so depth4/depth8/deep moving together while depth1
|
||||
// stays flat would point at the resource side; all four moving together
|
||||
// would point at the group-expansion path.
|
||||
|
||||
import { check } from 'k6';
|
||||
import http from 'k6/http';
|
||||
import { BASE, authParams } from '../lib/http.js';
|
||||
import { login } from '../lib/auth.js';
|
||||
import { thresholdsFromBaseline, loadManifest } from '../lib/metrics.js';
|
||||
|
||||
const manifest = loadManifest();
|
||||
|
||||
export const options = {
|
||||
vus: 1,
|
||||
// See folder_cascade.js for the iteration-count rationale: N=100 makes
|
||||
// p99 a real percentile instead of the worst-of-the-batch.
|
||||
iterations: 100,
|
||||
thresholds: thresholdsFromBaseline('subject_group_nested'),
|
||||
};
|
||||
|
||||
export function setup() {
|
||||
const memberToken = login(manifest.group_member.username, manifest.group_member.password);
|
||||
return { memberToken };
|
||||
}
|
||||
|
||||
export default function (data) {
|
||||
const { memberToken } = data;
|
||||
const t = manifest.group_subtree;
|
||||
|
||||
// Baseline: group member fetches the folder the grant is directly on.
|
||||
// Subject-side expansion runs (user → leaf → mid → root group → grant);
|
||||
// resource-side walk is zero ancestors. Captures the constant subject-
|
||||
// expansion overhead.
|
||||
const d1 = http.get(
|
||||
`${BASE}/api/folders/${t.root}/resources?resource_types=folder`,
|
||||
authParams(memberToken, 'subject_group_nested.fetch_as_member_depth1'),
|
||||
);
|
||||
check(d1, { 'fetch root 200': (r) => r.status === 200 });
|
||||
|
||||
// Mid-tree: 4 folder ancestors walked + subject expansion.
|
||||
const d4 = http.get(
|
||||
`${BASE}/api/folders/${t.depth4}/resources?resource_types=folder`,
|
||||
authParams(memberToken, 'subject_group_nested.fetch_as_member_depth4'),
|
||||
);
|
||||
check(d4, { 'fetch depth4 200': (r) => r.status === 200 });
|
||||
|
||||
// 8 folder ancestors walked + subject expansion.
|
||||
const d8 = http.get(
|
||||
`${BASE}/api/folders/${t.depth8}/resources?resource_types=folder`,
|
||||
authParams(memberToken, 'subject_group_nested.fetch_as_member_depth8'),
|
||||
);
|
||||
check(d8, { 'fetch depth8 200': (r) => r.status === 200 });
|
||||
|
||||
// Worst case: full-depth folder cascade × full-chain subject expansion.
|
||||
// group_member → leaf group → mid group → root group → grant → folder root → deepest.
|
||||
const dD = http.get(
|
||||
`${BASE}/api/folders/${t.deepest}/resources?resource_types=folder`,
|
||||
authParams(memberToken, 'subject_group_nested.fetch_as_member_depth_deep'),
|
||||
);
|
||||
check(dD, { 'fetch deepest 200': (r) => r.status === 200 });
|
||||
}
|
||||
Executable
+108
@@ -0,0 +1,108 @@
|
||||
#!/usr/bin/env bash
|
||||
# Smoke (PR-tier) k6 load runner.
|
||||
# Same shape as run.sh but: no seeder, only the smoke scenario, no regression
|
||||
# diff. Goal is harness liveness - does the server still boot and does k6 still
|
||||
# wire up - not perf gating.
|
||||
#
|
||||
# Usage (from repo root):
|
||||
# bash tests/load/smoke.sh
|
||||
#
|
||||
# Env overrides:
|
||||
# BUILD_TARGET=debug # debug is fine; smoke doesn't measure timings
|
||||
#
|
||||
# Prerequisites: docker, cargo, k6 >= 0.46
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
COMMON="$REPO_ROOT/tests/common"
|
||||
LOAD_DIR="$REPO_ROOT/tests/load"
|
||||
|
||||
# shellcheck source=test.env
|
||||
source "$LOAD_DIR/test.env"
|
||||
|
||||
SERVER_PORT="${base_url##*:}"
|
||||
|
||||
log() { echo "[load-smoke] $*"; }
|
||||
die() { echo "[load-smoke] ERROR: $*" >&2; exit 1; }
|
||||
|
||||
wait_for_http() {
|
||||
local url="$1" timeout="${2:-60}"
|
||||
local deadline=$(( $(date +%s) + timeout ))
|
||||
until curl -sf "$url" >/dev/null 2>&1; do
|
||||
[[ $(date +%s) -ge $deadline ]] && die "Timeout waiting for $url"
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
command -v k6 >/dev/null 2>&1 || die "k6 is required (https://k6.io/docs/get-started/installation/)"
|
||||
|
||||
SERVER_PID=""
|
||||
cleanup() {
|
||||
if [[ -n "$SERVER_PID" ]]; then
|
||||
log "Stopping OxiCloud server (pid $SERVER_PID)..."
|
||||
kill "$SERVER_PID" 2>/dev/null || true
|
||||
wait "$SERVER_PID" 2>/dev/null || true
|
||||
fi
|
||||
bash "$COMMON/stop-db.sh" || true
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
bash "$COMMON/spawn-db.sh"
|
||||
# Server applies sqlx migrations on startup; don't double-apply via psql here.
|
||||
# See run.sh for the gory details.
|
||||
|
||||
set -a
|
||||
# shellcheck source=../common/server.env
|
||||
source "$COMMON/server.env"
|
||||
OXICLOUD_SERVER_PORT=$SERVER_PORT
|
||||
OXICLOUD_STORAGE_PATH="$LOAD_DIR/storage"
|
||||
set +a
|
||||
|
||||
rm -rf "$OXICLOUD_STORAGE_PATH"
|
||||
mkdir -p "$OXICLOUD_STORAGE_PATH"
|
||||
|
||||
BUILD_TARGET="${BUILD_TARGET:-debug}"
|
||||
OXICLOUD_BIN="$REPO_ROOT/target/$BUILD_TARGET/oxicloud"
|
||||
|
||||
if [[ -x "$OXICLOUD_BIN" ]]; then
|
||||
log "Starting pre-built OxiCloud server ($BUILD_TARGET) on port $SERVER_PORT..."
|
||||
"$OXICLOUD_BIN" &
|
||||
else
|
||||
log "Building and starting OxiCloud server ($BUILD_TARGET) on port $SERVER_PORT..."
|
||||
cd "$REPO_ROOT"
|
||||
if [[ "$BUILD_TARGET" == "release" ]]; then
|
||||
cargo run --release &
|
||||
else
|
||||
cargo run &
|
||||
fi
|
||||
fi
|
||||
SERVER_PID=$!
|
||||
wait_for_http "$base_url/ready" 120
|
||||
log "Server ready."
|
||||
|
||||
# Bootstrap the admin account via /api/setup (one-shot — disabled once an
|
||||
# admin exists, mirrors tests/api/setup.hurl). Without this the smoke scenario
|
||||
# would have no one to log in as.
|
||||
log "Creating admin via /api/setup..."
|
||||
SETUP_BODY=$(printf '{"username":"%s","email":"%s","password":"%s"}' \
|
||||
"${username:-admin}" "${email:-admin@example.com}" "${password:-TestPassword1!}")
|
||||
SETUP_STATUS=$(curl -sS -o /dev/null -w '%{http_code}' \
|
||||
-X POST "$base_url/api/setup" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "$SETUP_BODY")
|
||||
if [[ "$SETUP_STATUS" != "201" ]]; then
|
||||
die "/api/setup returned $SETUP_STATUS (expected 201)"
|
||||
fi
|
||||
|
||||
export K6_BASE_URL="$base_url"
|
||||
export K6_USERNAME="${username:-admin}"
|
||||
export K6_PASSWORD="${password:-TestPassword1!}"
|
||||
|
||||
log "Running smoke scenario..."
|
||||
k6 run \
|
||||
--summary-trend-stats="avg,min,med,max,p(90),p(95),p(99)" \
|
||||
--quiet \
|
||||
"$LOAD_DIR/scenarios/smoke.js"
|
||||
|
||||
log "Smoke OK."
|
||||
@@ -0,0 +1,31 @@
|
||||
# Test credentials and base URL for tests/load/ — NOT real secrets.
|
||||
# Mirrors tests/api/test.env shape; uses a different server port (8088)
|
||||
# so it does not collide with api-test (8087).
|
||||
base_url=http://localhost:8088
|
||||
username=admin
|
||||
email=admin@example.com
|
||||
# gitguardian:ignore
|
||||
password=TestPassword1!
|
||||
|
||||
# ── Seed sizing ─────────────────────────────────────────────────────────────
|
||||
# Folder count is exponential: total folders per subtree ≈ fanout^depth.
|
||||
# The seeder builds TWO subtrees (one for the user-grant scenario, one for the
|
||||
# group-grant scenario), so multiply by 2 for the grand total. Each leaf folder
|
||||
# then gets `files_per_leaf` rows.
|
||||
#
|
||||
# Examples (folders per subtree):
|
||||
# depth=4 fanout=3 → 121 (seed ≈ 1 s)
|
||||
# depth=5 fanout=4 → 1 365 (seed ≈ 10 s) ← default
|
||||
# depth=6 fanout=3 → 1 093 (seed ≈ 10 s)
|
||||
# depth=8 fanout=3 → 9 841 (seed ≈ 30 s)
|
||||
# depth=8 fanout=5 → 488 281 (seed ≈ several minutes — exponential blow-up)
|
||||
#
|
||||
# Override per run with LOAD_DEPTH / LOAD_FANOUT / LOAD_FILES_PER_LEAF.
|
||||
|
||||
load_depth=10
|
||||
load_fanout=3
|
||||
|
||||
load_files_per_leaf=3
|
||||
load_extra_users=20
|
||||
load_group_depth=3
|
||||
load_group_fanout=5
|
||||
Reference in New Issue
Block a user