Files
Oxicloud/Cargo.toml
T
Claude 79b94126be perf(authz): round 8 — cache the File/Folder grant-cascade decision for shared-album thumbnails
get_thumbnail_impl runs require_permission(Read) on every request. For a drive
member that's a drive_role_cache hit, but a shared-album recipient — granted a
folder (the album), not drive membership — fails the drive-role precheck and
falls through to file_cascade_grant_exists (a role_grants ⋈ folders lpath
ancestor query), once per file. Browsers revalidate immutable thumbnails
constantly, so the same (recipient, file, Read) decision was recomputed on
every thumbnail of every view — ~100 grant queries per 100-photo album per
navigate-away-and-back.

New cascade_grant_cache ((Subject, Resource, Permission) → bool, 30 s TTL)
memoises that decision. The check is NEVER skipped — the ordering is unchanged,
authz still runs on every request; only the result is cached, and only after
the drive-role precheck fails (so a later drive grant can't be shadowed by a
stale entry). Invalidation mirrors drive_role_cache's convention: explicit
invalidate_all on every File/Folder set_role/clear_role (immediate revoke on
the direct share path), 30 s TTL for the indirect paths (group membership,
moves, expiry) "rather than a deep invalidation tree".

Bench (bench_thumbnail_cascade_cache) with hard safety gates — recipient
allowed, outsider denied, and a clear_role revoke denies the very next check
(proving the grant-write flush): 100-photo album revalidation 2576 → 2.70
µs/thumb (~950x), 257.6 → 0.27 ms/view. Validated against the full
--cfg integration_tests authz suite (554 tests) + 524 workspace tests, clippy
-D warnings clean.

Deliberately not done: moving authz after the 304/cache short-circuit (a
security-posture change — a revoked user could serve cached thumbnails). With
the decision cached, the authz on the 304 path is now a memory hit, so the
"zero DB work on a 304" intent is restored without weakening the check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-18 14:03:43 +00:00

495 lines
20 KiB
TOML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
[package]
name = "oxicloud"
version = "0.8.2"
edition = "2024"
default-run = "oxicloud"
[dependencies]
mimalloc = { version = "0.1.52", default-features = false }
axum = { version = "0.8.9", features = ["multipart", "http1", "http2", "tokio", "macros"] }
# "process" was previously enabled implicitly through aws-config's feature
# unification; ffmpeg_video_frame_service needs it, so declare it ourselves.
tokio = { version = "1.52.3", features = ["rt-multi-thread", "macros", "io-util", "net", "time", "sync", "fs", "process"] }
tokio-util = { version = "0.7.18", features = ["io", "codec", "compat"] }
tokio-stream = { version = "0.1.18", features = ["fs", "sync"] }
bytes = "1.11.1"
tempfile = "3.27.0"
tower = "0.5.3"
tower-http = { version = "0.6.11", features = ["fs", "compression-gzip", "compression-br", "trace", "cors", "add-extension", "request-id", "set-header", "limit"] }
flate2 = "1.1.9"
tracing = "0.1.44"
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
chrono = { version = "0.4.45", features = ["serde"] }
# RFC 5545 iCalendar parser + emitter.
#
# Adopted 2026-07-14 to replace the hand-rolled property-scan in
# `src/domain/entities/calendar_event.rs::extract_ical_property`,
# which used a naive `format!("\n{}:", name)` substring search and
# refused ANY property carrying parameters (`DTSTART;VALUE=DATE:...`,
# `RECURRENCE-ID;VALUE=DATE:...`, `ATTENDEE;CN=…;PARTSTAT=…:mailto:…`).
# That broke all-day events and made the domain unaware of exception
# instances (see AtalayaLabs/OxiCloud#528).
#
# The crate is the widely-used Rust parser (~1500 SLOC, MIT/Apache),
# actively maintained by @Peltoche as `ical-rs` on GitHub. It handles
# line-folding, escaped characters, parameter maps, and every standard
# component. If a spec conformance gap is found, we contribute upstream.
ical = "0.11"
http-body = "1.0.1"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.150"
futures = "0.3.32"
async-stream = "0.3.6"
async-trait = "0.1.89"
mime_guess = "2.0.5"
uuid = { version = "1.23.3", features = ["v4", "v7", "serde"] }
thiserror = "2.0.18"
mockall = { version = "0.14.0", optional = true }
sqlx = { version = "0.8.6", features = ["postgres", "runtime-tokio", "tls-rustls", "chrono", "uuid", "json", "migrate"] }
jsonwebtoken = { version = "10.4.0", features = ["rust_crypto"] }
argon2 = "0.5.3"
rand_core = { version = "0.6", features = ["std", "getrandom"] }
quick-xml = "0.41.0"
dotenvy = "0.15.7"
moka = { version = "0.12.15", features = ["future", "sync"] }
http-range-header = "0.4"
image = { version = "0.25.10", default-features = false, features = ["jpeg", "png", "gif", "webp"] }
# Shrink-on-load JPEG decode (DCT 1/2·1/4·1/8 scaling) for thumbnails — pure
# Rust, no C toolchain. The `image` crate's zune-jpeg backend can't scale during
# decode; this can, cutting decode time/RAM ~order-of-magnitude on large photos.
jpeg-decoder = "0.3"
# SIMD (AVX2/SSE4.1/NEON) image resizing for thumbnails — far faster than the
# `image` crate's scalar resampler, and it speeds every format (incl. PNG/WebP).
fast_image_resize = "5"
# Lossy WebP encoding for thumbnails. The `image` crate's webp feature is
# lossless-only; this binds libwebp (vendored + built via `cc`, no system dep)
# for quality-controlled lossy output (~25-30% smaller than JPEG at equal SSIM).
webp = "0.3"
id3 = "1.17"
mp3-duration = "0.1"
kamadak-exif = "0.6.1"
md-5 = "0.11.0"
sha2 = "0.11.0"
unicode-normalization = "0.1.25"
blake3 = { version = "1.8.5", features = ["rayon", "mmap"] }
hex = "0.4.3"
http-body-util = "0.1.3"
percent-encoding = "2.3.2"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots","rustls-tls-native-roots"] }
base64 = "0.22.1"
fs2 = "0.4"
rayon = "1.12.0"
infer = "0.19"
async-compression = { version = "0.4.42", features = ["tokio", "gzip"] }
async_zip = { version = "0.0.18", features = ["tokio", "deflate"] }
dashmap = "6.2.1"
socket2 = { version = "0.6.4", features = ["all"] }
urlencoding = "2.1.3"
utoipa = { version = "5.5.0", features = ["axum_extras", "uuid", "chrono"] }
# NOTE: aws-config and aws-smithy-types were removed as direct deps in the
# round-3 perf pass — S3BlobBackend builds its client purely from
# aws_sdk_s3::config with static credentials; nothing referenced either
# crate, and aws-config alone pulled aws-sdk-sso/ssooidc/sts (~90 crates)
# into every build (benches/ROUND3.md).
aws-sdk-s3 = "1.136.0"
azure_core = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage_blobs = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
aes-gcm = "0.10.3"
lru = "0.16.4"
fastcdc = { version = "4.0.0", features = ["tokio"] }
lettre = { version = "0.11.18", default-features = false, features = ["smtp-transport", "tokio1-rustls-tls", "rustls-native-certs", "builder"] }
idna = "1.1"
smol_str = { version = "0.3.6", features = ["serde"] }
accept-language = "3.1.0"
askama = "0.16.0"
tantivy = "0.26.1"
zip = { version = "8.6.0", default-features = false, features = ["deflate"] }
pdf-extract = "0.12.0"
nom-exif = "3.6.1"
extism = { version = "1.30.0", optional = true }
toml = { version = "1.1.2", optional = true }
file-rotate = { version = "0.7.6", optional = true }
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "tracing", "api-24"], optional = true }
ndarray = { version = "0.17.2", optional = true }
[features]
default = []
test_utils = ["mockall"]
integration_tests = []
# WASM plugin runtime (Extism). Opt-in: bundles wasmtime, a large engine most
# deployments won't use. Activation also requires OXICLOUD_ENABLE_PLUGINS=true.
plugins = ["dep:extism", "dep:toml", "dep:file-rotate"]
# Empty marker feature that gates the `load-seed` binary so it isn't built
# by default (and skipped in prod Docker builds). Kept separate from
# test_utils so enabling it doesn't change the oxicloud dependency graph —
# this lets one `cargo build` produce both `oxicloud` and `load-seed`
# without recompiling oxicloud with mockall in scope.
load_seed_bin = []
# Real ONNX-backed face analyzer (detector + embedder) for the People feature.
# Opt-in: pulls `ort` (ONNX Runtime, load-dynamic — dlopen's libonnxruntime at
# runtime) + `ndarray`, a heavy stack most deployments won't use. Activation also
# requires OXICLOUD_ENABLE_FACES=true *and* operator-provided ONNX models; without
# this feature the People pipeline falls back to the inert NoopFaceAnalyzer.
faces-onnx = ["dep:ort", "dep:ndarray"]
# Performance benchmark harness (Phase 0). Exposes `bench_support` + thin public
# wrappers over the private thumbnail render functions so `benches/` and
# `examples/` can measure them. Off by default — adds nothing to prod builds.
# Run with: `cargo bench --features bench` / `cargo run --release --features bench --example bench_thumbnails_mem`.
bench = []
[dev-dependencies]
criterion = "0.5"
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(integration_tests)'] }
[[bin]]
name = "generate-openapi"
path = "src/bin/generate-openapi.rs"
[[bin]]
name = "migrate-nfc-filenames"
path = "src/bin/migrate-nfc-filenames.rs"
[[bin]]
name = "load-seed"
path = "src/bin/load-seed.rs"
# Test fixture seeder. Gated behind the empty `load_seed_bin` feature so
# `cargo build --release` (and the prod Dockerfile) skip it. tests/load/run.sh
# and load-nightly.yml build it explicitly with --features load_seed_bin.
required-features = ["load_seed_bin"]
# Phase 0 perf harness — Task 0.2 (criterion latency + output-size bench).
[[bench]]
name = "thumbnails"
path = "benches/thumbnails.rs"
harness = false
required-features = ["bench"]
# Phase 0 perf harness — Task 0.3 (peak-RAM + saturated-throughput baseline).
[[example]]
name = "bench_thumbnails_mem"
path = "examples/bench_thumbnails_mem.rs"
required-features = ["bench"]
# ZIP entry-compression benchmark — Deflate-always vs MIME-aware Stored for
# already-compressed media on the folder/batch ZIP download path. No Postgres.
[[example]]
name = "bench_zip_media"
path = "examples/bench_zip_media.rs"
required-features = ["bench"]
# WebDAV dead-properties fetch benchmark — PROPFIND's per-child N+1 (with a
# non-indexable IS NOT DISTINCT FROM predicate) vs batched = ANY($1) per page
# (needs the dev Postgres up).
[[example]]
name = "bench_dead_props"
path = "examples/bench_dead_props.rs"
required-features = ["bench"]
# NC chroot / default-drive resolution benchmark — the middleware's 2 uncached
# queries per request vs the moka caches (needs the dev Postgres up).
[[example]]
name = "bench_chroot_cache"
path = "examples/bench_chroot_cache.rs"
required-features = ["bench"]
# Quota-path benchmark — full auth.users row (incl. 512 KiB avatar) vs the
# narrow 2-column read, on every upload check / quota PROPFIND (needs Postgres).
[[example]]
name = "bench_quota_path"
path = "examples/bench_quota_path.rs"
required-features = ["bench"]
# People-tab benchmark — full faces scan (2 KiB embedding per row) vs grouped
# COUNT + batched cover lookup (needs Postgres).
[[example]]
name = "bench_people_list"
path = "examples/bench_people_list.rs"
required-features = ["bench"]
# PROPFIND folder-paging benchmark — LIMIT/OFFSET full-folder rescan per page
# vs keyset + (folder_id, name) index (needs Postgres).
[[example]]
name = "bench_propfind_paging"
path = "examples/bench_propfind_paging.rs"
required-features = ["bench"]
# Static-asset compression benchmark — per-request Brotli vs precompressed
# sibling read. No Postgres.
[[example]]
name = "bench_static_precompress"
path = "examples/bench_static_precompress.rs"
required-features = ["bench"]
# Round-2 battery: range-from-cache, NC chunk gate, delta prefetch, ingest
# overlap (real store_from_stream; run with OXICLOUD_INGEST_OVERLAP=0/1),
# ZIP streaming TTFB. Sections 1 and 4 need Postgres.
[[example]]
name = "bench_round2"
path = "examples/bench_round2.rs"
required-features = ["bench"]
# Video thumbnail benchmark — Option B (server-side ffmpeg frame → WebP). Needs
# `ffmpeg` on PATH (libx264/libx265/libvpx-vp9 to synthesize the test corpus).
[[example]]
name = "bench_video_thumbnails"
path = "examples/bench_video_thumbnails.rs"
required-features = ["bench"]
# DB connection-pool tail-latency benchmark (needs the dev Postgres up).
[[example]]
name = "bench_db_pool"
path = "examples/bench_db_pool.rs"
required-features = ["bench"]
# Blob-manifest read round-trip benchmark: OLD 2 queries vs NEW 1 (needs dev Postgres).
[[example]]
name = "bench_blob_manifest"
path = "examples/bench_blob_manifest.rs"
required-features = ["bench"]
# Blob download read-ahead benchmark — sweeps the local backend's chunk prefetch
# depth (read_prefetch / buffered(N)) under disk-bound vs network-bound consumers
# and warm vs cold page cache. No Postgres needed.
[[example]]
name = "bench_blob_prefetch"
path = "examples/bench_blob_prefetch.rs"
required-features = ["bench"]
# Tokio runtime tuning benchmark — worker over-subscription (throughput + p99
# under CPU contention) and blocking-pool RSS blast radius, default vs tuned.
# No Postgres needed; run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_tokio_runtime"
path = "examples/bench_tokio_runtime.rs"
required-features = ["bench"]
# CPU pool concurrency benchmark — thumbnail decode throughput + p99 as the
# decode-permit count is swept, showing the effect of sizing the image pools to
# the CFS quota (effective_parallelism) vs the host core count. No Postgres;
# run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_pool_concurrency"
path = "examples/bench_pool_concurrency.rs"
required-features = ["bench"]
# ACL owner-cache benchmark — owner query vs moka hit (needs the dev Postgres up).
[[example]]
name = "bench_owner_cache"
path = "examples/bench_owner_cache.rs"
required-features = ["bench"]
# Round-4 battery ─────────────────────────────────────────────────────────────
# PG row → entity path materialization — the per-listing-row make_file_path
# split→rejoin + NFC copy chain vs the one-pass builders. No Postgres.
[[example]]
name = "bench_row_path"
path = "examples/bench_row_path.rs"
required-features = ["bench"]
# WebDAV drive-selector resolution — the per-request list_readable_by grants
# join vs the per-user readable_cache (needs the dev Postgres up).
[[example]]
name = "bench_drive_selector"
path = "examples/bench_drive_selector.rs"
required-features = ["bench"]
# CalDAV parse path — from_ical's 8×-reparse vs single parse, per-event
# uppercase copies on REPORT/GET, UID clone churn. No Postgres.
[[example]]
name = "bench_caldav_parse"
path = "examples/bench_caldav_parse.rs"
required-features = ["bench"]
# PROPFIND per-row XML emit — partition Vec churn + chrono format-interpreter
# dates vs single-pass + stack-rendered fields. No Postgres.
[[example]]
name = "bench_propfind_xml"
path = "examples/bench_propfind_xml.rs"
required-features = ["bench"]
# Grant-listing hydration N+1 (calendars / address books / playlists) +
# user-flags cold-cache herd (needs the dev Postgres up).
[[example]]
name = "bench_n1_hydration"
path = "examples/bench_n1_hydration.rs"
required-features = ["bench"]
# Face-indexing fan-out — unbounded per-image spawn vs core-count semaphore;
# peak-live-heap + wall on the bench_support photo corpus. No Postgres.
[[example]]
name = "bench_faces_bound"
path = "examples/bench_faces_bound.rs"
required-features = ["bench"]
# Azure download path — whole-blob collect vs streamed pages, TTFB + peak
# live heap against a local Azure-GET stub (endpoint_url hook). No Postgres.
[[example]]
name = "bench_azure_stream"
path = "examples/bench_azure_stream.rs"
required-features = ["bench"]
# Round-5 battery ─────────────────────────────────────────────────────────────
# CalDAV whole-calendar REPORT/GET — buffered double-residency vs uid-keyset
# streaming; TTFB + peak live heap (needs the dev Postgres up).
[[example]]
name = "bench_caldav_stream"
path = "examples/bench_caldav_stream.rs"
required-features = ["bench"]
# Round-5 micro-allocation pack — suggest clones, readable-cache Arc hit,
# SPA-listing interning, NC href prefix, CardDAV REPORT churn. No Postgres.
[[example]]
name = "bench_micro_allocs"
path = "examples/bench_micro_allocs.rs"
required-features = ["bench"]
# Round-8 battery ─────────────────────────────────────────────────────────────
# Shared-album thumbnail authz — folder-grant cascade query per thumbnail vs
# the cascade_grant_cache; includes a revocation safety gate (needs the dev
# Postgres up).
[[example]]
name = "bench_thumbnail_cascade_cache"
path = "examples/bench_thumbnail_cascade_cache.rs"
required-features = ["bench"]
# Round-7 battery ─────────────────────────────────────────────────────────────
# Range-seek per-request authz duplication — the per-seek require the range
# branch used to run (warm CPU + cold drive-resolve query) vs 0 after routing
# through the non-perms range read (needs the dev Postgres up).
[[example]]
name = "bench_range_seek_authz"
path = "examples/bench_range_seek_authz.rs"
required-features = ["bench"]
# `/api/folders/{id}/resources` row→DTO mapping — per-row name clone vs move
# (pure CPU; counting allocator).
[[example]]
name = "bench_resource_row_map"
path = "examples/bench_resource_row_map.rs"
required-features = ["bench"]
# Round-6 battery ─────────────────────────────────────────────────────────────
# CardDAV whole-book REPORT/PROPFIND — buffered double-residency vs cursor
# streaming; TTFB + peak live heap (needs the dev Postgres up).
[[example]]
name = "bench_carddav_stream"
path = "examples/bench_carddav_stream.rs"
required-features = ["bench"]
# Batch-favorites authz pre-check — serial require loop vs try_join_all
# against the real PgAclEngine (needs the dev Postgres up).
[[example]]
name = "bench_favorites_authz"
path = "examples/bench_favorites_authz.rs"
required-features = ["bench"]
# Digest-hex rendering + NC id-batch marshalling micro-allocs (pure CPU).
[[example]]
name = "bench_hex_ids"
path = "examples/bench_hex_ids.rs"
required-features = ["bench"]
# `id::text` server cast vs binary UUID decode + app-side formatting A/B
# (needs the dev Postgres up).
[[example]]
name = "bench_uuid_text_cast"
path = "examples/bench_uuid_text_cast.rs"
required-features = ["bench"]
# Round-3 battery ─────────────────────────────────────────────────────────────
# Web-UI folder listing — whole-folder rescan + top-N sort per page vs keyset
# pushdown into the UNION-ALL branches + (folder_id, LOWER(name), id) indexes
# (needs the dev Postgres up).
[[example]]
name = "bench_listing_keyset"
path = "examples/bench_listing_keyset.rs"
required-features = ["bench"]
# Photos timeline — full-library scan + top-N above the grants join vs
# per-drive LATERAL top-N on the media-timeline index (needs Postgres).
[[example]]
name = "bench_photos_timeline"
path = "examples/bench_photos_timeline.rs"
required-features = ["bench"]
# PROPFIND subfolder paging — LIMIT/OFFSET + COUNT(*) OVER() per page vs
# keyset batch, mirroring the files-side PROPFIND-PAGING fix (needs Postgres).
[[example]]
name = "bench_folder_keyset"
path = "examples/bench_folder_keyset.rs"
required-features = ["bench"]
# Basic-auth thundering herd — K concurrent cache misses each paying Argon2id
# vs single-flight try_get_with (needs Postgres).
[[example]]
name = "bench_auth_herd"
path = "examples/bench_auth_herd.rs"
required-features = ["bench"]
# CachedBlobBackend — miss stampede (N duplicate remote fetches racing on one
# .tmp) vs per-hash single-flight; warm-hit index throughput. No Postgres.
[[example]]
name = "bench_blob_cache"
path = "examples/bench_blob_cache.rs"
required-features = ["bench"]
# Upload spool/assembly I/O — ReaderStream capacity sweep on part-file reads
# and BufWriter vs bare-File frame writes on the chunk spool path. No Postgres.
[[example]]
name = "bench_upload_spool"
path = "examples/bench_upload_spool.rs"
required-features = ["bench"]
# S3 chunk PUT — HEAD-before-PUT vs unconditional PUT against a local axum
# stub with injected latency; Azure Bytes-vs-to_vec copy micro. No Postgres.
[[example]]
name = "bench_s3_put"
path = "examples/bench_s3_put.rs"
required-features = ["bench"]
# File/Folder -> DTO mapping allocations — Arc<str> interning of closed-set
# display fields, 1-alloc etag/size formatting. No Postgres.
[[example]]
name = "bench_dto_map"
path = "examples/bench_dto_map.rs"
required-features = ["bench"]
# CardDAV REPORT — dead per-contact vCard pre-generation + O(N^2) uid scan vs
# single on-demand generation. No Postgres.
[[example]]
name = "bench_carddav_report"
path = "examples/bench_carddav_report.rs"
required-features = ["bench"]
# Search-results cache RSS — entry-count capacity vs byte weigher. No Postgres.
[[example]]
name = "bench_search_cache_mem"
path = "examples/bench_search_cache_mem.rs"
required-features = ["bench"]
[profile.release]
lto = "thin"
codegen-units = 1
opt-level = 3
strip = true
[profile.dev]
opt-level = 1
debug = true
[profile.bench]
lto = "fat"
codegen-units = 1
opt-level = 3