Files
Oxicloud/Cargo.toml
T
Claude 63cf6646d0 perf: round 5 — CalDAV cursor streaming, SPA interning gaps, NC href prefix, per-request micro-allocs
Seven benchmark-gated changes (benches/ROUND5.md; BEFORE/AFTER bench +
equivalence gate each, rollback rule as ROUND2-4 — two intermediate
CalDAV shapes measured worse and were themselves rolled back before
shipping):

- CalDAV whole-calendar responses (REPORT no-range/sync-collection,
  depth-1 collection PROPFIND, .ics GET): buffered double-residency →
  ONE window-ordered scan (MIN(start_time) OVER (PARTITION BY ical_uid))
  streamed through a PG cursor, pages cut at UID boundaries. TTFB
  23.3→11.0 ms (2.1x), peak heap 14.2→8.0 MiB at 4k events / 45→24 MiB
  at 12k, wall +9-15% (documented trade, ZIP-streaming class); both
  multistatus and ICS byte-identical to the buffered output. Rejected
  shapes kept in the doc: per-page GROUP-BY keyset (3-4x wall) and
  per-uid ANY hydration (~20 µs/index descent).
- SPA listing interning gaps: folder/recent/favorites resources handlers
  (and the WebDAV pseudo-root) called raw Arc::from per row for the
  closed display set ROUND3 interned — now intern_display/intern_mime,
  4→0 allocs/row, byte-identical Arc contents.
- NC PROPFIND child hrefs: username + parent path encoded once per
  request instead of per child (543→165 ns/row, 13→4 allocs); native
  WebDAV href drops its intermediate encode String.
- suggest enrichment: entity clone + field re-clones per keystroke row →
  consume + move (166.5→126.8 µs/200 rows, 20→7 allocs/row).
- list_readable_by returns the cache's Arc (246→128 ns warm hit, 4→0
  allocs) — deep Vec clone per DAV-selector request removed.
- CardDAV REPORT: borrowed props, reused href buffer, exact-size etag
  quoting (3.04→2.34 ms per 5k-contact getetag poll).
- Auth span records: user_id.to_string() per request ×3 →
  tracing::field::display.

Checks: cargo fmt, clippy --all-features --all-targets -D warnings,
cargo test --workspace (523 passed). Follow-ups (CardDAV streaming,
&[&str] id batches, ::text UUID casts A/B, share-landing join) recorded
in benches/ROUND5.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-17 15:19:00 +00:00

439 lines
17 KiB
TOML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
[package]
name = "oxicloud"
version = "0.8.2"
edition = "2024"
default-run = "oxicloud"
[dependencies]
mimalloc = { version = "0.1.52", default-features = false }
axum = { version = "0.8.9", features = ["multipart", "http1", "http2", "tokio", "macros"] }
# "process" was previously enabled implicitly through aws-config's feature
# unification; ffmpeg_video_frame_service needs it, so declare it ourselves.
tokio = { version = "1.52.3", features = ["rt-multi-thread", "macros", "io-util", "net", "time", "sync", "fs", "process"] }
tokio-util = { version = "0.7.18", features = ["io", "codec", "compat"] }
tokio-stream = { version = "0.1.18", features = ["fs", "sync"] }
bytes = "1.11.1"
tempfile = "3.27.0"
tower = "0.5.3"
tower-http = { version = "0.6.11", features = ["fs", "compression-gzip", "compression-br", "trace", "cors", "add-extension", "request-id", "set-header", "limit"] }
flate2 = "1.1.9"
tracing = "0.1.44"
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
chrono = { version = "0.4.45", features = ["serde"] }
# RFC 5545 iCalendar parser + emitter.
#
# Adopted 2026-07-14 to replace the hand-rolled property-scan in
# `src/domain/entities/calendar_event.rs::extract_ical_property`,
# which used a naive `format!("\n{}:", name)` substring search and
# refused ANY property carrying parameters (`DTSTART;VALUE=DATE:...`,
# `RECURRENCE-ID;VALUE=DATE:...`, `ATTENDEE;CN=…;PARTSTAT=…:mailto:…`).
# That broke all-day events and made the domain unaware of exception
# instances (see AtalayaLabs/OxiCloud#528).
#
# The crate is the widely-used Rust parser (~1500 SLOC, MIT/Apache),
# actively maintained by @Peltoche as `ical-rs` on GitHub. It handles
# line-folding, escaped characters, parameter maps, and every standard
# component. If a spec conformance gap is found, we contribute upstream.
ical = "0.11"
http-body = "1.0.1"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.150"
futures = "0.3.32"
async-stream = "0.3.6"
async-trait = "0.1.89"
mime_guess = "2.0.5"
uuid = { version = "1.23.3", features = ["v4", "v7", "serde"] }
thiserror = "2.0.18"
mockall = { version = "0.14.0", optional = true }
sqlx = { version = "0.8.6", features = ["postgres", "runtime-tokio", "tls-rustls", "chrono", "uuid", "json", "migrate"] }
jsonwebtoken = { version = "10.4.0", features = ["rust_crypto"] }
argon2 = "0.5.3"
rand_core = { version = "0.6", features = ["std", "getrandom"] }
quick-xml = "0.41.0"
dotenvy = "0.15.7"
moka = { version = "0.12.15", features = ["future", "sync"] }
http-range-header = "0.4"
image = { version = "0.25.10", default-features = false, features = ["jpeg", "png", "gif", "webp"] }
# Shrink-on-load JPEG decode (DCT 1/2·1/4·1/8 scaling) for thumbnails — pure
# Rust, no C toolchain. The `image` crate's zune-jpeg backend can't scale during
# decode; this can, cutting decode time/RAM ~order-of-magnitude on large photos.
jpeg-decoder = "0.3"
# SIMD (AVX2/SSE4.1/NEON) image resizing for thumbnails — far faster than the
# `image` crate's scalar resampler, and it speeds every format (incl. PNG/WebP).
fast_image_resize = "5"
# Lossy WebP encoding for thumbnails. The `image` crate's webp feature is
# lossless-only; this binds libwebp (vendored + built via `cc`, no system dep)
# for quality-controlled lossy output (~25-30% smaller than JPEG at equal SSIM).
webp = "0.3"
id3 = "1.17"
mp3-duration = "0.1"
kamadak-exif = "0.6.1"
md-5 = "0.11.0"
sha2 = "0.11.0"
unicode-normalization = "0.1.25"
blake3 = { version = "1.8.5", features = ["rayon", "mmap"] }
hex = "0.4.3"
http-body-util = "0.1.3"
percent-encoding = "2.3.2"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots","rustls-tls-native-roots"] }
base64 = "0.22.1"
fs2 = "0.4"
rayon = "1.12.0"
infer = "0.19"
async-compression = { version = "0.4.42", features = ["tokio", "gzip"] }
async_zip = { version = "0.0.18", features = ["tokio", "deflate"] }
dashmap = "6.2.1"
socket2 = { version = "0.6.4", features = ["all"] }
urlencoding = "2.1.3"
utoipa = { version = "5.5.0", features = ["axum_extras", "uuid", "chrono"] }
# NOTE: aws-config and aws-smithy-types were removed as direct deps in the
# round-3 perf pass — S3BlobBackend builds its client purely from
# aws_sdk_s3::config with static credentials; nothing referenced either
# crate, and aws-config alone pulled aws-sdk-sso/ssooidc/sts (~90 crates)
# into every build (benches/ROUND3.md).
aws-sdk-s3 = "1.136.0"
azure_core = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage_blobs = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
aes-gcm = "0.10.3"
lru = "0.16.4"
fastcdc = { version = "4.0.0", features = ["tokio"] }
lettre = { version = "0.11.18", default-features = false, features = ["smtp-transport", "tokio1-rustls-tls", "rustls-native-certs", "builder"] }
idna = "1.1"
smol_str = { version = "0.3.6", features = ["serde"] }
accept-language = "3.1.0"
askama = "0.16.0"
tantivy = "0.26.1"
zip = { version = "8.6.0", default-features = false, features = ["deflate"] }
pdf-extract = "0.12.0"
nom-exif = "3.6.1"
extism = { version = "1.30.0", optional = true }
toml = { version = "1.1.2", optional = true }
file-rotate = { version = "0.7.6", optional = true }
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "tracing", "api-24"], optional = true }
ndarray = { version = "0.17.2", optional = true }
[features]
default = []
test_utils = ["mockall"]
integration_tests = []
# WASM plugin runtime (Extism). Opt-in: bundles wasmtime, a large engine most
# deployments won't use. Activation also requires OXICLOUD_ENABLE_PLUGINS=true.
plugins = ["dep:extism", "dep:toml", "dep:file-rotate"]
# Empty marker feature that gates the `load-seed` binary so it isn't built
# by default (and skipped in prod Docker builds). Kept separate from
# test_utils so enabling it doesn't change the oxicloud dependency graph —
# this lets one `cargo build` produce both `oxicloud` and `load-seed`
# without recompiling oxicloud with mockall in scope.
load_seed_bin = []
# Real ONNX-backed face analyzer (detector + embedder) for the People feature.
# Opt-in: pulls `ort` (ONNX Runtime, load-dynamic — dlopen's libonnxruntime at
# runtime) + `ndarray`, a heavy stack most deployments won't use. Activation also
# requires OXICLOUD_ENABLE_FACES=true *and* operator-provided ONNX models; without
# this feature the People pipeline falls back to the inert NoopFaceAnalyzer.
faces-onnx = ["dep:ort", "dep:ndarray"]
# Performance benchmark harness (Phase 0). Exposes `bench_support` + thin public
# wrappers over the private thumbnail render functions so `benches/` and
# `examples/` can measure them. Off by default — adds nothing to prod builds.
# Run with: `cargo bench --features bench` / `cargo run --release --features bench --example bench_thumbnails_mem`.
bench = []
[dev-dependencies]
criterion = "0.5"
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(integration_tests)'] }
[[bin]]
name = "generate-openapi"
path = "src/bin/generate-openapi.rs"
[[bin]]
name = "migrate-nfc-filenames"
path = "src/bin/migrate-nfc-filenames.rs"
[[bin]]
name = "load-seed"
path = "src/bin/load-seed.rs"
# Test fixture seeder. Gated behind the empty `load_seed_bin` feature so
# `cargo build --release` (and the prod Dockerfile) skip it. tests/load/run.sh
# and load-nightly.yml build it explicitly with --features load_seed_bin.
required-features = ["load_seed_bin"]
# Phase 0 perf harness — Task 0.2 (criterion latency + output-size bench).
[[bench]]
name = "thumbnails"
path = "benches/thumbnails.rs"
harness = false
required-features = ["bench"]
# Phase 0 perf harness — Task 0.3 (peak-RAM + saturated-throughput baseline).
[[example]]
name = "bench_thumbnails_mem"
path = "examples/bench_thumbnails_mem.rs"
required-features = ["bench"]
# ZIP entry-compression benchmark — Deflate-always vs MIME-aware Stored for
# already-compressed media on the folder/batch ZIP download path. No Postgres.
[[example]]
name = "bench_zip_media"
path = "examples/bench_zip_media.rs"
required-features = ["bench"]
# WebDAV dead-properties fetch benchmark — PROPFIND's per-child N+1 (with a
# non-indexable IS NOT DISTINCT FROM predicate) vs batched = ANY($1) per page
# (needs the dev Postgres up).
[[example]]
name = "bench_dead_props"
path = "examples/bench_dead_props.rs"
required-features = ["bench"]
# NC chroot / default-drive resolution benchmark — the middleware's 2 uncached
# queries per request vs the moka caches (needs the dev Postgres up).
[[example]]
name = "bench_chroot_cache"
path = "examples/bench_chroot_cache.rs"
required-features = ["bench"]
# Quota-path benchmark — full auth.users row (incl. 512 KiB avatar) vs the
# narrow 2-column read, on every upload check / quota PROPFIND (needs Postgres).
[[example]]
name = "bench_quota_path"
path = "examples/bench_quota_path.rs"
required-features = ["bench"]
# People-tab benchmark — full faces scan (2 KiB embedding per row) vs grouped
# COUNT + batched cover lookup (needs Postgres).
[[example]]
name = "bench_people_list"
path = "examples/bench_people_list.rs"
required-features = ["bench"]
# PROPFIND folder-paging benchmark — LIMIT/OFFSET full-folder rescan per page
# vs keyset + (folder_id, name) index (needs Postgres).
[[example]]
name = "bench_propfind_paging"
path = "examples/bench_propfind_paging.rs"
required-features = ["bench"]
# Static-asset compression benchmark — per-request Brotli vs precompressed
# sibling read. No Postgres.
[[example]]
name = "bench_static_precompress"
path = "examples/bench_static_precompress.rs"
required-features = ["bench"]
# Round-2 battery: range-from-cache, NC chunk gate, delta prefetch, ingest
# overlap (real store_from_stream; run with OXICLOUD_INGEST_OVERLAP=0/1),
# ZIP streaming TTFB. Sections 1 and 4 need Postgres.
[[example]]
name = "bench_round2"
path = "examples/bench_round2.rs"
required-features = ["bench"]
# Video thumbnail benchmark — Option B (server-side ffmpeg frame → WebP). Needs
# `ffmpeg` on PATH (libx264/libx265/libvpx-vp9 to synthesize the test corpus).
[[example]]
name = "bench_video_thumbnails"
path = "examples/bench_video_thumbnails.rs"
required-features = ["bench"]
# DB connection-pool tail-latency benchmark (needs the dev Postgres up).
[[example]]
name = "bench_db_pool"
path = "examples/bench_db_pool.rs"
required-features = ["bench"]
# Blob-manifest read round-trip benchmark: OLD 2 queries vs NEW 1 (needs dev Postgres).
[[example]]
name = "bench_blob_manifest"
path = "examples/bench_blob_manifest.rs"
required-features = ["bench"]
# Blob download read-ahead benchmark — sweeps the local backend's chunk prefetch
# depth (read_prefetch / buffered(N)) under disk-bound vs network-bound consumers
# and warm vs cold page cache. No Postgres needed.
[[example]]
name = "bench_blob_prefetch"
path = "examples/bench_blob_prefetch.rs"
required-features = ["bench"]
# Tokio runtime tuning benchmark — worker over-subscription (throughput + p99
# under CPU contention) and blocking-pool RSS blast radius, default vs tuned.
# No Postgres needed; run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_tokio_runtime"
path = "examples/bench_tokio_runtime.rs"
required-features = ["bench"]
# CPU pool concurrency benchmark — thumbnail decode throughput + p99 as the
# decode-permit count is swept, showing the effect of sizing the image pools to
# the CFS quota (effective_parallelism) vs the host core count. No Postgres;
# run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_pool_concurrency"
path = "examples/bench_pool_concurrency.rs"
required-features = ["bench"]
# ACL owner-cache benchmark — owner query vs moka hit (needs the dev Postgres up).
[[example]]
name = "bench_owner_cache"
path = "examples/bench_owner_cache.rs"
required-features = ["bench"]
# Round-4 battery ─────────────────────────────────────────────────────────────
# PG row → entity path materialization — the per-listing-row make_file_path
# split→rejoin + NFC copy chain vs the one-pass builders. No Postgres.
[[example]]
name = "bench_row_path"
path = "examples/bench_row_path.rs"
required-features = ["bench"]
# WebDAV drive-selector resolution — the per-request list_readable_by grants
# join vs the per-user readable_cache (needs the dev Postgres up).
[[example]]
name = "bench_drive_selector"
path = "examples/bench_drive_selector.rs"
required-features = ["bench"]
# CalDAV parse path — from_ical's 8×-reparse vs single parse, per-event
# uppercase copies on REPORT/GET, UID clone churn. No Postgres.
[[example]]
name = "bench_caldav_parse"
path = "examples/bench_caldav_parse.rs"
required-features = ["bench"]
# PROPFIND per-row XML emit — partition Vec churn + chrono format-interpreter
# dates vs single-pass + stack-rendered fields. No Postgres.
[[example]]
name = "bench_propfind_xml"
path = "examples/bench_propfind_xml.rs"
required-features = ["bench"]
# Grant-listing hydration N+1 (calendars / address books / playlists) +
# user-flags cold-cache herd (needs the dev Postgres up).
[[example]]
name = "bench_n1_hydration"
path = "examples/bench_n1_hydration.rs"
required-features = ["bench"]
# Face-indexing fan-out — unbounded per-image spawn vs core-count semaphore;
# peak-live-heap + wall on the bench_support photo corpus. No Postgres.
[[example]]
name = "bench_faces_bound"
path = "examples/bench_faces_bound.rs"
required-features = ["bench"]
# Azure download path — whole-blob collect vs streamed pages, TTFB + peak
# live heap against a local Azure-GET stub (endpoint_url hook). No Postgres.
[[example]]
name = "bench_azure_stream"
path = "examples/bench_azure_stream.rs"
required-features = ["bench"]
# Round-5 battery ─────────────────────────────────────────────────────────────
# CalDAV whole-calendar REPORT/GET — buffered double-residency vs uid-keyset
# streaming; TTFB + peak live heap (needs the dev Postgres up).
[[example]]
name = "bench_caldav_stream"
path = "examples/bench_caldav_stream.rs"
required-features = ["bench"]
# Round-5 micro-allocation pack — suggest clones, readable-cache Arc hit,
# SPA-listing interning, NC href prefix, CardDAV REPORT churn. No Postgres.
[[example]]
name = "bench_micro_allocs"
path = "examples/bench_micro_allocs.rs"
required-features = ["bench"]
# Round-3 battery ─────────────────────────────────────────────────────────────
# Web-UI folder listing — whole-folder rescan + top-N sort per page vs keyset
# pushdown into the UNION-ALL branches + (folder_id, LOWER(name), id) indexes
# (needs the dev Postgres up).
[[example]]
name = "bench_listing_keyset"
path = "examples/bench_listing_keyset.rs"
required-features = ["bench"]
# Photos timeline — full-library scan + top-N above the grants join vs
# per-drive LATERAL top-N on the media-timeline index (needs Postgres).
[[example]]
name = "bench_photos_timeline"
path = "examples/bench_photos_timeline.rs"
required-features = ["bench"]
# PROPFIND subfolder paging — LIMIT/OFFSET + COUNT(*) OVER() per page vs
# keyset batch, mirroring the files-side PROPFIND-PAGING fix (needs Postgres).
[[example]]
name = "bench_folder_keyset"
path = "examples/bench_folder_keyset.rs"
required-features = ["bench"]
# Basic-auth thundering herd — K concurrent cache misses each paying Argon2id
# vs single-flight try_get_with (needs Postgres).
[[example]]
name = "bench_auth_herd"
path = "examples/bench_auth_herd.rs"
required-features = ["bench"]
# CachedBlobBackend — miss stampede (N duplicate remote fetches racing on one
# .tmp) vs per-hash single-flight; warm-hit index throughput. No Postgres.
[[example]]
name = "bench_blob_cache"
path = "examples/bench_blob_cache.rs"
required-features = ["bench"]
# Upload spool/assembly I/O — ReaderStream capacity sweep on part-file reads
# and BufWriter vs bare-File frame writes on the chunk spool path. No Postgres.
[[example]]
name = "bench_upload_spool"
path = "examples/bench_upload_spool.rs"
required-features = ["bench"]
# S3 chunk PUT — HEAD-before-PUT vs unconditional PUT against a local axum
# stub with injected latency; Azure Bytes-vs-to_vec copy micro. No Postgres.
[[example]]
name = "bench_s3_put"
path = "examples/bench_s3_put.rs"
required-features = ["bench"]
# File/Folder -> DTO mapping allocations — Arc<str> interning of closed-set
# display fields, 1-alloc etag/size formatting. No Postgres.
[[example]]
name = "bench_dto_map"
path = "examples/bench_dto_map.rs"
required-features = ["bench"]
# CardDAV REPORT — dead per-contact vCard pre-generation + O(N^2) uid scan vs
# single on-demand generation. No Postgres.
[[example]]
name = "bench_carddav_report"
path = "examples/bench_carddav_report.rs"
required-features = ["bench"]
# Search-results cache RSS — entry-count capacity vs byte weigher. No Postgres.
[[example]]
name = "bench_search_cache_mem"
path = "examples/bench_search_cache_mem.rs"
required-features = ["bench"]
[profile.release]
lto = "thin"
codegen-units = 1
opt-level = 3
strip = true
[profile.dev]
opt-level = 1
debug = true
[profile.bench]
lto = "fat"
codegen-units = 1
opt-level = 3