Files
Oxicloud/Cargo.toml
T
Claude 9729f033b2 perf: round 6 backend — CardDAV cursor streaming, borrowed NC id chain, binary UUID decode, one-alloc hex
Benchmark-gated (equivalence + BEFORE/AFTER in examples/bench_*, results
and reproduce commands in benches/ROUND6.md):

- CardDAV whole-book REPORT + depth-1 PROPFIND stream through a PG
  cursor (stream_contacts_by_book, 500-contact pages) instead of
  materialising every vCard twice: 8 000 contacts TTFB 37.4 → 7.6 ms
  (4.9x), peak heap 19.0 → 7.0 MiB (2.7x), wall -23%; REPORT and
  PROPFIND byte-identical to the buffered writers.

- NC numeric-id chain fully borrowed: get_or_create_file_ids/folder_ids
  take &[&str] and return HashMap<Uuid, i64>; batch_resolve_ids callers
  (PROPFIND pages, REPORT, trashbin, OCS search) pass id slices and look
  up via nc_id_of. 2.006 → 0.006 allocs/child (334x), 1.53x wall per
  500-child page. batch_check_favorites binds &[&str] as text[].

- file_blob_read_repository listing SELECTs drop id::text/folder_id::text
  server casts: rows decode binary Uuid (16 vs 36 bytes on the wire) and
  render once in row_to_file. A/B on 500-row pages: 1.225 → 1.044 ms
  mean (1.17x), p95 1.686 → 1.345 (bench_uuid_text_cast; single-row,
  param and min() sites left as-is deliberately).

- IncrementalHasher::finalize_hex renders through common::fmt::hex_lower
  instead of one format! per digest byte: 18 → 1 (md5) / 35 → 1 (sha256)
  allocs per chunk finalize, 14-15x wall.

- Share landing overlaps the access-count UPDATE with the unlock fetch
  via tokio::join! (one round-trip off every public link hit).

- REJECTED by benchmark and reverted: try_join_all fan-out of the
  batch-favorites authz pre-check — 42.6 → 56.4 ms cold, 0.15 → 0.23 ms
  warm against local-socket PG (bench_favorites_authz kept as evidence).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-18 09:03:33 +00:00

468 lines
18 KiB
TOML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
[package]
name = "oxicloud"
version = "0.8.2"
edition = "2024"
default-run = "oxicloud"
[dependencies]
mimalloc = { version = "0.1.52", default-features = false }
axum = { version = "0.8.9", features = ["multipart", "http1", "http2", "tokio", "macros"] }
# "process" was previously enabled implicitly through aws-config's feature
# unification; ffmpeg_video_frame_service needs it, so declare it ourselves.
tokio = { version = "1.52.3", features = ["rt-multi-thread", "macros", "io-util", "net", "time", "sync", "fs", "process"] }
tokio-util = { version = "0.7.18", features = ["io", "codec", "compat"] }
tokio-stream = { version = "0.1.18", features = ["fs", "sync"] }
bytes = "1.11.1"
tempfile = "3.27.0"
tower = "0.5.3"
tower-http = { version = "0.6.11", features = ["fs", "compression-gzip", "compression-br", "trace", "cors", "add-extension", "request-id", "set-header", "limit"] }
flate2 = "1.1.9"
tracing = "0.1.44"
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
chrono = { version = "0.4.45", features = ["serde"] }
# RFC 5545 iCalendar parser + emitter.
#
# Adopted 2026-07-14 to replace the hand-rolled property-scan in
# `src/domain/entities/calendar_event.rs::extract_ical_property`,
# which used a naive `format!("\n{}:", name)` substring search and
# refused ANY property carrying parameters (`DTSTART;VALUE=DATE:...`,
# `RECURRENCE-ID;VALUE=DATE:...`, `ATTENDEE;CN=…;PARTSTAT=…:mailto:…`).
# That broke all-day events and made the domain unaware of exception
# instances (see AtalayaLabs/OxiCloud#528).
#
# The crate is the widely-used Rust parser (~1500 SLOC, MIT/Apache),
# actively maintained by @Peltoche as `ical-rs` on GitHub. It handles
# line-folding, escaped characters, parameter maps, and every standard
# component. If a spec conformance gap is found, we contribute upstream.
ical = "0.11"
http-body = "1.0.1"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.150"
futures = "0.3.32"
async-stream = "0.3.6"
async-trait = "0.1.89"
mime_guess = "2.0.5"
uuid = { version = "1.23.3", features = ["v4", "v7", "serde"] }
thiserror = "2.0.18"
mockall = { version = "0.14.0", optional = true }
sqlx = { version = "0.8.6", features = ["postgres", "runtime-tokio", "tls-rustls", "chrono", "uuid", "json", "migrate"] }
jsonwebtoken = { version = "10.4.0", features = ["rust_crypto"] }
argon2 = "0.5.3"
rand_core = { version = "0.6", features = ["std", "getrandom"] }
quick-xml = "0.41.0"
dotenvy = "0.15.7"
moka = { version = "0.12.15", features = ["future", "sync"] }
http-range-header = "0.4"
image = { version = "0.25.10", default-features = false, features = ["jpeg", "png", "gif", "webp"] }
# Shrink-on-load JPEG decode (DCT 1/2·1/4·1/8 scaling) for thumbnails — pure
# Rust, no C toolchain. The `image` crate's zune-jpeg backend can't scale during
# decode; this can, cutting decode time/RAM ~order-of-magnitude on large photos.
jpeg-decoder = "0.3"
# SIMD (AVX2/SSE4.1/NEON) image resizing for thumbnails — far faster than the
# `image` crate's scalar resampler, and it speeds every format (incl. PNG/WebP).
fast_image_resize = "5"
# Lossy WebP encoding for thumbnails. The `image` crate's webp feature is
# lossless-only; this binds libwebp (vendored + built via `cc`, no system dep)
# for quality-controlled lossy output (~25-30% smaller than JPEG at equal SSIM).
webp = "0.3"
id3 = "1.17"
mp3-duration = "0.1"
kamadak-exif = "0.6.1"
md-5 = "0.11.0"
sha2 = "0.11.0"
unicode-normalization = "0.1.25"
blake3 = { version = "1.8.5", features = ["rayon", "mmap"] }
hex = "0.4.3"
http-body-util = "0.1.3"
percent-encoding = "2.3.2"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots","rustls-tls-native-roots"] }
base64 = "0.22.1"
fs2 = "0.4"
rayon = "1.12.0"
infer = "0.19"
async-compression = { version = "0.4.42", features = ["tokio", "gzip"] }
async_zip = { version = "0.0.18", features = ["tokio", "deflate"] }
dashmap = "6.2.1"
socket2 = { version = "0.6.4", features = ["all"] }
urlencoding = "2.1.3"
utoipa = { version = "5.5.0", features = ["axum_extras", "uuid", "chrono"] }
# NOTE: aws-config and aws-smithy-types were removed as direct deps in the
# round-3 perf pass — S3BlobBackend builds its client purely from
# aws_sdk_s3::config with static credentials; nothing referenced either
# crate, and aws-config alone pulled aws-sdk-sso/ssooidc/sts (~90 crates)
# into every build (benches/ROUND3.md).
aws-sdk-s3 = "1.136.0"
azure_core = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage_blobs = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
aes-gcm = "0.10.3"
lru = "0.16.4"
fastcdc = { version = "4.0.0", features = ["tokio"] }
lettre = { version = "0.11.18", default-features = false, features = ["smtp-transport", "tokio1-rustls-tls", "rustls-native-certs", "builder"] }
idna = "1.1"
smol_str = { version = "0.3.6", features = ["serde"] }
accept-language = "3.1.0"
askama = "0.16.0"
tantivy = "0.26.1"
zip = { version = "8.6.0", default-features = false, features = ["deflate"] }
pdf-extract = "0.12.0"
nom-exif = "3.6.1"
extism = { version = "1.30.0", optional = true }
toml = { version = "1.1.2", optional = true }
file-rotate = { version = "0.7.6", optional = true }
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "tracing", "api-24"], optional = true }
ndarray = { version = "0.17.2", optional = true }
[features]
default = []
test_utils = ["mockall"]
integration_tests = []
# WASM plugin runtime (Extism). Opt-in: bundles wasmtime, a large engine most
# deployments won't use. Activation also requires OXICLOUD_ENABLE_PLUGINS=true.
plugins = ["dep:extism", "dep:toml", "dep:file-rotate"]
# Empty marker feature that gates the `load-seed` binary so it isn't built
# by default (and skipped in prod Docker builds). Kept separate from
# test_utils so enabling it doesn't change the oxicloud dependency graph —
# this lets one `cargo build` produce both `oxicloud` and `load-seed`
# without recompiling oxicloud with mockall in scope.
load_seed_bin = []
# Real ONNX-backed face analyzer (detector + embedder) for the People feature.
# Opt-in: pulls `ort` (ONNX Runtime, load-dynamic — dlopen's libonnxruntime at
# runtime) + `ndarray`, a heavy stack most deployments won't use. Activation also
# requires OXICLOUD_ENABLE_FACES=true *and* operator-provided ONNX models; without
# this feature the People pipeline falls back to the inert NoopFaceAnalyzer.
faces-onnx = ["dep:ort", "dep:ndarray"]
# Performance benchmark harness (Phase 0). Exposes `bench_support` + thin public
# wrappers over the private thumbnail render functions so `benches/` and
# `examples/` can measure them. Off by default — adds nothing to prod builds.
# Run with: `cargo bench --features bench` / `cargo run --release --features bench --example bench_thumbnails_mem`.
bench = []
[dev-dependencies]
criterion = "0.5"
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(integration_tests)'] }
[[bin]]
name = "generate-openapi"
path = "src/bin/generate-openapi.rs"
[[bin]]
name = "migrate-nfc-filenames"
path = "src/bin/migrate-nfc-filenames.rs"
[[bin]]
name = "load-seed"
path = "src/bin/load-seed.rs"
# Test fixture seeder. Gated behind the empty `load_seed_bin` feature so
# `cargo build --release` (and the prod Dockerfile) skip it. tests/load/run.sh
# and load-nightly.yml build it explicitly with --features load_seed_bin.
required-features = ["load_seed_bin"]
# Phase 0 perf harness — Task 0.2 (criterion latency + output-size bench).
[[bench]]
name = "thumbnails"
path = "benches/thumbnails.rs"
harness = false
required-features = ["bench"]
# Phase 0 perf harness — Task 0.3 (peak-RAM + saturated-throughput baseline).
[[example]]
name = "bench_thumbnails_mem"
path = "examples/bench_thumbnails_mem.rs"
required-features = ["bench"]
# ZIP entry-compression benchmark — Deflate-always vs MIME-aware Stored for
# already-compressed media on the folder/batch ZIP download path. No Postgres.
[[example]]
name = "bench_zip_media"
path = "examples/bench_zip_media.rs"
required-features = ["bench"]
# WebDAV dead-properties fetch benchmark — PROPFIND's per-child N+1 (with a
# non-indexable IS NOT DISTINCT FROM predicate) vs batched = ANY($1) per page
# (needs the dev Postgres up).
[[example]]
name = "bench_dead_props"
path = "examples/bench_dead_props.rs"
required-features = ["bench"]
# NC chroot / default-drive resolution benchmark — the middleware's 2 uncached
# queries per request vs the moka caches (needs the dev Postgres up).
[[example]]
name = "bench_chroot_cache"
path = "examples/bench_chroot_cache.rs"
required-features = ["bench"]
# Quota-path benchmark — full auth.users row (incl. 512 KiB avatar) vs the
# narrow 2-column read, on every upload check / quota PROPFIND (needs Postgres).
[[example]]
name = "bench_quota_path"
path = "examples/bench_quota_path.rs"
required-features = ["bench"]
# People-tab benchmark — full faces scan (2 KiB embedding per row) vs grouped
# COUNT + batched cover lookup (needs Postgres).
[[example]]
name = "bench_people_list"
path = "examples/bench_people_list.rs"
required-features = ["bench"]
# PROPFIND folder-paging benchmark — LIMIT/OFFSET full-folder rescan per page
# vs keyset + (folder_id, name) index (needs Postgres).
[[example]]
name = "bench_propfind_paging"
path = "examples/bench_propfind_paging.rs"
required-features = ["bench"]
# Static-asset compression benchmark — per-request Brotli vs precompressed
# sibling read. No Postgres.
[[example]]
name = "bench_static_precompress"
path = "examples/bench_static_precompress.rs"
required-features = ["bench"]
# Round-2 battery: range-from-cache, NC chunk gate, delta prefetch, ingest
# overlap (real store_from_stream; run with OXICLOUD_INGEST_OVERLAP=0/1),
# ZIP streaming TTFB. Sections 1 and 4 need Postgres.
[[example]]
name = "bench_round2"
path = "examples/bench_round2.rs"
required-features = ["bench"]
# Video thumbnail benchmark — Option B (server-side ffmpeg frame → WebP). Needs
# `ffmpeg` on PATH (libx264/libx265/libvpx-vp9 to synthesize the test corpus).
[[example]]
name = "bench_video_thumbnails"
path = "examples/bench_video_thumbnails.rs"
required-features = ["bench"]
# DB connection-pool tail-latency benchmark (needs the dev Postgres up).
[[example]]
name = "bench_db_pool"
path = "examples/bench_db_pool.rs"
required-features = ["bench"]
# Blob-manifest read round-trip benchmark: OLD 2 queries vs NEW 1 (needs dev Postgres).
[[example]]
name = "bench_blob_manifest"
path = "examples/bench_blob_manifest.rs"
required-features = ["bench"]
# Blob download read-ahead benchmark — sweeps the local backend's chunk prefetch
# depth (read_prefetch / buffered(N)) under disk-bound vs network-bound consumers
# and warm vs cold page cache. No Postgres needed.
[[example]]
name = "bench_blob_prefetch"
path = "examples/bench_blob_prefetch.rs"
required-features = ["bench"]
# Tokio runtime tuning benchmark — worker over-subscription (throughput + p99
# under CPU contention) and blocking-pool RSS blast radius, default vs tuned.
# No Postgres needed; run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_tokio_runtime"
path = "examples/bench_tokio_runtime.rs"
required-features = ["bench"]
# CPU pool concurrency benchmark — thumbnail decode throughput + p99 as the
# decode-permit count is swept, showing the effect of sizing the image pools to
# the CFS quota (effective_parallelism) vs the host core count. No Postgres;
# run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_pool_concurrency"
path = "examples/bench_pool_concurrency.rs"
required-features = ["bench"]
# ACL owner-cache benchmark — owner query vs moka hit (needs the dev Postgres up).
[[example]]
name = "bench_owner_cache"
path = "examples/bench_owner_cache.rs"
required-features = ["bench"]
# Round-4 battery ─────────────────────────────────────────────────────────────
# PG row → entity path materialization — the per-listing-row make_file_path
# split→rejoin + NFC copy chain vs the one-pass builders. No Postgres.
[[example]]
name = "bench_row_path"
path = "examples/bench_row_path.rs"
required-features = ["bench"]
# WebDAV drive-selector resolution — the per-request list_readable_by grants
# join vs the per-user readable_cache (needs the dev Postgres up).
[[example]]
name = "bench_drive_selector"
path = "examples/bench_drive_selector.rs"
required-features = ["bench"]
# CalDAV parse path — from_ical's 8×-reparse vs single parse, per-event
# uppercase copies on REPORT/GET, UID clone churn. No Postgres.
[[example]]
name = "bench_caldav_parse"
path = "examples/bench_caldav_parse.rs"
required-features = ["bench"]
# PROPFIND per-row XML emit — partition Vec churn + chrono format-interpreter
# dates vs single-pass + stack-rendered fields. No Postgres.
[[example]]
name = "bench_propfind_xml"
path = "examples/bench_propfind_xml.rs"
required-features = ["bench"]
# Grant-listing hydration N+1 (calendars / address books / playlists) +
# user-flags cold-cache herd (needs the dev Postgres up).
[[example]]
name = "bench_n1_hydration"
path = "examples/bench_n1_hydration.rs"
required-features = ["bench"]
# Face-indexing fan-out — unbounded per-image spawn vs core-count semaphore;
# peak-live-heap + wall on the bench_support photo corpus. No Postgres.
[[example]]
name = "bench_faces_bound"
path = "examples/bench_faces_bound.rs"
required-features = ["bench"]
# Azure download path — whole-blob collect vs streamed pages, TTFB + peak
# live heap against a local Azure-GET stub (endpoint_url hook). No Postgres.
[[example]]
name = "bench_azure_stream"
path = "examples/bench_azure_stream.rs"
required-features = ["bench"]
# Round-5 battery ─────────────────────────────────────────────────────────────
# CalDAV whole-calendar REPORT/GET — buffered double-residency vs uid-keyset
# streaming; TTFB + peak live heap (needs the dev Postgres up).
[[example]]
name = "bench_caldav_stream"
path = "examples/bench_caldav_stream.rs"
required-features = ["bench"]
# Round-5 micro-allocation pack — suggest clones, readable-cache Arc hit,
# SPA-listing interning, NC href prefix, CardDAV REPORT churn. No Postgres.
[[example]]
name = "bench_micro_allocs"
path = "examples/bench_micro_allocs.rs"
required-features = ["bench"]
# Round-6 battery ─────────────────────────────────────────────────────────────
# CardDAV whole-book REPORT/PROPFIND — buffered double-residency vs cursor
# streaming; TTFB + peak live heap (needs the dev Postgres up).
[[example]]
name = "bench_carddav_stream"
path = "examples/bench_carddav_stream.rs"
required-features = ["bench"]
# Batch-favorites authz pre-check — serial require loop vs try_join_all
# against the real PgAclEngine (needs the dev Postgres up).
[[example]]
name = "bench_favorites_authz"
path = "examples/bench_favorites_authz.rs"
required-features = ["bench"]
# Digest-hex rendering + NC id-batch marshalling micro-allocs (pure CPU).
[[example]]
name = "bench_hex_ids"
path = "examples/bench_hex_ids.rs"
required-features = ["bench"]
# `id::text` server cast vs binary UUID decode + app-side formatting A/B
# (needs the dev Postgres up).
[[example]]
name = "bench_uuid_text_cast"
path = "examples/bench_uuid_text_cast.rs"
required-features = ["bench"]
# Round-3 battery ─────────────────────────────────────────────────────────────
# Web-UI folder listing — whole-folder rescan + top-N sort per page vs keyset
# pushdown into the UNION-ALL branches + (folder_id, LOWER(name), id) indexes
# (needs the dev Postgres up).
[[example]]
name = "bench_listing_keyset"
path = "examples/bench_listing_keyset.rs"
required-features = ["bench"]
# Photos timeline — full-library scan + top-N above the grants join vs
# per-drive LATERAL top-N on the media-timeline index (needs Postgres).
[[example]]
name = "bench_photos_timeline"
path = "examples/bench_photos_timeline.rs"
required-features = ["bench"]
# PROPFIND subfolder paging — LIMIT/OFFSET + COUNT(*) OVER() per page vs
# keyset batch, mirroring the files-side PROPFIND-PAGING fix (needs Postgres).
[[example]]
name = "bench_folder_keyset"
path = "examples/bench_folder_keyset.rs"
required-features = ["bench"]
# Basic-auth thundering herd — K concurrent cache misses each paying Argon2id
# vs single-flight try_get_with (needs Postgres).
[[example]]
name = "bench_auth_herd"
path = "examples/bench_auth_herd.rs"
required-features = ["bench"]
# CachedBlobBackend — miss stampede (N duplicate remote fetches racing on one
# .tmp) vs per-hash single-flight; warm-hit index throughput. No Postgres.
[[example]]
name = "bench_blob_cache"
path = "examples/bench_blob_cache.rs"
required-features = ["bench"]
# Upload spool/assembly I/O — ReaderStream capacity sweep on part-file reads
# and BufWriter vs bare-File frame writes on the chunk spool path. No Postgres.
[[example]]
name = "bench_upload_spool"
path = "examples/bench_upload_spool.rs"
required-features = ["bench"]
# S3 chunk PUT — HEAD-before-PUT vs unconditional PUT against a local axum
# stub with injected latency; Azure Bytes-vs-to_vec copy micro. No Postgres.
[[example]]
name = "bench_s3_put"
path = "examples/bench_s3_put.rs"
required-features = ["bench"]
# File/Folder -> DTO mapping allocations — Arc<str> interning of closed-set
# display fields, 1-alloc etag/size formatting. No Postgres.
[[example]]
name = "bench_dto_map"
path = "examples/bench_dto_map.rs"
required-features = ["bench"]
# CardDAV REPORT — dead per-contact vCard pre-generation + O(N^2) uid scan vs
# single on-demand generation. No Postgres.
[[example]]
name = "bench_carddav_report"
path = "examples/bench_carddav_report.rs"
required-features = ["bench"]
# Search-results cache RSS — entry-count capacity vs byte weigher. No Postgres.
[[example]]
name = "bench_search_cache_mem"
path = "examples/bench_search_cache_mem.rs"
required-features = ["bench"]
[profile.release]
lto = "thin"
codegen-units = 1
opt-level = 3
strip = true
[profile.dev]
opt-level = 1
debug = true
[profile.bench]
lto = "fat"
codegen-units = 1
opt-level = 3