Files
Oxicloud/Cargo.toml
T
Claude 7626dc95c1 perf: round 7 — photos timeline O(N²)→incremental, range-seek authz duplication, resources row-map clone
Benchmark-gated (equivalence + BEFORE/AFTER; results + reproduce commands in
benches/ROUND7.md):

- Photos timeline re-grouped + re-laid-out the whole accumulated library on
  every 60-item page (both `groups` and `photoRows` were $derived over the
  full list), Σ ≈ O(N²/60) main-thread work during a scroll. Pages arrive
  newest-first so grouping is append-only: the new PhotoTimeline
  (lib/utils/photoTimeline.ts) re-buckets only the fresh page and re-lays-out
  only changed groups, reusing untouched groups' cached rows, falling back to
  a full rebuild on any config/deletion/non-append change. The pure
  buildPhotoRows is the verbatim reference the gate holds it equal to at every
  page. 50×60 drain: 76 500 → 3 000 grouping ops (25.5x), 23.0 → 2.2 ms
  (10.6x).

- Range downloads paid authz + access-notify twice: download_file_impl
  resolves the file via get_file_with_perms, then the Range branch re-ran
  require_file + notify_file_accessed per request. Media/PDF viewers fetch
  exclusively via Range (one request per seek), so every seek in a scrub
  re-authorized an already-cleared file. Now routed through the non-perms
  get_file_range_preloaded (matching the share-landing + WebDAV range paths);
  the unused _with_perms range method is removed. The request-level gate still
  denies before the branch runs (bench asserts member granted, outsider
  denied). Per seek removed: WARM 0.67 µs, COLD 1362.66 µs — a grant-cascade
  drive-resolve query per seek for a shared-drive recipient on a cold cache.

- /api/folders/{id}/resources row→DTO mapping cloned row.name into the DTO
  though the row is owned; folders move it (fixed icons), files compute the
  name-derived icon/category classes first then move it. 500-row page:
  10.004 → 9.004 allocs/row (500 clones removed), output identical.

Deferred with rationale in ROUND7.md: thumbnail ACL-before-304 (security
posture — needs a security review, not a perf tweak), batch_operations
Arc<str>→String widening, list-view O(N²) on smaller lists, and the serial→
join! pairs (decide-by-bench with injected latency, per the round-6 rejection).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-18 13:11:41 +00:00

485 lines
19 KiB
TOML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
[package]
name = "oxicloud"
version = "0.8.2"
edition = "2024"
default-run = "oxicloud"
[dependencies]
mimalloc = { version = "0.1.52", default-features = false }
axum = { version = "0.8.9", features = ["multipart", "http1", "http2", "tokio", "macros"] }
# "process" was previously enabled implicitly through aws-config's feature
# unification; ffmpeg_video_frame_service needs it, so declare it ourselves.
tokio = { version = "1.52.3", features = ["rt-multi-thread", "macros", "io-util", "net", "time", "sync", "fs", "process"] }
tokio-util = { version = "0.7.18", features = ["io", "codec", "compat"] }
tokio-stream = { version = "0.1.18", features = ["fs", "sync"] }
bytes = "1.11.1"
tempfile = "3.27.0"
tower = "0.5.3"
tower-http = { version = "0.6.11", features = ["fs", "compression-gzip", "compression-br", "trace", "cors", "add-extension", "request-id", "set-header", "limit"] }
flate2 = "1.1.9"
tracing = "0.1.44"
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
chrono = { version = "0.4.45", features = ["serde"] }
# RFC 5545 iCalendar parser + emitter.
#
# Adopted 2026-07-14 to replace the hand-rolled property-scan in
# `src/domain/entities/calendar_event.rs::extract_ical_property`,
# which used a naive `format!("\n{}:", name)` substring search and
# refused ANY property carrying parameters (`DTSTART;VALUE=DATE:...`,
# `RECURRENCE-ID;VALUE=DATE:...`, `ATTENDEE;CN=…;PARTSTAT=…:mailto:…`).
# That broke all-day events and made the domain unaware of exception
# instances (see AtalayaLabs/OxiCloud#528).
#
# The crate is the widely-used Rust parser (~1500 SLOC, MIT/Apache),
# actively maintained by @Peltoche as `ical-rs` on GitHub. It handles
# line-folding, escaped characters, parameter maps, and every standard
# component. If a spec conformance gap is found, we contribute upstream.
ical = "0.11"
http-body = "1.0.1"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.150"
futures = "0.3.32"
async-stream = "0.3.6"
async-trait = "0.1.89"
mime_guess = "2.0.5"
uuid = { version = "1.23.3", features = ["v4", "v7", "serde"] }
thiserror = "2.0.18"
mockall = { version = "0.14.0", optional = true }
sqlx = { version = "0.8.6", features = ["postgres", "runtime-tokio", "tls-rustls", "chrono", "uuid", "json", "migrate"] }
jsonwebtoken = { version = "10.4.0", features = ["rust_crypto"] }
argon2 = "0.5.3"
rand_core = { version = "0.6", features = ["std", "getrandom"] }
quick-xml = "0.41.0"
dotenvy = "0.15.7"
moka = { version = "0.12.15", features = ["future", "sync"] }
http-range-header = "0.4"
image = { version = "0.25.10", default-features = false, features = ["jpeg", "png", "gif", "webp"] }
# Shrink-on-load JPEG decode (DCT 1/2·1/4·1/8 scaling) for thumbnails — pure
# Rust, no C toolchain. The `image` crate's zune-jpeg backend can't scale during
# decode; this can, cutting decode time/RAM ~order-of-magnitude on large photos.
jpeg-decoder = "0.3"
# SIMD (AVX2/SSE4.1/NEON) image resizing for thumbnails — far faster than the
# `image` crate's scalar resampler, and it speeds every format (incl. PNG/WebP).
fast_image_resize = "5"
# Lossy WebP encoding for thumbnails. The `image` crate's webp feature is
# lossless-only; this binds libwebp (vendored + built via `cc`, no system dep)
# for quality-controlled lossy output (~25-30% smaller than JPEG at equal SSIM).
webp = "0.3"
id3 = "1.17"
mp3-duration = "0.1"
kamadak-exif = "0.6.1"
md-5 = "0.11.0"
sha2 = "0.11.0"
unicode-normalization = "0.1.25"
blake3 = { version = "1.8.5", features = ["rayon", "mmap"] }
hex = "0.4.3"
http-body-util = "0.1.3"
percent-encoding = "2.3.2"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots","rustls-tls-native-roots"] }
base64 = "0.22.1"
fs2 = "0.4"
rayon = "1.12.0"
infer = "0.19"
async-compression = { version = "0.4.42", features = ["tokio", "gzip"] }
async_zip = { version = "0.0.18", features = ["tokio", "deflate"] }
dashmap = "6.2.1"
socket2 = { version = "0.6.4", features = ["all"] }
urlencoding = "2.1.3"
utoipa = { version = "5.5.0", features = ["axum_extras", "uuid", "chrono"] }
# NOTE: aws-config and aws-smithy-types were removed as direct deps in the
# round-3 perf pass — S3BlobBackend builds its client purely from
# aws_sdk_s3::config with static credentials; nothing referenced either
# crate, and aws-config alone pulled aws-sdk-sso/ssooidc/sts (~90 crates)
# into every build (benches/ROUND3.md).
aws-sdk-s3 = "1.136.0"
azure_core = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
azure_storage_blobs = { version = "0.21", default-features = false, features = ["enable_reqwest_rustls", "hmac_rust"] }
aes-gcm = "0.10.3"
lru = "0.16.4"
fastcdc = { version = "4.0.0", features = ["tokio"] }
lettre = { version = "0.11.18", default-features = false, features = ["smtp-transport", "tokio1-rustls-tls", "rustls-native-certs", "builder"] }
idna = "1.1"
smol_str = { version = "0.3.6", features = ["serde"] }
accept-language = "3.1.0"
askama = "0.16.0"
tantivy = "0.26.1"
zip = { version = "8.6.0", default-features = false, features = ["deflate"] }
pdf-extract = "0.12.0"
nom-exif = "3.6.1"
extism = { version = "1.30.0", optional = true }
toml = { version = "1.1.2", optional = true }
file-rotate = { version = "0.7.6", optional = true }
ort = { version = "2.0.0-rc.12", default-features = false, features = ["load-dynamic", "ndarray", "tracing", "api-24"], optional = true }
ndarray = { version = "0.17.2", optional = true }
[features]
default = []
test_utils = ["mockall"]
integration_tests = []
# WASM plugin runtime (Extism). Opt-in: bundles wasmtime, a large engine most
# deployments won't use. Activation also requires OXICLOUD_ENABLE_PLUGINS=true.
plugins = ["dep:extism", "dep:toml", "dep:file-rotate"]
# Empty marker feature that gates the `load-seed` binary so it isn't built
# by default (and skipped in prod Docker builds). Kept separate from
# test_utils so enabling it doesn't change the oxicloud dependency graph —
# this lets one `cargo build` produce both `oxicloud` and `load-seed`
# without recompiling oxicloud with mockall in scope.
load_seed_bin = []
# Real ONNX-backed face analyzer (detector + embedder) for the People feature.
# Opt-in: pulls `ort` (ONNX Runtime, load-dynamic — dlopen's libonnxruntime at
# runtime) + `ndarray`, a heavy stack most deployments won't use. Activation also
# requires OXICLOUD_ENABLE_FACES=true *and* operator-provided ONNX models; without
# this feature the People pipeline falls back to the inert NoopFaceAnalyzer.
faces-onnx = ["dep:ort", "dep:ndarray"]
# Performance benchmark harness (Phase 0). Exposes `bench_support` + thin public
# wrappers over the private thumbnail render functions so `benches/` and
# `examples/` can measure them. Off by default — adds nothing to prod builds.
# Run with: `cargo bench --features bench` / `cargo run --release --features bench --example bench_thumbnails_mem`.
bench = []
[dev-dependencies]
criterion = "0.5"
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(integration_tests)'] }
[[bin]]
name = "generate-openapi"
path = "src/bin/generate-openapi.rs"
[[bin]]
name = "migrate-nfc-filenames"
path = "src/bin/migrate-nfc-filenames.rs"
[[bin]]
name = "load-seed"
path = "src/bin/load-seed.rs"
# Test fixture seeder. Gated behind the empty `load_seed_bin` feature so
# `cargo build --release` (and the prod Dockerfile) skip it. tests/load/run.sh
# and load-nightly.yml build it explicitly with --features load_seed_bin.
required-features = ["load_seed_bin"]
# Phase 0 perf harness — Task 0.2 (criterion latency + output-size bench).
[[bench]]
name = "thumbnails"
path = "benches/thumbnails.rs"
harness = false
required-features = ["bench"]
# Phase 0 perf harness — Task 0.3 (peak-RAM + saturated-throughput baseline).
[[example]]
name = "bench_thumbnails_mem"
path = "examples/bench_thumbnails_mem.rs"
required-features = ["bench"]
# ZIP entry-compression benchmark — Deflate-always vs MIME-aware Stored for
# already-compressed media on the folder/batch ZIP download path. No Postgres.
[[example]]
name = "bench_zip_media"
path = "examples/bench_zip_media.rs"
required-features = ["bench"]
# WebDAV dead-properties fetch benchmark — PROPFIND's per-child N+1 (with a
# non-indexable IS NOT DISTINCT FROM predicate) vs batched = ANY($1) per page
# (needs the dev Postgres up).
[[example]]
name = "bench_dead_props"
path = "examples/bench_dead_props.rs"
required-features = ["bench"]
# NC chroot / default-drive resolution benchmark — the middleware's 2 uncached
# queries per request vs the moka caches (needs the dev Postgres up).
[[example]]
name = "bench_chroot_cache"
path = "examples/bench_chroot_cache.rs"
required-features = ["bench"]
# Quota-path benchmark — full auth.users row (incl. 512 KiB avatar) vs the
# narrow 2-column read, on every upload check / quota PROPFIND (needs Postgres).
[[example]]
name = "bench_quota_path"
path = "examples/bench_quota_path.rs"
required-features = ["bench"]
# People-tab benchmark — full faces scan (2 KiB embedding per row) vs grouped
# COUNT + batched cover lookup (needs Postgres).
[[example]]
name = "bench_people_list"
path = "examples/bench_people_list.rs"
required-features = ["bench"]
# PROPFIND folder-paging benchmark — LIMIT/OFFSET full-folder rescan per page
# vs keyset + (folder_id, name) index (needs Postgres).
[[example]]
name = "bench_propfind_paging"
path = "examples/bench_propfind_paging.rs"
required-features = ["bench"]
# Static-asset compression benchmark — per-request Brotli vs precompressed
# sibling read. No Postgres.
[[example]]
name = "bench_static_precompress"
path = "examples/bench_static_precompress.rs"
required-features = ["bench"]
# Round-2 battery: range-from-cache, NC chunk gate, delta prefetch, ingest
# overlap (real store_from_stream; run with OXICLOUD_INGEST_OVERLAP=0/1),
# ZIP streaming TTFB. Sections 1 and 4 need Postgres.
[[example]]
name = "bench_round2"
path = "examples/bench_round2.rs"
required-features = ["bench"]
# Video thumbnail benchmark — Option B (server-side ffmpeg frame → WebP). Needs
# `ffmpeg` on PATH (libx264/libx265/libvpx-vp9 to synthesize the test corpus).
[[example]]
name = "bench_video_thumbnails"
path = "examples/bench_video_thumbnails.rs"
required-features = ["bench"]
# DB connection-pool tail-latency benchmark (needs the dev Postgres up).
[[example]]
name = "bench_db_pool"
path = "examples/bench_db_pool.rs"
required-features = ["bench"]
# Blob-manifest read round-trip benchmark: OLD 2 queries vs NEW 1 (needs dev Postgres).
[[example]]
name = "bench_blob_manifest"
path = "examples/bench_blob_manifest.rs"
required-features = ["bench"]
# Blob download read-ahead benchmark — sweeps the local backend's chunk prefetch
# depth (read_prefetch / buffered(N)) under disk-bound vs network-bound consumers
# and warm vs cold page cache. No Postgres needed.
[[example]]
name = "bench_blob_prefetch"
path = "examples/bench_blob_prefetch.rs"
required-features = ["bench"]
# Tokio runtime tuning benchmark — worker over-subscription (throughput + p99
# under CPU contention) and blocking-pool RSS blast radius, default vs tuned.
# No Postgres needed; run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_tokio_runtime"
path = "examples/bench_tokio_runtime.rs"
required-features = ["bench"]
# CPU pool concurrency benchmark — thumbnail decode throughput + p99 as the
# decode-permit count is swept, showing the effect of sizing the image pools to
# the CFS quota (effective_parallelism) vs the host core count. No Postgres;
# run under `taskset -c 0,1` to model a 2-core quota.
[[example]]
name = "bench_pool_concurrency"
path = "examples/bench_pool_concurrency.rs"
required-features = ["bench"]
# ACL owner-cache benchmark — owner query vs moka hit (needs the dev Postgres up).
[[example]]
name = "bench_owner_cache"
path = "examples/bench_owner_cache.rs"
required-features = ["bench"]
# Round-4 battery ─────────────────────────────────────────────────────────────
# PG row → entity path materialization — the per-listing-row make_file_path
# split→rejoin + NFC copy chain vs the one-pass builders. No Postgres.
[[example]]
name = "bench_row_path"
path = "examples/bench_row_path.rs"
required-features = ["bench"]
# WebDAV drive-selector resolution — the per-request list_readable_by grants
# join vs the per-user readable_cache (needs the dev Postgres up).
[[example]]
name = "bench_drive_selector"
path = "examples/bench_drive_selector.rs"
required-features = ["bench"]
# CalDAV parse path — from_ical's 8×-reparse vs single parse, per-event
# uppercase copies on REPORT/GET, UID clone churn. No Postgres.
[[example]]
name = "bench_caldav_parse"
path = "examples/bench_caldav_parse.rs"
required-features = ["bench"]
# PROPFIND per-row XML emit — partition Vec churn + chrono format-interpreter
# dates vs single-pass + stack-rendered fields. No Postgres.
[[example]]
name = "bench_propfind_xml"
path = "examples/bench_propfind_xml.rs"
required-features = ["bench"]
# Grant-listing hydration N+1 (calendars / address books / playlists) +
# user-flags cold-cache herd (needs the dev Postgres up).
[[example]]
name = "bench_n1_hydration"
path = "examples/bench_n1_hydration.rs"
required-features = ["bench"]
# Face-indexing fan-out — unbounded per-image spawn vs core-count semaphore;
# peak-live-heap + wall on the bench_support photo corpus. No Postgres.
[[example]]
name = "bench_faces_bound"
path = "examples/bench_faces_bound.rs"
required-features = ["bench"]
# Azure download path — whole-blob collect vs streamed pages, TTFB + peak
# live heap against a local Azure-GET stub (endpoint_url hook). No Postgres.
[[example]]
name = "bench_azure_stream"
path = "examples/bench_azure_stream.rs"
required-features = ["bench"]
# Round-5 battery ─────────────────────────────────────────────────────────────
# CalDAV whole-calendar REPORT/GET — buffered double-residency vs uid-keyset
# streaming; TTFB + peak live heap (needs the dev Postgres up).
[[example]]
name = "bench_caldav_stream"
path = "examples/bench_caldav_stream.rs"
required-features = ["bench"]
# Round-5 micro-allocation pack — suggest clones, readable-cache Arc hit,
# SPA-listing interning, NC href prefix, CardDAV REPORT churn. No Postgres.
[[example]]
name = "bench_micro_allocs"
path = "examples/bench_micro_allocs.rs"
required-features = ["bench"]
# Round-7 battery ─────────────────────────────────────────────────────────────
# Range-seek per-request authz duplication — the per-seek require the range
# branch used to run (warm CPU + cold drive-resolve query) vs 0 after routing
# through the non-perms range read (needs the dev Postgres up).
[[example]]
name = "bench_range_seek_authz"
path = "examples/bench_range_seek_authz.rs"
required-features = ["bench"]
# `/api/folders/{id}/resources` row→DTO mapping — per-row name clone vs move
# (pure CPU; counting allocator).
[[example]]
name = "bench_resource_row_map"
path = "examples/bench_resource_row_map.rs"
required-features = ["bench"]
# Round-6 battery ─────────────────────────────────────────────────────────────
# CardDAV whole-book REPORT/PROPFIND — buffered double-residency vs cursor
# streaming; TTFB + peak live heap (needs the dev Postgres up).
[[example]]
name = "bench_carddav_stream"
path = "examples/bench_carddav_stream.rs"
required-features = ["bench"]
# Batch-favorites authz pre-check — serial require loop vs try_join_all
# against the real PgAclEngine (needs the dev Postgres up).
[[example]]
name = "bench_favorites_authz"
path = "examples/bench_favorites_authz.rs"
required-features = ["bench"]
# Digest-hex rendering + NC id-batch marshalling micro-allocs (pure CPU).
[[example]]
name = "bench_hex_ids"
path = "examples/bench_hex_ids.rs"
required-features = ["bench"]
# `id::text` server cast vs binary UUID decode + app-side formatting A/B
# (needs the dev Postgres up).
[[example]]
name = "bench_uuid_text_cast"
path = "examples/bench_uuid_text_cast.rs"
required-features = ["bench"]
# Round-3 battery ─────────────────────────────────────────────────────────────
# Web-UI folder listing — whole-folder rescan + top-N sort per page vs keyset
# pushdown into the UNION-ALL branches + (folder_id, LOWER(name), id) indexes
# (needs the dev Postgres up).
[[example]]
name = "bench_listing_keyset"
path = "examples/bench_listing_keyset.rs"
required-features = ["bench"]
# Photos timeline — full-library scan + top-N above the grants join vs
# per-drive LATERAL top-N on the media-timeline index (needs Postgres).
[[example]]
name = "bench_photos_timeline"
path = "examples/bench_photos_timeline.rs"
required-features = ["bench"]
# PROPFIND subfolder paging — LIMIT/OFFSET + COUNT(*) OVER() per page vs
# keyset batch, mirroring the files-side PROPFIND-PAGING fix (needs Postgres).
[[example]]
name = "bench_folder_keyset"
path = "examples/bench_folder_keyset.rs"
required-features = ["bench"]
# Basic-auth thundering herd — K concurrent cache misses each paying Argon2id
# vs single-flight try_get_with (needs Postgres).
[[example]]
name = "bench_auth_herd"
path = "examples/bench_auth_herd.rs"
required-features = ["bench"]
# CachedBlobBackend — miss stampede (N duplicate remote fetches racing on one
# .tmp) vs per-hash single-flight; warm-hit index throughput. No Postgres.
[[example]]
name = "bench_blob_cache"
path = "examples/bench_blob_cache.rs"
required-features = ["bench"]
# Upload spool/assembly I/O — ReaderStream capacity sweep on part-file reads
# and BufWriter vs bare-File frame writes on the chunk spool path. No Postgres.
[[example]]
name = "bench_upload_spool"
path = "examples/bench_upload_spool.rs"
required-features = ["bench"]
# S3 chunk PUT — HEAD-before-PUT vs unconditional PUT against a local axum
# stub with injected latency; Azure Bytes-vs-to_vec copy micro. No Postgres.
[[example]]
name = "bench_s3_put"
path = "examples/bench_s3_put.rs"
required-features = ["bench"]
# File/Folder -> DTO mapping allocations — Arc<str> interning of closed-set
# display fields, 1-alloc etag/size formatting. No Postgres.
[[example]]
name = "bench_dto_map"
path = "examples/bench_dto_map.rs"
required-features = ["bench"]
# CardDAV REPORT — dead per-contact vCard pre-generation + O(N^2) uid scan vs
# single on-demand generation. No Postgres.
[[example]]
name = "bench_carddav_report"
path = "examples/bench_carddav_report.rs"
required-features = ["bench"]
# Search-results cache RSS — entry-count capacity vs byte weigher. No Postgres.
[[example]]
name = "bench_search_cache_mem"
path = "examples/bench_search_cache_mem.rs"
required-features = ["bench"]
[profile.release]
lto = "thin"
codegen-units = 1
opt-level = 3
strip = true
[profile.dev]
opt-level = 1
debug = true
[profile.bench]
lto = "fat"
codegen-units = 1
opt-level = 3