cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

perf: round 10 — auth alloc purge, parent-herd batching, query-shape pack, NC 304s

Browse Source 
Benchmark-gated (benches/ROUND10.md; every change carries a BEFORE/AFTER
harness with equivalence/safety gates — two designs were rejected or
rewritten by their own benches before adoption):

- Auth hot path: TokenClaims/CurrentUser display fields to Arc<str>, role
  to inline SmolStr end-to-end (Bearer, cookie, Basic-auth cache) — 4→1
  allocs per authenticated request, 3→0 per warm DAV request; JWT
  Encoding/Decoding/Validation built once.
- Cold shared-album herd: leader-inline parent batching in PgAclEngine
  (+ cascade try_get_with single-flight) — 100→2 parent queries per
  100-thumb cold herd, herd wall 1.9x, sequential + warm paths unchanged,
  all ROUND8/9 safety gates plus new herd-equivalence gates.
- Query-shape pack: share download double-fetch 2→1 (2.18x), contact-group
  COUNT(*) 14.9x, save_faces UNNEST 3.9x, playlist reorder UNNEST 63.7x
  (now atomic), search files∥folders join! 1.45x, move drive-lookup join!
  2.14x, trash partial (drive_id, trashed_at) indexes, CalDAV event-gate
  narrow read, favorites/recents binary-decode port, dead count_files
  removed.
- NC surface: preview + avatar honour If-None-Match (e2e: 5 KB and 197 KB
  → 0 bytes per revalidation), avatar WebP→PNG transcode memoised,
  PROPFIND/trashbin integer+date emits on stack formatters, folder-header
  enrichment join!, chunk-PUT retry stat folded into create_new open.
- common::fmt integer rendering rewritten on the std 2-digit LUT after the
  round's own bench caught the div-loop losing to to_string (16.1 ns vs
  22.5; speeds every prior-round call site).
- Micro-pack: WebDAV scope probe borrow-only, ShareService base_url
  snapshot, cookie_secure OnceLock, Arc'd AES-GCM cipher, stack request-id,
  tantivy analyzer clone dropped.
- SPA: search stale-guard + AbortController (10→1 completed round-trips,
  stale-clobber gone), getFolder in-flight dedup, gridColumns matchMedia
  hoist (10k→0 style reads).

Backend: cargo fmt + clippy -D warnings clean, 524 tests green.
Frontend: npm run check clean, 301 vitest green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018DdM7V7M3QPW7HEHg3gLov
This commit is contained in:
Claude
2026-07-18 20:33:50 +00:00
parent 4fe429a109
commit c51af68432
64 changed files with 3452 additions and 442 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/interfaces/nextcloud/avatar_handler.rs
+79 -31
View File
@@ -1,13 +1,32 @@
use axum::{
extract::{Path, State},
http::{StatusCode, header},
http::{HeaderMap, StatusCode, header},
response::{IntoResponse, Response},
};
use base64::Engine;
use bytes::Bytes;
use std::sync::Arc;
use crate::common::di::AppState;
/// Transcoded-avatar memo: `blake3(stored data URI)` → PNG bytes.
///
/// The WebP→PNG transcode below is a full image decode + PNG encode (tens
/// of ms of CPU) that used to run on EVERY avatar request once the
/// client's 1 h cache lapsed — per client, per surface. Avatars are tiny
/// and rarely change; 32 entries bounds the memo to a few MB.
static AVATAR_PNG_CACHE: std::sync::OnceLock<moka::sync::Cache<[u8; 32], Bytes>> =
std::sync::OnceLock::new();
fn avatar_png_cache() -> &'static moka::sync::Cache<[u8; 32], Bytes> {
AVATAR_PNG_CACHE.get_or_init(|| {
moka::sync::Cache::builder()
.max_capacity(32)
.time_to_live(std::time::Duration::from_secs(24 * 3600))
.build()
})
}
/// Re-encode WebP image bytes as PNG. Returns `None` on decode/encode
/// failure (treated upstream as "fall through to SVG" — a bad stored
/// blob shouldn't break the rendering pipeline). PNG is universal:
@@ -77,10 +96,11 @@ fn parse_data_uri(uri: &str) -> Option<(String, Vec<u8>)> {
pub async fn handle_dav_avatar(
state: State<Arc<AppState>>,
Path((username, size_with_ext)): Path<(String, String)>,
headers: HeaderMap,
) -> Response {
let size_str = size_with_ext.strip_suffix(".png").unwrap_or(&size_with_ext);
let size: u32 = size_str.parse().unwrap_or(64);
handle_avatar(state, Path((username, size))).await
handle_avatar(state, Path((username, size)), headers).await
}
/// GET /index.php/avatar/{user}/{size}
@@ -98,6 +118,7 @@ pub async fn handle_dav_avatar(
pub async fn handle_avatar(
State(state): State<Arc<AppState>>,
Path((username, size)): Path<(String, u32)>,
headers: HeaderMap,
) -> Response {
let size = size.clamp(16, 1024);
@@ -113,39 +134,66 @@ pub async fn handle_avatar(
.get_user_by_username(&username)
.await
&& let Some(image_uri) = user.image.as_deref()
&& let Some((mime, bytes)) = parse_data_uri(image_uri)
{
// WebP is OxiCloud's storage format of choice (smaller files,
// better quality at a given size) but NextCloud clients have
// patchy WebP support — older Qt-based desktop builds, some
// mobile image stacks. Transcode to PNG before serving on the
// NC surface so every client renders it. PNG is bigger on the
// wire but small enough at avatar dimensions that the
// tradeoff is worth it. Decode failure falls through to SVG.
let (final_mime, final_bytes): (&str, Vec<u8>) = if mime == "image/webp" {
match webp_to_png(&bytes) {
Some(png) => ("image/png", png),
None => return svg_initials_response(&username, size),
}
} else {
// Whatever MIME we stored (`image/png`, `image/jpeg`,
// `image/gif`) is universally supported by NC clients.
// The `mime` String is moved out via `.as_str()` here, so
// bind it locally to keep the borrow alive for the response.
(mime_as_static_str(&mime), bytes)
};
return (
StatusCode::OK,
[
(header::CONTENT_TYPE, final_mime),
// Content-derived ETag over the STORED value — computable before
// any base64 decode or image work. NC desktop/mobile revalidate
// avatars every cache lapse (1 h) per surface; this endpoint used
// to re-decode (and for WebP re-transcode to PNG — a full image
// decode + encode) and re-ship the body every time (ROUND10).
let content_hash: [u8; 32] = blake3::hash(image_uri.as_bytes()).into();
let etag = format!(
"\"av-{}\"",
crate::common::fmt::hex_lower(&content_hash[..12])
);
if let Some(inm) = headers.get(header::IF_NONE_MATCH)
&& let Ok(client_etag) = inm.to_str()
&& (client_etag == etag || client_etag == "*")
{
return Response::builder()
.status(StatusCode::NOT_MODIFIED)
.header(header::CACHE_CONTROL, "public, max-age=3600")
.header(header::ETAG, etag)
.body(axum::body::Body::empty())
.unwrap();
}
if let Some((mime, bytes)) = parse_data_uri(image_uri) {
// WebP is OxiCloud's storage format of choice (smaller files,
// better quality at a given size) but NextCloud clients have
// patchy WebP support — older Qt-based desktop builds, some
// mobile image stacks. Transcode to PNG before serving on the
// NC surface so every client renders it. The transcode result
// is memoised by content hash — decode+encode ran per request
// before. Decode failure falls through to SVG.
let (final_mime, final_bytes): (&str, Bytes) = if mime == "image/webp" {
if let Some(png) = avatar_png_cache().get(&content_hash) {
("image/png", png)
} else {
match webp_to_png(&bytes) {
Some(png) => {
let png = Bytes::from(png);
avatar_png_cache().insert(content_hash, png.clone());
("image/png", png)
}
None => return svg_initials_response(&username, size),
}
}
} else {
// Whatever MIME we stored (`image/png`, `image/jpeg`,
// `image/gif`) is universally supported by NC clients.
(mime_as_static_str(&mime), Bytes::from(bytes))
};
return Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, final_mime)
// Shorter cache than the SVG fallback because users can
// re-upload their picture at any time — the URL is the
// same so a long immutable cache would pin the old one.
(header::CACHE_CONTROL, "public, max-age=3600"),
],
final_bytes,
)
.into_response();
.header(header::CACHE_CONTROL, "public, max-age=3600")
.header(header::ETAG, etag)
.body(axum::body::Body::from(final_bytes))
.unwrap();
}
}
svg_initials_response(&username, size)
src/interfaces/nextcloud/login_v2_handler.rs
+7 -7
View File
@@ -339,9 +339,9 @@ pub async fn handle_oidc_login_completion(
let current_user = CurrentUser {
id: user_id,
username: username.to_string(),
email: user_dto.email.clone(),
role: user_dto.role.clone(),
username: std::sync::Arc::from(username),
email: std::sync::Arc::from(user_dto.email.as_str()),
role: smol_str::SmolStr::new(&user_dto.role),
};
let drives = match state
@@ -438,7 +438,7 @@ async fn complete_flow(
let login_name = match drive_id {
Some(uuid) => format!("{}~{}", user.username, uuid),
None => user.username.clone(),
None => user.username.to_string(),
};
let base_url = state.core.config.base_url();
@@ -550,9 +550,9 @@ pub async fn handle_drive_pick(
};
let user = CurrentUser {
id: user_id,
username,
email: user_dto.email.clone(),
role: user_dto.role.clone(),
username: std::sync::Arc::from(username.as_str()),
email: std::sync::Arc::from(user_dto.email.as_str()),
role: smol_str::SmolStr::new(&user_dto.role),
};
let _folder = match state
src/interfaces/nextcloud/ocs_handler.rs
+3 -3
View File
@@ -109,11 +109,11 @@ pub async fn handle_user_info(
// than the raw UUID the wire form carries.
let id = session.raw_username.clone();
let displayname = if session.is_home() {
session.user.username.clone()
session.user.username.to_string()
} else {
match session.chroot.as_ref() {
Some(chroot) => format!("{}@{}", session.user.username, chroot.name),
None => session.user.username.clone(),
None => session.user.username.to_string(),
}
};
@@ -356,7 +356,7 @@ pub async fn handle_sharees_search(
.into_iter()
.filter_map(|u| {
let handle = u.username.clone()?;
if handle == user.username {
if handle.as_str() == &*user.username {
return None;
}
Some(json!({
src/interfaces/nextcloud/preview_handler.rs
+31 -13
View File
@@ -5,7 +5,7 @@
use axum::{
body::Body,
extract::{Query, State},
http::{StatusCode, header},
http::{HeaderMap, StatusCode, header},
response::{IntoResponse, Response},
};
use serde::Deserialize;
@@ -39,6 +39,7 @@ pub async fn handle_preview(
State(state): State<Arc<AppState>>,
user: AuthUser,
Query(params): Query<PreviewParams>,
headers: HeaderMap,
) -> impl IntoResponse {
// Parse the Nextcloud file ID — the NC app may append an instance suffix
// (e.g. "00000326ocnca"), so strip non-digit characters first.
@@ -135,6 +136,27 @@ pub async fn handle_preview(
}
};
// Conditional revalidation — the ETag is derived from (object id, size)
// only, so it is computable right here, BEFORE the blob-hash query and
// the thumbnail cache/disk read. NC clients revalidate gallery previews
// constantly; the REST thumbnail endpoint has honoured `If-None-Match`
// since PHOTOS-ETAG — this endpoint set an immutable ETag but never
// compared it, so every revalidation re-ran the whole pipeline and
// re-shipped the body (ROUND10). Authz already passed above; a 304
// must never skip the Read check.
let etag = format!("\"thumb-{}-{:?}\"", object_id, thumb_size);
if let Some(inm) = headers.get(header::IF_NONE_MATCH)
&& let Ok(client_etag) = inm.to_str()
&& (client_etag == etag || client_etag == "*")
{
return Response::builder()
.status(StatusCode::NOT_MODIFIED)
.header(header::CACHE_CONTROL, "public, max-age=31536000, immutable")
.header(header::ETAG, etag)
.body(Body::empty())
.unwrap();
}
// Check if file is an image
if !state
.core
@@ -175,7 +197,6 @@ pub async fn handle_preview(
)
.await
{
let etag = format!("\"thumb-{}-{:?}\"", object_id, thumb_size);
return Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, "image/jpeg")
@@ -201,17 +222,14 @@ pub async fn handle_preview(
)
.await
{
Ok(data) => {
let etag = format!("\"thumb-{}-{:?}\"", object_id, thumb_size);
Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, "image/jpeg")
.header(header::CONTENT_LENGTH, data.len())
.header(header::CACHE_CONTROL, "public, max-age=31536000, immutable")
.header(header::ETAG, etag)
.body(Body::from(data))
.unwrap()
}
Ok(data) => Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, "image/jpeg")
.header(header::CONTENT_LENGTH, data.len())
.header(header::CACHE_CONTROL, "public, max-age=31536000, immutable")
.header(header::ETAG, etag)
.body(Body::from(data))
.unwrap(),
Err(err) => {
tracing::error!("Thumbnail generation failed for {}: {}", object_id, err);
Response::builder()
src/interfaces/nextcloud/trashbin_handler.rs
+16 -11
View File
@@ -16,7 +16,7 @@ use crate::common::di::AppState;
use crate::interfaces::errors::AppError;
use crate::interfaces::nextcloud::webdav_handler::{
batch_resolve_ids, extract_nc_subpath_from_dest, format_oc_id, nc_id_of, nc_to_internal_path,
write_text_element,
write_date_element, write_etag_element, write_text_element,
};
const HEADER_DAV: HeaderName = HeaderName::from_static("dav");
@@ -445,11 +445,12 @@ fn write_trash_item_response<W: std::io::Write>(
// d:displayname
write_text_element(xml, "d:displayname", &item.name)?;
// d:getlastmodified
write_text_element(xml, "d:getlastmodified", &item.trashed_at.to_rfc2822())?;
// d:getlastmodified — stack-rendered (common::fmt), chrono fallback for
// out-of-range timestamps; byte-identical to the old `to_rfc2822()`.
write_date_element(xml, "d:getlastmodified", item.trashed_at.timestamp(), true)?;
// d:getetag
write_text_element(xml, "d:getetag", &format!("\"{}\"", item.original_id))?;
// d:getetag — exact-size quoted alloc instead of the format! interpreter.
write_etag_element(xml, "d:getetag", &item.original_id)?;
// d:resourcetype
if item.item_type == "folder" {
@@ -478,7 +479,8 @@ fn write_trash_item_response<W: std::io::Write>(
// oc:fileid and oc:id — resolved up front in a batch query.
let file_id = nc_id_of(id_map, &item.original_id);
if let Some(id) = file_id {
write_text_element(xml, "oc:fileid", &id.to_string())?;
let mut ibuf = [0u8; 21];
write_text_element(xml, "oc:fileid", crate::common::fmt::i64_str(&mut ibuf, id))?;
let oc_id = format_oc_id(id, file_id_svc);
write_text_element(xml, "oc:id", &oc_id)?;
}
@@ -491,11 +493,14 @@ fn write_trash_item_response<W: std::io::Write>(
write_text_element(xml, "nc:trashbin-original-location", original_location)?;
// nc:trashbin-deletion-time
write_text_element(
xml,
"nc:trashbin-deletion-time",
&item.trashed_at.timestamp().to_string(),
)?;
{
let mut ibuf = [0u8; 21];
write_text_element(
xml,
"nc:trashbin-deletion-time",
crate::common::fmt::i64_str(&mut ibuf, item.trashed_at.timestamp()),
)?;
}
// oc:permissions — empty in trash
write_text_element(xml, "oc:permissions", "")?;
src/interfaces/nextcloud/uploads_handler.rs
+5 -4
View File
@@ -325,15 +325,16 @@ async fn handle_put_chunk(
.map_err(|e| AppError::bad_request(format!("Invalid chunk path: {}", e)))?;
let max_chunk = state.core.config.storage.chunk_max_bytes;
// A re-PUT of an existing chunk (client retry) makes the running
// session counter stale — drop it so the next gate rebuilds from disk.
let overwrite = tokio::fs::metadata(&chunk_path).await.is_ok();
// No client-side integrity contract on the NC chunked surface — the
// NC desktop client validates the assembled-file ETag against the
// server-side `oc:checksums` after MOVE. So we skip per-chunk
// hashing here (peak heap stays at ~one HTTP frame).
//
// Retry detection (a re-PUT makes the running session counter stale)
// rides on the open itself now — `created_fresh` from the `create_new`
// probe replaces the extra per-chunk `stat` this path used to issue.
let streamed = stream_body_to_path(req.into_body(), &chunk_path, max_chunk, None).await?;
if overwrite {
if !streamed.created_fresh {
nc.chunked_uploads
.forget_session_bytes(&user.username, upload_id);
} else {
src/interfaces/nextcloud/webdav_handler.rs
+48 -18
View File
@@ -1510,16 +1510,24 @@ fn build_nc_streaming_propfind(
// ── <d:multistatus> + the folder's own entry ─────────────────
// Collection hrefs MUST end in `/` (RFC 4918 §5.2 + strict
// NC-client enforcement — see `nc_collection_href`).
let folder_favs = if let Some(fav) = fav_svc {
fav.batch_check_favorites(user_id, &[(folder.id.as_str(), "folder")])
.await
.unwrap_or_default()
} else {
HashSet::new()
};
let (_, folder_id_map) =
batch_resolve_ids(file_id_svc, &[], &[folder.id.as_str()]).await;
let folder_dead = folder_dead_props(&state.webdav_dead_props, &folder).await;
// Same three independent reads as the per-page child triple below,
// and on the critical path of EVERY folder PROPFIND's first byte —
// overlapped with `join!` (ROUND10; the header trio was left serial
// when ROUND9 converted the page loops).
let folder_id_arr = [folder.id.as_str()];
let (folder_favs, (_, folder_id_map), folder_dead) = tokio::join!(
async {
if let Some(fav) = fav_svc {
fav.batch_check_favorites(user_id, &[(folder.id.as_str(), "folder")])
.await
.unwrap_or_default()
} else {
HashSet::new()
}
},
batch_resolve_ids(file_id_svc, &[], &folder_id_arr),
folder_dead_props(&state.webdav_dead_props, &folder),
);
let mut buf = Vec::with_capacity(4096);
{
@@ -1756,7 +1764,8 @@ pub fn write_folder_response<W: std::io::Write>(
// Nextcloud/ownCloud properties
if let Some(id) = file_id {
write_text_element(xml, "oc:fileid", &id.to_string())?;
let mut buf = [0u8; 21];
write_text_element(xml, "oc:fileid", crate::common::fmt::i64_str(&mut buf, id))?;
}
if let Some(oid) = oc_id {
write_text_element(xml, "oc:id", oid)?;
@@ -1770,9 +1779,18 @@ pub fn write_folder_response<W: std::io::Write>(
// surface's `write_folder_standard_props` (see
// `AppState::resolve_webdav_quota`).
if let Some((used, available)) = quota {
write_text_element(xml, "d:quota-used-bytes", &used.to_string())?;
let mut buf = [0u8; 21];
write_text_element(
xml,
"d:quota-used-bytes",
crate::common::fmt::i64_str(&mut buf, used),
)?;
if let Some(avail) = available {
write_text_element(xml, "d:quota-available-bytes", &avail.to_string())?;
write_text_element(
xml,
"d:quota-available-bytes",
crate::common::fmt::i64_str(&mut buf, avail),
)?;
}
}
write_text_element(xml, "oc:owner-id", owner)?;
@@ -1858,7 +1876,8 @@ pub fn write_file_response<W: std::io::Write>(
// Nextcloud/ownCloud properties
if let Some(id) = file_id {
write_text_element(xml, "oc:fileid", &id.to_string())?;
let mut buf = [0u8; 21];
write_text_element(xml, "oc:fileid", crate::common::fmt::i64_str(&mut buf, id))?;
}
if let Some(oid) = oc_id {
write_text_element(xml, "oc:id", oid)?;
@@ -1900,8 +1919,19 @@ pub fn write_file_response<W: std::io::Write>(
write_text_element(xml, "nc:is-encrypted", "0")?;
write_text_element(xml, "nc:mount-type", "")?;
write_text_element(xml, "nc:creation_time", &file.created_at.to_string())?;
write_text_element(xml, "nc:upload_time", &file.modified_at.to_string())?;
{
let mut buf = [0u8; 20];
write_text_element(
xml,
"nc:creation_time",
crate::common::fmt::u64_str(&mut buf, file.created_at),
)?;
write_text_element(
xml,
"nc:upload_time",
crate::common::fmt::u64_str(&mut buf, file.modified_at),
)?;
}
xml.write_event(Event::End(BytesEnd::new("d:prop")))
.xml_err()?;
@@ -1921,7 +1951,7 @@ pub fn write_file_response<W: std::io::Write>(
/// (`common::fmt`) — the old per-row `to_rfc2822()` / `to_rfc3339()`
/// ran chrono's format interpreter and allocated a String each.
/// Out-of-range timestamps keep the chrono path, byte-identical.
fn write_date_element<W: std::io::Write>(
pub fn write_date_element<W: std::io::Write>(
xml: &mut Writer<W>,
tag: &str,
secs: i64,
@@ -1946,7 +1976,7 @@ fn write_date_element<W: std::io::Write>(
/// `d:getetag` with the HTTP quoting — one exactly-sized allocation
/// instead of `format!`'s grow-from-empty.
fn write_etag_element<W: std::io::Write>(
pub fn write_etag_element<W: std::io::Write>(
xml: &mut Writer<W>,
tag: &str,
etag: &str,