Merge upstream/main into feat/external-file-mounts

Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).

Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
  all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
  and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
  keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
  signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
  (the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
  regenerate Cargo.lock against the merged Cargo.toml.
This commit is contained in:
Bradley Nelson
2026-07-21 17:09:36 -06:00
600 changed files with 105575 additions and 14440 deletions
+446 -1
View File
@@ -310,6 +310,10 @@ pub struct AzureStorageConfig {
pub container: String,
/// Optional SAS token (alternative to account key).
pub sas_token: Option<String>,
/// Optional custom endpoint (Azurite emulator, private deployments,
/// benches). `None` = the public cloud URL derived from the account
/// name. Mirrors S3's `endpoint_url`.
pub endpoint_url: Option<String>,
}
/// LRU local disk cache configuration for remote blob backends.
@@ -470,6 +474,148 @@ pub struct AuthConfig {
pub hash_parallelism: u32,
/// Rate limiting / account lockout configuration
pub rate_limit: RateLimitConfig,
/// Allowlist of email domains accepted on the public `POST
/// /api/auth/register` endpoint. Empty = no restriction (any
/// domain is allowed). Entries are lowercased and trimmed at
/// load time; matching is case-insensitive exact-match on the
/// post-`@` part of the address.
///
/// This is DISTINCT from
/// [`MagicLinkConfig::allowed_email_domains`], which gates who
/// can be INVITED (email-typed grants + magic-link login for
/// existing recipients). This list gates SELF-registration
/// only. An operator can, for example, keep public registration
/// open to `partner-a.com` and `partner-b.io` while allowing
/// invitations to any domain — the two lists are independent.
///
/// Example: `["partner-a.com", "partner-b.io"]` — only
/// addresses `<anything>@partner-a.com` or
/// `<anything>@partner-b.io` can self-register; everything else
/// is rejected with 403 `RegistrationDomainNotAllowed`.
///
/// Wildcards / subdomain semantics are intentionally out of
/// scope (mirroring `MagicLinkConfig::allowed_email_domains`):
/// `partner.com` does NOT match `eng.partner.com`. List every
/// subdomain explicitly.
///
/// Env: `OXICLOUD_REGISTRATION_ALLOWED_EMAIL_DOMAINS` (comma-
/// separated).
pub registration_allowed_email_domains: Vec<String>,
/// Additive auth-policy toggles the operator has opted into.
/// Distinct from `allowed_auth_methods` (which enables/disables a
/// method wholesale) — this vector composes policy switches that
/// tweak the default auth behaviour. Empty = pure defaults in
/// effect, matching legacy behaviour.
///
/// Vector shape (rather than one boolean per policy) so future
/// switches can be added by appending a variant instead of
/// growing the env-var surface — `OXICLOUD_AUTH_POLICIES=policy_a,policy_b`.
/// Each variant's name carries its own polarity (`Permit...`,
/// future `Require...` / `Deny...`); the field name stays neutral
/// so a future deny-style policy reads correctly at the call site.
///
/// Env: `OXICLOUD_AUTH_POLICIES` (comma-separated).
///
/// Deprecated legacy alias: `OXICLOUD_MAGIC_LINK_OPEN_TO_PASSWORD_USERS=true`
/// still adds `PermitMagicLinkForPasswordUsers` to the vector for
/// backwards compatibility; emits a startup warning encouraging
/// migration to the vector form.
pub auth_policies: Vec<AuthPolicy>,
/// Allowlist of self-service auth methods offered on the login
/// page and accepted by their respective endpoints. Empty (the
/// default) = both methods allowed, matching legacy behaviour.
/// OIDC is orthogonal — controlled via `OxidcConfig::enabled`.
///
/// Semantics:
/// * `AuthMethod::Password` allowed → `POST /api/auth/login`
/// accepts credentials; password-based `register` works.
/// * `AuthMethod::MagicLink` allowed → `POST /api/auth/magic-
/// link/send` mints tokens; email-only `register` works.
///
/// A method NOT in the list returns 403 with a specific
/// `error_type` (`PasswordLoginDisabled`,
/// `MagicLinkLoginDisabled`) so frontends can render a
/// contextual message rather than a generic auth error.
///
/// Startup guard: when `MagicLink` is in the list but
/// `SmtpConfig::is_enabled()` is false, the server refuses to
/// start. A magic-link policy without a mail sender is a
/// misconfiguration that silently locks users out.
///
/// Env: `OXICLOUD_AUTH_METHODS` (comma-separated:
/// `password,magic_link`). Alias: the older
/// `OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN=true` still removes
/// Password from this list when set (backwards-compat).
pub allowed_auth_methods: Vec<AuthMethod>,
/// Require the user's email to be verified before login is
/// permitted. When `true`, `POST /api/auth/login` returns 403
/// `EmailNotVerified` for any account whose `email_verified_at`
/// is NULL. Users can prove control by clicking a magic-link
/// (which stamps `email_verified_at`) — so this composes with
/// `AuthMethod::MagicLink` in the allowlist above to provide a
/// verification path.
///
/// Admin-created users (`POST /api/admin/users`) and the
/// first-run setup admin (`POST /api/setup`) get
/// `email_verified_at = NOW()` at creation — admin fiat counts
/// as verification, matching the OIDC-JIT convention.
///
/// Env: `OXICLOUD_REQUIRE_VERIFIED_EMAIL` (default `false`).
pub require_verified_email: bool,
}
/// Self-service auth method. Exposed as `AuthConfig::allowed_auth_methods`
/// and parsed from `OXICLOUD_AUTH_METHODS` (comma-separated). OIDC is
/// deliberately excluded — it lives in `OidcConfig` with its own gate.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AuthMethod {
Password,
MagicLink,
}
impl AuthMethod {
/// Case-insensitive parse: accepts `password`, `magic_link`, and the
/// dash form `magic-link` (some operators habitually use dashes).
/// Unknown token returns `None` so the caller can log-and-skip.
pub fn parse(s: &str) -> Option<Self> {
match s.trim().to_ascii_lowercase().as_str() {
"password" => Some(Self::Password),
"magic_link" | "magic-link" | "magiclink" => Some(Self::MagicLink),
_ => None,
}
}
}
/// Additive auth-policy switches. Exposed as `AuthConfig::auth_policies`
/// and parsed from `OXICLOUD_AUTH_POLICIES` (comma-separated). Each
/// variant's name states its own polarity — `Permit...` grants an
/// exception, future `Require...` / `Deny...` variants restrict.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AuthPolicy {
/// Allow magic-link login for accounts that ALSO have a password
/// configured. Off by default — magic-link is otherwise gated by
/// `magic_link_eligibility()` to users without a password
/// (mailbox-strength should not shadow a stronger credential).
/// Enabling this weakens the password to mailbox-strength for
/// affected accounts; opt-in only.
///
/// Deprecated legacy alias: `OXICLOUD_MAGIC_LINK_OPEN_TO_PASSWORD_USERS=true`
/// adds this variant to the vector with a startup warning.
PermitMagicLinkForPasswordUsers,
}
impl AuthPolicy {
/// Case-insensitive parse: accepts `permit_magic_link_for_password_users`
/// (canonical) and the dash form. Unknown token returns `None` so
/// the caller can log-and-skip.
pub fn parse(s: &str) -> Option<Self> {
match s.trim().to_ascii_lowercase().as_str() {
"permit_magic_link_for_password_users" | "permit-magic-link-for-password-users" => {
Some(Self::PermitMagicLinkForPasswordUsers)
}
_ => None,
}
}
}
/// Rate limiting and brute-force protection configuration.
@@ -521,10 +667,30 @@ impl Default for AuthConfig {
hash_time_cost: 3,
hash_parallelism: 2,
rate_limit: RateLimitConfig::default(),
registration_allowed_email_domains: Vec::new(),
auth_policies: Vec::new(),
allowed_auth_methods: vec![AuthMethod::Password, AuthMethod::MagicLink],
require_verified_email: false,
}
}
}
impl AuthConfig {
/// True iff `method` is enabled (or the allowlist is empty — meaning
/// "all methods allowed", matching pre-`OXICLOUD_AUTH_METHODS`
/// behaviour when the operator hasn't opted in yet).
pub fn is_method_allowed(&self, method: AuthMethod) -> bool {
self.allowed_auth_methods.is_empty() || self.allowed_auth_methods.contains(&method)
}
/// True iff `policy` has been opted into via `OXICLOUD_AUTH_POLICIES`
/// (or its legacy alias). Default policies are OFF — the vector is
/// additive only, no invert / defaults.
pub fn has_policy(&self, policy: AuthPolicy) -> bool {
self.auth_policies.contains(&policy)
}
}
/// OpenID Connect (OIDC) configuration
#[derive(Debug, Clone)]
pub struct OidcConfig {
@@ -912,6 +1078,75 @@ pub struct FeaturesConfig {
/// trash/search). OFF by default — opt-in per deployment.
/// Env: `OXICLOUD_ENABLE_EXTERNAL_MOUNTS`.
pub enable_external_mounts: bool,
/// Expose `/api/admin/internal/*` test-only endpoints that trigger
/// background sweeps on demand (storage-usage reconciliation, blob
/// GC). Intended for Hurl / integration tests that need to wait
/// for these maintenance jobs deterministically rather than
/// polling the cached value. Off by default — these endpoints
/// short-circuit the operator-visible cadence, so production
/// deployments don't want them reachable. Env:
/// `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS`.
pub enable_admin_internal_endpoints: bool,
/// Native WebDAV path segment that lists the caller's drives.
///
/// * Default `"@drive"` — bare `/webdav/` addresses the caller's
/// default personal drive (back-compat). Drive listing lives at
/// `/webdav/@drive/`; explicit drive at
/// `/webdav/@drive/<uuid|name>/…`.
/// * `""` (empty) — no default-drive shortcut. Bare `/webdav/`
/// returns the drive listing; explicit drive at
/// `/webdav/<uuid|name>/…`. Operators who don't want a "default
/// drive" concept exposed via WebDAV pick this.
/// * Any other string (e.g. `"drives"`) — same shape as the default,
/// just with that path segment. Loaded via `trim_matches('/')`
/// so operators can safely pass `"/drives/"`.
///
/// Env: `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX`.
pub webdav_drive_listing_prefix: String,
/// Background purge of expired `storage.role_grants` rows.
///
/// The AuthZ engine already filters expired grants out of every
/// permission check at read time (`expires_at IS NULL OR
/// expires_at > NOW()`), so leaving the rows in place is a
/// hygiene issue — not a security one. This purge deletes rows
/// whose `expires_at` is more than [`GrantCleanupConfig::grace_days`]
/// in the past, preserving the audit / support answer to
/// "what happened to my access?" for the grace window.
///
/// Enabled by default: expired-auth-row cleanup is a
/// security-hygiene default, not opt-in.
pub grant_cleanup: GrantCleanupConfig,
}
/// Config for the daily expired-grant purge (see
/// [`FeaturesConfig::grant_cleanup`]).
#[derive(Debug, Clone)]
pub struct GrantCleanupConfig {
/// Master switch. Env: `OXICLOUD_GRANT_CLEANUP_ENABLED`
/// (default `true`).
pub enabled: bool,
/// Days past a grant's `expires_at` before the row is eligible
/// for deletion. Env: `OXICLOUD_GRANT_CLEANUP_GRACE_DAYS`
/// (default `15`).
///
/// The recommendation is `> 15` — enough to answer
/// support/audit questions about recently-lapsed grants without
/// keeping dead rows forever.
pub grace_days: u32,
/// How often the daemon fires, in hours. Env:
/// `OXICLOUD_GRANT_CLEANUP_INTERVAL_HOURS` (default `24`).
pub interval_hours: u64,
}
impl Default for GrantCleanupConfig {
fn default() -> Self {
Self {
enabled: true,
grace_days: 15,
interval_hours: 24,
}
}
}
impl Default for FeaturesConfig {
@@ -928,6 +1163,15 @@ impl Default for FeaturesConfig {
expose_system_users: true, // Expose OxiCloud users as address book by default
enable_video_thumbnails: true, // Video thumbs via ffmpeg (if detected)
enable_external_mounts: false, // External mounts — opt-in, off by default
// Test-only sweep triggers — strictly opt-in. Production
// deployments do NOT need this; the periodic ticker handles
// reconciliation transparently.
enable_admin_internal_endpoints: false,
// Back-compat with pre-multi-drive clients — bare `/webdav/`
// maps to the caller's default drive; drive listing is
// reachable at `/webdav/@drive/`.
webdav_drive_listing_prefix: "@drive".to_string(),
grant_cleanup: GrantCleanupConfig::default(),
}
}
}
@@ -1017,6 +1261,33 @@ impl Default for ContentSearchConfig {
}
}
/// Search-results cache configuration — the per-user results-page cache
/// inside `SearchService`, not the Tantivy content index above.
///
/// The cache is **byte-bounded**: each entry is weighed by the approximate
/// heap size of its result page (see `search_results_entry_weight`) and moka
/// evicts once the summed weight exceeds `max_bytes` — the same byte-budget
/// pattern the file-content cache and the dedup manifest cache use. This
/// replaced an entry-count capacity: with cache keys spanning
/// user × query × offset × limit and up to 500 enriched rows per page, an
/// entry count said nothing about resident memory (1000 entries could pin
/// ~300 MB for the TTL). No entry-count knob is kept — bytes are the only
/// dimension that matters here.
#[derive(Debug, Clone)]
pub struct SearchCacheConfig {
/// Byte budget for cached search-result pages. Default: 32 MiB.
/// Env: `OXICLOUD_SEARCH_CACHE_MAX_BYTES`.
pub max_bytes: u64,
}
impl Default for SearchCacheConfig {
fn default() -> Self {
Self {
max_bytes: 32 * 1024 * 1024,
}
}
}
/// WASM plugin runtime configuration (M0 walking skeleton).
///
/// The runtime is doubly gated: it is only compiled when the `plugins` cargo
@@ -1142,6 +1413,8 @@ pub struct AppConfig {
pub i18n: I18nConfig,
/// Content-search configuration (embedded full-text index)
pub content_search: ContentSearchConfig,
/// Search-results cache configuration (byte-bounded moka cache)
pub search_cache: SearchCacheConfig,
/// WASM plugin runtime configuration
pub plugins: PluginConfig,
/// Face-recognition (People) model configuration
@@ -1198,6 +1471,7 @@ impl Default for AppConfig {
magic_link: MagicLinkConfig::default(),
i18n: I18nConfig::default(),
content_search: ContentSearchConfig::default(),
search_cache: SearchCacheConfig::default(),
plugins: PluginConfig::default(),
faces: FacesConfig::default(),
}
@@ -1437,6 +1711,110 @@ impl AppConfig {
config.auth.rate_limit.lockout_duration_secs = val;
}
// Registration email-domain allowlist. Distinct from
// `OXICLOUD_EXTERNAL_EMAIL_DOMAINS` (which gates who can be
// INVITED via grants + magic link) — this one gates who can
// SELF-register via `POST /api/auth/register`. Empty = no
// restriction. Same parse shape as the external-domains list:
// comma-separated, lowercased, trimmed, empties dropped.
if let Ok(v) = env::var("OXICLOUD_REGISTRATION_ALLOWED_EMAIL_DOMAINS") {
config.auth.registration_allowed_email_domains = v
.split(',')
.map(|d| d.trim().to_ascii_lowercase())
.filter(|d| !d.is_empty())
.collect();
}
// Self-service auth-method allowlist. Empty (unset) = both methods
// allowed. Unknown tokens are logged-and-skipped; a completely
// unparseable value falls back to the default rather than locking
// the operator out. If the resulting list is empty (e.g. the
// operator wrote `OXICLOUD_AUTH_METHODS=nope`), we restore the
// default — a zero-method allowlist would refuse every login.
if let Ok(v) = env::var("OXICLOUD_AUTH_METHODS") {
let methods: Vec<AuthMethod> = v
.split(',')
.filter_map(|s| {
let parsed = AuthMethod::parse(s);
if parsed.is_none() && !s.trim().is_empty() {
eprintln!(
"⚠️ OXICLOUD_AUTH_METHODS: ignoring unknown token '{}' \
(expected: password, magic_link)",
s.trim()
);
}
parsed
})
.collect();
if methods.is_empty() {
eprintln!(
"⚠️ OXICLOUD_AUTH_METHODS parsed to an empty allowlist; \
falling back to default (password, magic_link)"
);
} else {
config.auth.allowed_auth_methods = methods;
}
}
// Legacy alias: OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN=true still
// removes Password from the allowlist. Its main handling in the
// OIDC config block below is preserved for the `login_options`
// response; this line makes the effect apply uniformly through
// `is_method_allowed(Password)` so services don't need to check
// both flags.
if let Ok(v) = env::var("OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN")
&& v.parse::<bool>().unwrap_or(false)
{
config
.auth
.allowed_auth_methods
.retain(|m| *m != AuthMethod::Password);
}
if let Ok(v) = env::var("OXICLOUD_REQUIRE_VERIFIED_EMAIL") {
config.auth.require_verified_email = v.parse::<bool>().unwrap_or(false);
}
// Auth-policy vector. Additive — each recognised token adds a
// variant; unknown tokens are logged-and-skipped so a typo
// doesn't silently zero the whole vector (an operator wanting
// "no policies" simply doesn't set the env var).
//
// The legacy alias
// `OXICLOUD_MAGIC_LINK_OPEN_TO_PASSWORD_USERS=true` is applied
// AFTER this block (see the MagicLinkConfig section below) so a
// deployment setting BOTH env vars ends up with a single copy
// of `PermitMagicLinkForPasswordUsers` regardless of order.
if let Ok(v) = env::var("OXICLOUD_AUTH_POLICIES") {
for token in v.split(',') {
match AuthPolicy::parse(token) {
Some(policy) => {
if !config.auth.auth_policies.contains(&policy) {
config.auth.auth_policies.push(policy);
}
}
None if !token.trim().is_empty() => {
eprintln!(
"⚠️ OXICLOUD_AUTH_POLICIES: ignoring unknown token '{}' \
(known: permit_magic_link_for_password_users)",
token.trim()
);
}
None => {}
}
}
// Reflect the vector into the legacy magic_link config field
// so `magic_link_eligibility()` (the site that reads the
// boolean today) doesn't need to know about the new form.
if config
.auth
.auth_policies
.contains(&AuthPolicy::PermitMagicLinkForPasswordUsers)
{
config.magic_link.open_to_password_users = true;
}
}
// Feature flags
if let Ok(enable_auth) = env::var("OXICLOUD_ENABLE_AUTH").map(|v| v.parse::<bool>())
&& let Ok(val) = enable_auth
@@ -1489,6 +1867,44 @@ impl AppConfig {
config.features.enable_video_thumbnails = val;
}
// `/api/admin/internal/*` test-only triggers. Disabled by
// default; production deployments never need this. The Hurl
// suite flips it on via `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`.
if let Ok(enable_internal) =
env::var("OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS").map(|v| v.parse::<bool>())
&& let Ok(val) = enable_internal
{
config.features.enable_admin_internal_endpoints = val;
}
// Grant-cleanup daemon. Purges rows from `storage.role_grants`
// whose `expires_at` is more than `grace_days` in the past.
// See `GrantCleanupConfig` for defaults + rationale.
if let Ok(v) = env::var("OXICLOUD_GRANT_CLEANUP_ENABLED").map(|v| v.parse::<bool>())
&& let Ok(val) = v
{
config.features.grant_cleanup.enabled = val;
}
if let Ok(v) = env::var("OXICLOUD_GRANT_CLEANUP_GRACE_DAYS").map(|v| v.parse::<u32>())
&& let Ok(val) = v
{
config.features.grant_cleanup.grace_days = val;
}
if let Ok(v) = env::var("OXICLOUD_GRANT_CLEANUP_INTERVAL_HOURS").map(|v| v.parse::<u64>())
&& let Ok(val) = v
{
config.features.grant_cleanup.interval_hours = val.max(1);
}
// Native WebDAV drive-picker path segment. Sanitised by
// stripping leading/trailing slashes so operators can pass
// `/drives/` or `drives` interchangeably; empty string means
// "no default-drive shortcut, `/webdav/` IS the drive listing".
// See `FeaturesConfig::webdav_drive_listing_prefix`.
if let Ok(raw) = env::var("OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX") {
config.features.webdav_drive_listing_prefix = raw.trim_matches('/').to_string();
}
if let Ok(enable_faces) = env::var("OXICLOUD_ENABLE_FACES").map(|v| v.parse::<bool>())
&& let Ok(val) = enable_faces
{
@@ -1566,6 +1982,13 @@ impl AppConfig {
config.content_search.max_text_bytes = val;
}
// Search-results cache (byte-bounded)
if let Ok(v) = env::var("OXICLOUD_SEARCH_CACHE_MAX_BYTES").map(|v| v.parse::<u64>())
&& let Ok(val) = v
{
config.search_cache.max_bytes = val;
}
// WASM plugin runtime
if let Ok(v) = env::var("OXICLOUD_ENABLE_PLUGINS").map(|v| v.parse::<bool>())
&& let Ok(val) = v
@@ -1735,6 +2158,7 @@ impl AppConfig {
account_key: env::var("OXICLOUD_AZURE_ACCOUNT_KEY").unwrap_or_default(),
container,
sas_token: env::var("OXICLOUD_AZURE_SAS_TOKEN").ok(),
endpoint_url: env::var("OXICLOUD_AZURE_ENDPOINT_URL").ok(),
});
}
@@ -1970,8 +2394,29 @@ impl AppConfig {
{
config.magic_link.send_per_ip_per_hour = n;
}
// Legacy alias — writes the same effect as
// `OXICLOUD_AUTH_POLICIES=permit_magic_link_for_password_users`.
// Warn once at boot so operators know to migrate before we drop
// the old var. Kept indefinitely for compat, but the encouraged
// form is the vector.
if let Ok(v) = env::var("OXICLOUD_MAGIC_LINK_OPEN_TO_PASSWORD_USERS") {
config.magic_link.open_to_password_users = v == "true" || v == "1";
let enabled = v == "true" || v == "1";
config.magic_link.open_to_password_users = enabled;
if enabled
&& !config
.auth
.auth_policies
.contains(&AuthPolicy::PermitMagicLinkForPasswordUsers)
{
config
.auth
.auth_policies
.push(AuthPolicy::PermitMagicLinkForPasswordUsers);
}
eprintln!(
"⚠️ OXICLOUD_MAGIC_LINK_OPEN_TO_PASSWORD_USERS is deprecated. \
Use `OXICLOUD_AUTH_POLICIES=permit_magic_link_for_password_users` instead."
);
}
if let Ok(v) = env::var("OXICLOUD_NOTIFY_INTERNAL_USERS_ON_SHARE") {
config.magic_link.notify_internal_users_on_share = v == "true" || v == "1";