cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

feat(mounts): external file mounts P1 — pluggable provider + read-only REST

Browse Source 
Adds the foundation for external file mounts: admin-configured backends
(raw host filesystem in v1; sftp/webdav/… as future provider kinds) surfaced
as a folder inside a user's drive. Mount contents are virtual/live-passthrough
— read straight from the backend, never stored in storage.files — and are a
deliberately separate, limited storage type (no dedup/sharing/trash/search).
The feature is dark by default (OXICLOUD_ENABLE_EXTERNAL_MOUNTS=false).

P1 scope (this PR): data model, the pluggable provider abstraction, and the
read-only REST surface (mount listing + download). Read-write (P2),
WebDAV/NextCloud path resolution (P3), and the admin UI (P4) follow.

Core model
- Mount root = a real storage.folders row; authorization for everything inside
  collapses onto that folder UUID (ltree-ancestry grant cascade).
- Children are virtual, addressed by ext:<mount_id>:<base64url(node_id)> where
  node_id is provider-owned and opaque to the rest of the system.
- A lock-free (arc-swap) MountRegistry maps mount-root UUID -> provider; a thin
  MountRouter::classify() is the single cheap hook handlers call before parsing
  an id as a UUID. With no mounts configured it always returns Regular, so
  existing code paths are unchanged.

Added
- migrations/20260805000000_external_mounts.sql (storage.external_mounts, kind + config JSONB)
- domain/services/external_mount_id (id envelope + virtual etags)
- application/ports/external_mount_ports (ExternalMountProvider, MountProviderFactory, repo port)
- infrastructure local_fs_mount_provider (tokio::fs, symlink-escape-safe) + factory
- application MountRegistry + MountRouter, pg ExternalMountRepository
- DI wiring (AppState.mount_router), FeaturesConfig.enable_external_mounts
- listing branch (FolderService::list_mount_dir_with_perms + folder_handler) and
  download branch (FileRetrievalService stat/open mount methods + file_handler)

Authorization stays in the service layer (authz.require(Resource::Folder(mount_id)));
handlers only classify. Cross-backend operations are out of scope for P1.

Tests: 529 unit tests + 5 testcontainers integration tests (real Postgres 17),
including end-to-end authorization (owner allowed, stranger denied). Line
coverage of the new modules is 84–100% (cargo-llvm-cov). Known gap:
file_handler::download_mount_file (HTTP glue) needs a full-app test (P4).
This commit is contained in:
Bradley Nelson
2026-06-24 23:52:01 -06:00
parent 1d0fa27991
commit 3c31695579
24 changed files with 3941 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/interfaces/api/handlers/file_handler.rs
+161
View File
@@ -11,13 +11,18 @@ use serde::Deserialize;
use std::collections::HashMap;
use utoipa::ToSchema;
use crate::application::ports::external_mount_ports::MountStat;
use crate::application::ports::file_ports::{
FileManagementUseCase, FileRetrievalUseCase, FileUploadUseCase,
};
use crate::application::ports::storage_ports::{FileReadPort, StorageUsagePort};
use crate::application::ports::thumbnail_ports::ThumbnailPort;
use crate::application::ports::{file_ports::OptimizedFileContent, folder_ports::FolderUseCase};
use crate::application::services::external_mount_router::ResolvedId;
use crate::application::services::mount_registry::MountConfig;
use crate::common::di::AppState;
use crate::domain::errors::DomainError;
use crate::domain::services::external_mount_id::{NodeId, virtual_file_etag};
use crate::interfaces::errors::AppError;
use crate::interfaces::middleware::auth::AuthUser;
use crate::interfaces::range_requests::not_modified_response;
@@ -631,6 +636,22 @@ impl FileHandler {
Query(params): Query<HashMap<String, String>>,
headers: HeaderMap,
) -> impl IntoResponse {
// External mount: download a file living on the provider's backend.
// (A mount-root UUID is a folder and is not downloadable — it falls
// through and 404s as a non-file.)
if let ResolvedId::MountChild { cfg, node_id } = state.mount_router.classify(&id) {
return Self::download_mount_file(
&state,
&cfg,
&node_id,
&id,
auth_user.id,
&params,
&headers,
)
.await;
}
let retrieval = &state.applications.file_retrieval_service;
// ── Get file metadata (ownership-scoped) ────────────────────────
@@ -769,6 +790,146 @@ impl FileHandler {
}
}
/// Download a file living inside an external mount: stat via the provider
/// (authorized against the mount root), then serve metadata / 304 / Range /
/// full stream straight from the backend. No blob cache, dedup, or WebP
/// transcode — mount content is served as-is.
#[allow(clippy::too_many_arguments)]
pub(super) async fn download_mount_file(
state: &AppState,
cfg: &MountConfig,
node_id: &NodeId,
id: &str,
caller_id: uuid::Uuid,
params: &HashMap<String, String>,
headers: &HeaderMap,
) -> axum::response::Response {
let retrieval = &state.applications.file_retrieval_service;
let stat: MountStat = match retrieval
.stat_mount_file_with_perms(cfg, node_id, caller_id)
.await
{
Ok(s) => s,
Err(err) => return AppError::from(err).into_response(),
};
if stat.is_dir {
// Directories are not downloadable through this endpoint.
return AppError::from(DomainError::not_found("File", id)).into_response();
}
let name = node_id
.as_str()
.rsplit('/')
.next()
.unwrap_or_else(|| node_id.as_str());
// ── Metadata-only request ────────────────────────────────────
if params
.get("metadata")
.is_some_and(|v| v == "true" || v == "1")
{
return (
StatusCode::OK,
Json(serde_json::json!({
"id": id,
"name": name,
"size": stat.size,
"mime_type": stat.mime_type,
"modified_at": stat.modified_at,
})),
)
.into_response();
}
let etag = format!("\"{}\"", virtual_file_etag(stat.size, stat.modified_at));
if let Some(resp) = not_modified_response(headers, &etag) {
return resp.into_response();
}
// ── Range Requests ───────────────────────────────────────────
if let Some(range_header) = headers.get(header::RANGE)
&& let Ok(range_str) = range_header.to_str()
&& let Ok(ranges) = parse_range_header(range_str)
{
match ranges.validate(stat.size) {
Ok(valid_ranges) => {
if let Some(range) = valid_ranges.first() {
let start = *range.start();
let end = *range.end();
let range_length = end - start + 1;
let disposition = Self::content_disposition(name, &stat.mime_type, params);
match retrieval
.open_mount_file_with_perms(
cfg,
node_id,
caller_id,
Some((start, Some(end))),
)
.await
{
Ok(stream) => {
return Response::builder()
.status(StatusCode::PARTIAL_CONTENT)
.header(header::CONTENT_TYPE, &stat.mime_type)
.header(header::CONTENT_DISPOSITION, &disposition)
.header(header::CONTENT_LENGTH, range_length)
.header(
header::CONTENT_RANGE,
format!("bytes {}-{}/{}", start, end, stat.size),
)
.header(header::ACCEPT_RANGES, "bytes")
.header(header::ETAG, &etag)
.header(
header::CACHE_CONTROL,
"private, max-age=3600, must-revalidate",
)
.body(Body::from_stream(stream))
.unwrap()
.into_response();
}
Err(err) => {
tracing::error!("Error creating mount range stream: {}", err);
// fall through to full download
}
}
}
}
Err(_) => {
return Response::builder()
.status(StatusCode::RANGE_NOT_SATISFIABLE)
.header(header::CONTENT_RANGE, format!("bytes */{}", stat.size))
.body(Body::empty())
.unwrap()
.into_response();
}
}
}
// ── Normal download ──────────────────────────────────────────
let disposition = Self::content_disposition(name, &stat.mime_type, params);
match retrieval
.open_mount_file_with_perms(cfg, node_id, caller_id, None)
.await
{
Ok(stream) => Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, &stat.mime_type)
.header(header::CONTENT_DISPOSITION, &disposition)
.header(header::CONTENT_LENGTH, stat.size)
.header(header::ETAG, &etag)
.header(
header::CACHE_CONTROL,
"private, max-age=3600, must-revalidate",
)
.header(header::ACCEPT_RANGES, "bytes")
.body(Body::from_stream(stream))
.unwrap()
.into_response(),
Err(err) => AppError::from(err).into_response(),
}
}
// ═══════════════════════════════════════════════════════════════════════
// LIST
// ═══════════════════════════════════════════════════════════════════════
src/interfaces/api/handlers/folder_handler.rs
+201
View File
@@ -17,11 +17,17 @@ use crate::application::dtos::folder_dto::{
ListResourcesOptions, MoveFolderDto, RenameFolderDto,
};
use crate::application::dtos::grant_dto::{ResourceContentDto, ResourceTypeDto};
use crate::application::ports::external_mount_ports::MountEntry;
use crate::application::ports::folder_ports::FolderUseCase;
use crate::application::ports::trash_ports::TrashUseCase;
use crate::application::services::external_mount_router::ResolvedId;
use crate::application::services::folder_service::FolderService;
use crate::application::services::mount_registry::MountConfig;
use crate::common::di::AppState as GlobalAppState;
use crate::domain::entities::file::File;
use crate::domain::services::external_mount_id::{
NodeId, encode_child_id, virtual_file_etag, virtual_folder_etag,
};
use crate::interfaces::errors::AppError;
use crate::interfaces::middleware::auth::AuthUser;
@@ -485,6 +491,28 @@ pub async fn list_folder_resources(
reverse: q.reverse,
};
// External mount branch: a mount-root UUID or an `ext:` id lists live from
// the provider instead of the PostgreSQL UNION. The parent of each entry is
// the requested id itself.
match service.mount_router().classify(&id) {
ResolvedId::MountRoot { cfg } => {
return list_mount_dir_response(
&service,
&cfg,
&NodeId::default(),
&id,
auth_user.id,
opts,
)
.await;
}
ResolvedId::MountChild { cfg, node_id } => {
return list_mount_dir_response(&service, &cfg, &node_id, &id, auth_user.id, opts)
.await;
}
ResolvedId::Regular => {}
}
match service
.list_resources_paged_with_perms(&id, auth_user.id, opts)
.await
@@ -578,3 +606,176 @@ pub async fn list_folder_resources(
Err(e) => AppError::from(e).into_response(),
}
}
/// List one directory inside an external mount and render the standard
/// `/resources` envelope, mapping each live provider entry to a
/// `FolderResourceItemDto` with a synthetic `ext:` id. `parent_id` is the
/// requested id (the directory being listed), which becomes each entry's parent.
async fn list_mount_dir_response(
service: &FolderService,
cfg: &MountConfig,
node_id: &NodeId,
parent_id: &str,
caller_id: uuid::Uuid,
opts: ListResourcesOptions<'_>,
) -> axum::response::Response {
match service
.list_mount_dir_with_perms(cfg, node_id, caller_id, opts)
.await
{
Ok((entries, next_cursor)) => {
let items: Vec<FolderResourceItemDto> = entries
.into_iter()
.map(|entry| mount_entry_to_item(cfg, parent_id, entry))
.collect();
(
StatusCode::OK,
Json(FolderResourcesDto::with_cursor(items, next_cursor)),
)
.into_response()
}
Err(e) => AppError::from(e).into_response(),
}
}
/// Map a live mount entry to a `/resources` item with a synthetic `ext:` id and
/// virtual (size+mtime / mtime) etag. Mount entries have no blob hash.
fn mount_entry_to_item(
cfg: &MountConfig,
parent_id: &str,
entry: MountEntry,
) -> FolderResourceItemDto {
let id = encode_child_id(cfg.mount_id, entry.node_id.clone());
if entry.is_dir {
let dto = FolderDto {
etag: virtual_folder_etag(entry.modified_at),
id,
name: entry.name.clone(),
path: String::new(),
parent_id: Some(parent_id.to_owned()),
owner_id: Some(cfg.owner_id.to_string()),
drive_id: cfg.drive_id,
created_at: entry.created_at,
modified_at: entry.modified_at,
is_root: false,
icon_class: Arc::from("fas fa-folder"),
icon_special_class: Arc::from("folder-icon"),
category: Arc::from("Folder"),
created_by: None,
updated_by: None,
};
FolderResourceItemDto {
resource_type: ResourceTypeDto::Folder,
resource: ResourceContentDto::Folder(dto),
}
} else {
let mime = mime_guess::from_path(&entry.name)
.first_or_octet_stream()
.to_string();
let dto = FileDto {
id,
name: entry.name.clone(),
path: String::new(),
size: entry.size,
mime_type: Arc::from(mime.as_str()),
folder_id: Some(parent_id.to_owned()),
created_at: entry.created_at,
modified_at: entry.modified_at,
icon_class: Arc::from(icon_class_for(&entry.name, &mime)),
icon_special_class: Arc::from(icon_special_class_for(&entry.name, &mime)),
category: Arc::from(category_for(&entry.name, &mime)),
size_formatted: format_file_size(entry.size),
owner_id: Some(cfg.owner_id.to_string()),
sort_date: None,
content_hash: String::new(),
etag: virtual_file_etag(entry.size, entry.modified_at),
created_by: None,
updated_by: None,
};
FolderResourceItemDto {
resource_type: ResourceTypeDto::File,
resource: ResourceContentDto::File(dto),
}
}
}
#[cfg(test)]
mod mount_mapping_tests {
use super::*;
use crate::application::services::mount_registry::MountConfig;
use crate::infrastructure::services::local_fs_mount_provider::LocalFsMountProvider;
use uuid::Uuid;
fn config() -> MountConfig {
let dir = tempfile::tempdir().unwrap();
// Leak the tempdir so the path stays valid for the provider's lifetime;
// the provider is never exercised here (mapping is pure metadata).
let path = dir.keep();
MountConfig {
mount_id: Uuid::new_v4(),
kind: "local_fs".to_string(),
name: "Media".to_string(),
owner_id: Uuid::new_v4(),
drive_id: Uuid::new_v4(),
read_only: false,
mount_path: "Personal/Media".to_string(),
provider: Arc::new(LocalFsMountProvider::new(&path, false).unwrap()),
}
}
fn mount_entry(name: &str, node_id: &str, is_dir: bool, size: u64, mtime: u64) -> MountEntry {
MountEntry {
name: name.to_string(),
node_id: NodeId(node_id.to_string()),
is_dir,
size,
modified_at: mtime,
created_at: mtime,
}
}
#[test]
fn maps_folder_entry_to_item() {
let cfg = config();
let parent = cfg.mount_id.to_string();
let item = mount_entry_to_item(&cfg, &parent, mount_entry("docs", "docs", true, 0, 1234));
assert!(matches!(item.resource_type, ResourceTypeDto::Folder));
let ResourceContentDto::Folder(dto) = item.resource else {
panic!("expected folder");
};
assert_eq!(dto.name, "docs");
// id is the synthetic ext: envelope for (mount_id, node_id).
assert_eq!(dto.id, encode_child_id(cfg.mount_id, "docs"));
assert_eq!(dto.parent_id.as_deref(), Some(parent.as_str()));
assert_eq!(dto.etag, virtual_folder_etag(1234));
assert_eq!(dto.drive_id, cfg.drive_id);
assert!(!dto.is_root);
// Hierarchy is intentionally cleared on this listing.
assert_eq!(dto.path, "");
}
#[test]
fn maps_file_entry_to_item_with_virtual_etag_and_no_hash() {
let cfg = config();
let parent = encode_child_id(cfg.mount_id, "docs");
let item = mount_entry_to_item(
&cfg,
&parent,
mount_entry("report.json", "docs/report.json", false, 42, 999),
);
assert!(matches!(item.resource_type, ResourceTypeDto::File));
let ResourceContentDto::File(dto) = item.resource else {
panic!("expected file");
};
assert_eq!(dto.id, encode_child_id(cfg.mount_id, "docs/report.json"));
assert_eq!(dto.folder_id.as_deref(), Some(parent.as_str()));
assert_eq!(dto.size, 42);
assert_eq!(dto.etag, virtual_file_etag(42, 999));
// Virtual files have no blob hash.
assert_eq!(dto.content_hash, "");
// Mime is sniffed from the name.
assert_eq!(&*dto.mime_type, "application/json");
}
}