feat(thumbnails): server-side video thumbnails via ffmpeg
Videos now get a thumbnail generated eagerly server-side on upload, through the same WebP/blob-hash pipeline as photos — instead of the old browser path that only ran when the Photos grid first rendered a video tile, re-downloaded the whole video to seek a frame, and PUT 3 JPEGs back (and produced nothing at all for HEVC/.mov, which a browser <video> cannot decode). - New VideoFramePort (application) + FfmpegVideoFrameService / NoopVideoFrameService (infrastructure): shell out to the system ffmpeg (no compile-time libav dep), extract one representative frame as PNG, bounded by its own semaphore + a per-process timeout + kill_on_drop. Noop when ffmpeg is absent/disabled, so videos degrade gracefully to no thumbnail. - ThumbnailRefreshHook.on_file_created routes video/* to generate_video_thumbnails_background: stream the (decrypted, reassembled) blob to a size- and time-bounded temp file on the data volume, extract a frame, and reuse the shared render_and_persist_all_webp helper — so video thumbnails are WebP, blob-hash keyed (dedup'd) and content-negotiated, exactly like photos. - GET thumbnail serves the video's WebP to every client (byte-sniffed Content-Type); a genuine miss returns 204. - Config: OXICLOUD_ENABLE_VIDEO_THUMBNAILS (default true, needs ffmpeg detected at startup) + OXICLOUD_FFMPEG_PATH / _CONCURRENCY / _TIMEOUT_SECS / _MAX_MB. - Dockerfile installs ffmpeg in the runtime image. - Frontend: drop the client-side generateVideoThumb/frameFromVideo re-download path; the server is now the source of truth. Benchmark (examples/bench_video_thumbnails.rs, needs ffmpeg): 4/4 codecs incl. HEVC/.mov produce a thumbnail server-side (was 0% for HEVC); ~50-70 ms/frame in the background; ~3.9 KB preview WebP; up to ~23x less per-first-view transfer on the test corpus (far more on real multi-MB clips). Methodology in benches/VIDEO-THUMB.md. Hardening from an adversarial review: video render holds the decode_semaphore like the image path; the ffmpeg scale filter bounds both dimensions; the blob stream has a timeout; the temp file lives on the data volume; the size cap uses saturating_mul. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -26,4 +26,5 @@ pub mod thumbnail_ports;
|
||||
pub mod transcode_ports;
|
||||
pub mod trash_ports;
|
||||
pub mod user_lifecycle;
|
||||
pub mod video_frame_ports;
|
||||
pub mod zip_ports;
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
//! Video frame extraction port.
|
||||
//!
|
||||
//! Pulls a single representative still frame out of a video so the existing
|
||||
//! image thumbnail pipeline (shrink-on-load → SIMD resize → WebP → blob-hash
|
||||
//! storage → HTTP content negotiation) can treat videos exactly like photos —
|
||||
//! eagerly, server-side, on upload. Keeping this behind a port lets the
|
||||
//! composition root swap in a no-op when `ffmpeg` is absent or the feature is
|
||||
//! off, so video uploads degrade gracefully (no thumbnail) instead of erroring.
|
||||
|
||||
use crate::common::errors::DomainError;
|
||||
use async_trait::async_trait;
|
||||
use bytes::Bytes;
|
||||
use std::path::Path;
|
||||
|
||||
/// Extracts a representative still frame from a video file.
|
||||
///
|
||||
/// Implementations:
|
||||
/// - `FfmpegVideoFrameService` — shells out to the system `ffmpeg`, covering
|
||||
/// every container/codec (incl. HEVC/MOV, which a browser `<video>` cannot
|
||||
/// decode). Bounds its own process concurrency and per-call timeout.
|
||||
/// - `NoopVideoFrameService` — registered when `ffmpeg` is unavailable or the
|
||||
/// feature is disabled; `is_supported_video` returns false so the lifecycle
|
||||
/// hook never attempts video thumbnails.
|
||||
#[async_trait]
|
||||
pub trait VideoFramePort: Send + Sync + 'static {
|
||||
/// Whether `mime_type` is a video this extractor will attempt to thumbnail.
|
||||
/// The no-op implementation always returns false.
|
||||
fn is_supported_video(&self, mime_type: &str) -> bool;
|
||||
|
||||
/// Extract one representative frame from the video file at `path`, returning
|
||||
/// encoded **PNG** bytes ready to feed into the image thumbnail renderer.
|
||||
/// `path` must point at the decoded (decrypted, reassembled) video on disk.
|
||||
async fn extract_frame(&self, path: &Path) -> Result<Bytes, DomainError>;
|
||||
}
|
||||
+20
-7
@@ -901,6 +901,11 @@ pub struct FeaturesConfig {
|
||||
/// Expose other OxiCloud users as a read-only "system" address book
|
||||
/// at GET /api/address-books. Set to false to hide the user directory.
|
||||
pub expose_system_users: bool,
|
||||
/// Generate video thumbnails server-side via `ffmpeg` on upload. When true
|
||||
/// (and ffmpeg is detected at startup) videos get a representative-frame
|
||||
/// thumbnail through the same WebP pipeline as photos; otherwise videos have
|
||||
/// no thumbnail. Env: `OXICLOUD_ENABLE_VIDEO_THUMBNAILS`.
|
||||
pub enable_video_thumbnails: bool,
|
||||
}
|
||||
|
||||
impl Default for FeaturesConfig {
|
||||
@@ -908,13 +913,14 @@ impl Default for FeaturesConfig {
|
||||
Self {
|
||||
enable_auth: true, // Enable authentication by default
|
||||
enable_user_storage_quotas: false,
|
||||
enable_file_sharing: true, // Enable file sharing by default
|
||||
enable_trash: true, // Enable trash feature
|
||||
enable_search: true, // Enable search feature
|
||||
enable_music: true, // Enable music feature
|
||||
enable_places: true, // Photo map (GET /api/photos/geo + Places tab)
|
||||
enable_faces: false, // People/faces (biometric) — opt-in, off by default
|
||||
expose_system_users: true, // Expose OxiCloud users as address book by default
|
||||
enable_file_sharing: true, // Enable file sharing by default
|
||||
enable_trash: true, // Enable trash feature
|
||||
enable_search: true, // Enable search feature
|
||||
enable_music: true, // Enable music feature
|
||||
enable_places: true, // Photo map (GET /api/photos/geo + Places tab)
|
||||
enable_faces: false, // People/faces (biometric) — opt-in, off by default
|
||||
expose_system_users: true, // Expose OxiCloud users as address book by default
|
||||
enable_video_thumbnails: true, // Video thumbs via ffmpeg (if detected)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1469,6 +1475,13 @@ impl AppConfig {
|
||||
config.features.enable_places = val;
|
||||
}
|
||||
|
||||
if let Ok(enable_video_thumbnails) =
|
||||
env::var("OXICLOUD_ENABLE_VIDEO_THUMBNAILS").map(|v| v.parse::<bool>())
|
||||
&& let Ok(val) = enable_video_thumbnails
|
||||
{
|
||||
config.features.enable_video_thumbnails = val;
|
||||
}
|
||||
|
||||
if let Ok(enable_faces) = env::var("OXICLOUD_ENABLE_FACES").map(|v| v.parse::<bool>())
|
||||
&& let Ok(val) = enable_faces
|
||||
{
|
||||
|
||||
@@ -46,6 +46,7 @@ use crate::infrastructure::services::search_index::content_index_worker::Content
|
||||
use crate::infrastructure::services::search_index::tantivy_content_index::TantivyContentIndex;
|
||||
use crate::infrastructure::services::trash_cleanup_service::TrashCleanupService;
|
||||
|
||||
use crate::application::ports::video_frame_ports::VideoFramePort;
|
||||
use crate::application::services::app_password_service::AppPasswordService;
|
||||
use crate::application::services::blob_lifecycle_service::BlobLifecycleService;
|
||||
use crate::application::services::calendar_service::CalendarService;
|
||||
@@ -66,6 +67,9 @@ use crate::infrastructure::repositories::pg::{
|
||||
use crate::infrastructure::services::audio_metadata_service::AudioMetadataService;
|
||||
use crate::infrastructure::services::chunked_upload_service::ChunkedUploadService;
|
||||
use crate::infrastructure::services::dedup_service::DedupService;
|
||||
use crate::infrastructure::services::ffmpeg_video_frame_service::{
|
||||
FfmpegVideoFrameService, NoopVideoFrameService,
|
||||
};
|
||||
use crate::infrastructure::services::image_transcode_service::ImageTranscodeService;
|
||||
use crate::infrastructure::services::jwt_service::JwtTokenService;
|
||||
use crate::infrastructure::services::media_metadata_service::MediaMetadataService;
|
||||
@@ -350,9 +354,64 @@ impl AppServiceFactory {
|
||||
// ThumbnailRefreshHook: handles FileLifecycleHook events (create/update/delete).
|
||||
// Implemented on ThumbnailRefreshHook (not ThumbnailService) to avoid circular Arc:
|
||||
// DedupService → BlobLifecycleService → ThumbnailRefreshHook → DedupService.
|
||||
// Video frame extractor for thumbnails. Detect ffmpeg once at startup so
|
||||
// the choice (real extractor vs. no-op) is logged here instead of failing
|
||||
// per upload.
|
||||
let video_frame: Arc<dyn VideoFramePort> = {
|
||||
let ffmpeg_path =
|
||||
std::env::var("OXICLOUD_FFMPEG_PATH").unwrap_or_else(|_| "ffmpeg".to_string());
|
||||
if self.config.features.enable_video_thumbnails
|
||||
&& FfmpegVideoFrameService::is_available(&ffmpeg_path)
|
||||
{
|
||||
let cpus = std::thread::available_parallelism()
|
||||
.map(|n| n.get())
|
||||
.unwrap_or(4);
|
||||
let concurrency = std::env::var("OXICLOUD_VIDEO_THUMBNAIL_CONCURRENCY")
|
||||
.ok()
|
||||
.and_then(|v| v.parse::<usize>().ok())
|
||||
.unwrap_or((cpus / 2).max(1));
|
||||
let timeout = std::time::Duration::from_secs(
|
||||
std::env::var("OXICLOUD_VIDEO_THUMBNAIL_TIMEOUT_SECS")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(30),
|
||||
);
|
||||
tracing::info!(
|
||||
"🎬 Video thumbnails enabled (ffmpeg '{}', concurrency {})",
|
||||
ffmpeg_path,
|
||||
concurrency
|
||||
);
|
||||
Arc::new(FfmpegVideoFrameService::new(
|
||||
ffmpeg_path,
|
||||
concurrency,
|
||||
timeout,
|
||||
))
|
||||
} else {
|
||||
if self.config.features.enable_video_thumbnails {
|
||||
tracing::warn!(
|
||||
"🎬 Video thumbnails enabled but ffmpeg not found at '{}' \
|
||||
(set OXICLOUD_FFMPEG_PATH) — videos will have no thumbnail",
|
||||
ffmpeg_path
|
||||
);
|
||||
} else {
|
||||
tracing::info!("🎬 Video thumbnails disabled");
|
||||
}
|
||||
Arc::new(NoopVideoFrameService)
|
||||
}
|
||||
};
|
||||
// Cap on bytes streamed to a temp file for frame extraction (default 2 GB).
|
||||
// saturating_mul so an absurd MB value can't silently wrap to a tiny cap.
|
||||
let video_max_bytes: u64 = std::env::var("OXICLOUD_VIDEO_THUMBNAIL_MAX_MB")
|
||||
.ok()
|
||||
.and_then(|v| v.parse::<u64>().ok())
|
||||
.unwrap_or(2048)
|
||||
.saturating_mul(1024 * 1024);
|
||||
|
||||
let thumbnail_refresh_hook = Arc::new(ThumbnailRefreshHook::new(
|
||||
thumbnail_service.clone(),
|
||||
dedup_service.clone(),
|
||||
video_frame,
|
||||
video_max_bytes,
|
||||
));
|
||||
|
||||
// Build the unified FileLifecycleService dispatcher.
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
//! ffmpeg-backed video frame extractor (and a no-op fallback).
|
||||
//!
|
||||
//! Shells out to the system `ffmpeg` binary rather than linking libav*: no
|
||||
//! compile-time dependency, no binary bloat, and it decodes every container the
|
||||
//! browser `<video>` element cannot (HEVC/MOV, ProRes, mkv/avi/wmv…). The
|
||||
//! extracted still frame is handed to the existing image thumbnail pipeline, so
|
||||
//! video thumbnails become first-class: WebP, blob-hash keyed (dedup'd), and
|
||||
//! served through the same content negotiation as photos.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use bytes::Bytes;
|
||||
use std::path::Path;
|
||||
use std::process::Stdio;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tokio::process::Command;
|
||||
use tokio::sync::Semaphore;
|
||||
use tokio::time::timeout;
|
||||
|
||||
use crate::application::ports::video_frame_ports::VideoFramePort;
|
||||
use crate::common::errors::DomainError;
|
||||
|
||||
/// PNG file signature — guards against feeding a non-image (or empty) ffmpeg
|
||||
/// output into the renderer.
|
||||
const PNG_MAGIC: &[u8; 8] = b"\x89PNG\r\n\x1a\n";
|
||||
|
||||
/// Extracts a representative frame by invoking `ffmpeg`.
|
||||
pub struct FfmpegVideoFrameService {
|
||||
ffmpeg_path: String,
|
||||
/// Bounds concurrent ffmpeg processes — video decode is CPU-heavy and runs
|
||||
/// outside the async runtime, so it gets its own (smaller) limit rather than
|
||||
/// sharing the image decode semaphore.
|
||||
semaphore: Arc<Semaphore>,
|
||||
/// Per-extraction wall-clock cap; the child is killed on overrun.
|
||||
timeout: Duration,
|
||||
}
|
||||
|
||||
impl FfmpegVideoFrameService {
|
||||
pub fn new(ffmpeg_path: String, concurrency: usize, timeout: Duration) -> Self {
|
||||
Self {
|
||||
ffmpeg_path,
|
||||
semaphore: Arc::new(Semaphore::new(concurrency.max(1))),
|
||||
timeout,
|
||||
}
|
||||
}
|
||||
|
||||
/// Best-effort startup probe: true if `<ffmpeg_path> -version` runs and exits
|
||||
/// 0. Synchronous so the composition root can decide — register the real
|
||||
/// extractor or fall back to [`NoopVideoFrameService`] — without an async
|
||||
/// context, and so a misconfigured path is logged once at boot instead of
|
||||
/// failing per upload.
|
||||
pub fn is_available(ffmpeg_path: &str) -> bool {
|
||||
std::process::Command::new(ffmpeg_path)
|
||||
.arg("-version")
|
||||
.stdin(Stdio::null())
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.status()
|
||||
.map(|s| s.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl VideoFramePort for FfmpegVideoFrameService {
|
||||
fn is_supported_video(&self, mime_type: &str) -> bool {
|
||||
// ffmpeg is the arbiter of what actually decodes; gate broadly on video/*
|
||||
// and let extraction fail gracefully for the rare unsupported container.
|
||||
mime_type.starts_with("video/")
|
||||
}
|
||||
|
||||
async fn extract_frame(&self, path: &Path) -> Result<Bytes, DomainError> {
|
||||
let _permit =
|
||||
self.semaphore.acquire().await.map_err(|_| {
|
||||
DomainError::internal_error("VideoFrame", "extractor semaphore closed")
|
||||
})?;
|
||||
|
||||
// One representative still → PNG on stdout. The `thumbnail` filter scans a
|
||||
// window of frames and picks the most representative one (skipping black
|
||||
// intros) without needing a separate duration probe. Fit within a
|
||||
// 1024×1024 box (preserving aspect) — enough for the 800px `large`
|
||||
// thumbnail, and bounding BOTH dimensions caps the emitted PNG size so a
|
||||
// hostile/extreme geometry can't balloon the buffered output. Arguments
|
||||
// are passed individually (never through a shell), so a hostile file name
|
||||
// cannot inject anything.
|
||||
let run = Command::new(&self.ffmpeg_path)
|
||||
.arg("-nostdin")
|
||||
.arg("-loglevel")
|
||||
.arg("error")
|
||||
.arg("-i")
|
||||
.arg(path)
|
||||
.arg("-vf")
|
||||
.arg("thumbnail,scale=w='min(1024,iw)':h='min(1024,ih)':force_original_aspect_ratio=decrease")
|
||||
.arg("-frames:v")
|
||||
.arg("1")
|
||||
.arg("-f")
|
||||
.arg("image2pipe")
|
||||
.arg("-vcodec")
|
||||
.arg("png")
|
||||
.arg("pipe:1")
|
||||
.stdin(Stdio::null())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.kill_on_drop(true)
|
||||
.output();
|
||||
|
||||
let output = timeout(self.timeout, run)
|
||||
.await
|
||||
.map_err(|_| DomainError::internal_error("VideoFrame", "ffmpeg timed out"))?
|
||||
.map_err(|e| {
|
||||
DomainError::internal_error("VideoFrame", format!("ffmpeg spawn failed: {e}"))
|
||||
})?;
|
||||
|
||||
if !output.status.success() {
|
||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||
return Err(DomainError::internal_error(
|
||||
"VideoFrame",
|
||||
format!("ffmpeg exited {}: {}", output.status, stderr.trim()),
|
||||
));
|
||||
}
|
||||
|
||||
let png = output.stdout;
|
||||
if png.len() < PNG_MAGIC.len() || &png[..PNG_MAGIC.len()] != PNG_MAGIC {
|
||||
return Err(DomainError::internal_error(
|
||||
"VideoFrame",
|
||||
"ffmpeg produced no decodable frame",
|
||||
));
|
||||
}
|
||||
Ok(Bytes::from(png))
|
||||
}
|
||||
}
|
||||
|
||||
/// No-op extractor used when `ffmpeg` is unavailable or video thumbnails are
|
||||
/// disabled. `is_supported_video` returns false so the lifecycle hook never
|
||||
/// attempts generation; videos simply have no thumbnail (the prior behaviour).
|
||||
pub struct NoopVideoFrameService;
|
||||
|
||||
#[async_trait]
|
||||
impl VideoFramePort for NoopVideoFrameService {
|
||||
fn is_supported_video(&self, _mime_type: &str) -> bool {
|
||||
false
|
||||
}
|
||||
|
||||
async fn extract_frame(&self, _path: &Path) -> Result<Bytes, DomainError> {
|
||||
Err(DomainError::internal_error(
|
||||
"VideoFrame",
|
||||
"video thumbnail extraction is disabled (no ffmpeg)",
|
||||
))
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,7 @@ pub mod encrypted_blob_backend;
|
||||
pub mod exif_service;
|
||||
pub mod face_geometry;
|
||||
pub mod face_indexing_service;
|
||||
pub mod ffmpeg_video_frame_service;
|
||||
pub mod file_content_cache;
|
||||
pub mod file_system_i18n_service;
|
||||
pub mod image_transcode_service;
|
||||
|
||||
@@ -25,6 +25,7 @@ use tokio::time::timeout;
|
||||
use crate::application::ports::thumbnail_ports::{
|
||||
ThumbnailFormat, ThumbnailPort, ThumbnailSize as PortThumbnailSize, ThumbnailStatsDto,
|
||||
};
|
||||
use crate::application::ports::video_frame_ports::VideoFramePort;
|
||||
use crate::domain::errors::{DomainError, ErrorKind};
|
||||
use crate::infrastructure::services::dedup_service::DedupService;
|
||||
|
||||
@@ -94,6 +95,11 @@ const WEBP_QUALITY: f32 = 82.0;
|
||||
/// Environment override for the decode-concurrency cap (ops tuning).
|
||||
const DECODE_CONCURRENCY_ENV: &str = "OXICLOUD_THUMBNAIL_DECODE_CONCURRENCY";
|
||||
|
||||
/// Wall-clock cap for streaming a video blob to a temp file before frame
|
||||
/// extraction. Bounds a stalled remote object-store read so the background task
|
||||
/// (and its temp file) can't hang forever.
|
||||
const STREAM_TO_TEMP_TIMEOUT: Duration = Duration::from_secs(120);
|
||||
|
||||
/// Compute max concurrent thumbnail decode operations at runtime.
|
||||
///
|
||||
/// Uses all available CPUs (min 2). Before shrink-on-load each decode
|
||||
@@ -1094,45 +1100,195 @@ impl ThumbnailService {
|
||||
}
|
||||
};
|
||||
|
||||
let results = tokio::task::spawn_blocking(move || {
|
||||
Self::render_all_thumbnails_from_data(original_data.as_ref(), ThumbnailFormat::Webp)
|
||||
})
|
||||
.await;
|
||||
self.render_and_persist_all_webp(&file_id, &blob_hash, original_data)
|
||||
.await;
|
||||
|
||||
let thumbnails = match results {
|
||||
tracing::info!("✅ Background thumbnail generation complete: {}", file_id);
|
||||
});
|
||||
}
|
||||
|
||||
/// Render every size as WebP from a decoded source image and persist them by
|
||||
/// blob_hash (disk `{hash}.webp` + moka). Shared by the image upload path and
|
||||
/// the video path (which passes the extracted frame as the source), so both
|
||||
/// produce identical, dedup-able, content-negotiable thumbnails.
|
||||
async fn render_and_persist_all_webp(&self, file_id: &str, blob_hash: &str, source: Bytes) {
|
||||
let results = tokio::task::spawn_blocking(move || {
|
||||
Self::render_all_thumbnails_from_data(source.as_ref(), ThumbnailFormat::Webp)
|
||||
})
|
||||
.await;
|
||||
|
||||
let thumbnails = match results {
|
||||
Ok(Ok(t)) => t,
|
||||
Ok(Err(e)) => {
|
||||
tracing::warn!("Thumbnail generation failed for {}: {}", file_id, e);
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!("Thumbnail task panicked for {}: {}", file_id, e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
for (size, bytes) in thumbnails {
|
||||
let thumb_path = self.get_thumbnail_path(blob_hash, size, ThumbnailFormat::Webp);
|
||||
if let Some(parent) = thumb_path.parent() {
|
||||
let _ = fs::create_dir_all(parent).await;
|
||||
}
|
||||
if let Err(e) = fs::write(&thumb_path, &bytes).await {
|
||||
tracing::warn!("Failed to save thumbnail {} {:?}: {}", file_id, size, e);
|
||||
} else {
|
||||
let cache_key = ThumbnailCacheKey {
|
||||
file_id: file_id.to_string(),
|
||||
size,
|
||||
format: ThumbnailFormat::Webp,
|
||||
};
|
||||
self.cache.insert(cache_key, bytes).await;
|
||||
tracing::debug!("✅ Generated thumbnail: {} {:?}", file_id, size);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Eagerly generate WebP thumbnails for a freshly-uploaded video.
|
||||
///
|
||||
/// Streams the (decrypted, reassembled) blob to a temp file — bounded by
|
||||
/// `max_bytes` so a giant upload never materialises in full — extracts one
|
||||
/// representative frame via `video_frame`, then runs it through the same
|
||||
/// WebP/blob-hash pipeline as photos. Any miss or failure leaves the video
|
||||
/// without a thumbnail (the prior behaviour), never an error to the user.
|
||||
pub fn generate_video_thumbnails_background(
|
||||
self: Arc<Self>,
|
||||
file_id: String,
|
||||
blob_hash: String,
|
||||
dedup: Arc<DedupService>,
|
||||
video_frame: Arc<dyn VideoFramePort>,
|
||||
max_bytes: u64,
|
||||
) {
|
||||
tokio::spawn(async move {
|
||||
// Blob deleted before this task ran → nothing to do.
|
||||
if !dedup.blob_exists(&blob_hash).await {
|
||||
return;
|
||||
}
|
||||
|
||||
// Dedup hit: thumbnails for this content already exist on disk — just
|
||||
// warm the moka cache (zero CPU), mirroring the image path.
|
||||
let all_exist = {
|
||||
let mut ok = true;
|
||||
for size in ThumbnailSize::all() {
|
||||
let p = self.get_thumbnail_path(&blob_hash, *size, ThumbnailFormat::Webp);
|
||||
if fs::metadata(&p).await.is_err() {
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
ok
|
||||
};
|
||||
if all_exist {
|
||||
for size in ThumbnailSize::all() {
|
||||
let p = self.get_thumbnail_path(&blob_hash, *size, ThumbnailFormat::Webp);
|
||||
if let Ok(data) = fs::read(&p).await {
|
||||
let key = ThumbnailCacheKey {
|
||||
file_id: file_id.clone(),
|
||||
size: *size,
|
||||
format: ThumbnailFormat::Webp,
|
||||
};
|
||||
self.cache.insert(key, Bytes::from(data)).await;
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// ffmpeg needs a seekable file, and the blob may be encrypted/chunked
|
||||
// on disk — stream through the normal decrypting read path into a
|
||||
// bounded temp file (on the data volume) rather than reading the raw
|
||||
// blob path. Bounded by size (max_bytes) AND time, so a stalled remote
|
||||
// backend can't hang the task or leak its temp file forever.
|
||||
let tmp = match tokio::time::timeout(
|
||||
STREAM_TO_TEMP_TIMEOUT,
|
||||
Self::stream_blob_to_temp(&dedup, &blob_hash, max_bytes, &self.thumbnails_root),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(Ok(t)) => t,
|
||||
Ok(Err(e)) => {
|
||||
tracing::warn!("Thumbnail generation failed for {}: {}", file_id, e);
|
||||
tracing::warn!("🎬 video thumb: stream {} failed: {}", file_id, e);
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!("Thumbnail task panicked for {}: {}", file_id, e);
|
||||
Err(_) => {
|
||||
tracing::warn!("🎬 video thumb: stream {} timed out", file_id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
for (size, bytes) in thumbnails {
|
||||
let thumb_path = self.get_thumbnail_path(&blob_hash, size, ThumbnailFormat::Webp);
|
||||
if let Some(parent) = thumb_path.parent() {
|
||||
let _ = fs::create_dir_all(parent).await;
|
||||
let frame = match video_frame.extract_frame(tmp.path()).await {
|
||||
Ok(f) => f,
|
||||
Err(e) => {
|
||||
tracing::info!("🎬 video thumb: extract {} skipped: {}", file_id, e);
|
||||
return;
|
||||
}
|
||||
if let Err(e) = fs::write(&thumb_path, &bytes).await {
|
||||
tracing::warn!("Failed to save thumbnail {} {:?}: {}", file_id, size, e);
|
||||
} else {
|
||||
let cache_key = ThumbnailCacheKey {
|
||||
file_id: file_id.clone(),
|
||||
size,
|
||||
format: ThumbnailFormat::Webp,
|
||||
};
|
||||
self.cache.insert(cache_key, bytes).await;
|
||||
tracing::debug!("✅ Generated thumbnail: {} {:?}", file_id, size);
|
||||
}
|
||||
}
|
||||
};
|
||||
drop(tmp); // remove the temp video promptly; we have the frame bytes
|
||||
|
||||
tracing::info!("✅ Background thumbnail generation complete: {}", file_id);
|
||||
// Bound the CPU-bound decode+resize with the same decode_semaphore the
|
||||
// image path uses (the ffmpeg extractor's own permit covered only
|
||||
// extraction and is already released), so concurrent video uploads
|
||||
// can't spawn unbounded render jobs.
|
||||
let _permit = match self.decode_semaphore.acquire().await {
|
||||
Ok(p) => p,
|
||||
Err(_) => return,
|
||||
};
|
||||
self.render_and_persist_all_webp(&file_id, &blob_hash, frame)
|
||||
.await;
|
||||
tracing::info!("✅ Video thumbnail generation complete: {}", file_id);
|
||||
});
|
||||
}
|
||||
|
||||
/// Stream a blob through the decrypting/CDC-aware read path into a temp file,
|
||||
/// aborting if it exceeds `max_bytes`. The returned handle deletes the file
|
||||
/// on drop. Bounded memory: chunks are written straight to disk, never
|
||||
/// buffered whole.
|
||||
async fn stream_blob_to_temp(
|
||||
dedup: &DedupService,
|
||||
blob_hash: &str,
|
||||
max_bytes: u64,
|
||||
temp_dir: &Path,
|
||||
) -> Result<tempfile::NamedTempFile, DomainError> {
|
||||
use futures::StreamExt;
|
||||
use tokio::io::AsyncWriteExt;
|
||||
|
||||
let mut stream = dedup.read_blob_stream(blob_hash).await?;
|
||||
// Colocate the temp video with the data volume (sized for blobs) instead
|
||||
// of the OS temp dir, which can be small or a tmpfs in containers.
|
||||
let tmp = tempfile::Builder::new()
|
||||
.prefix("oxithumb-")
|
||||
.tempfile_in(temp_dir)
|
||||
.map_err(|e| DomainError::internal_error("VideoFrame", format!("temp file: {e}")))?;
|
||||
let mut out = tokio::fs::File::create(tmp.path())
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("VideoFrame", format!("temp open: {e}")))?;
|
||||
|
||||
let mut written: u64 = 0;
|
||||
while let Some(chunk) = stream.next().await {
|
||||
let chunk = chunk.map_err(|e| {
|
||||
DomainError::internal_error("VideoFrame", format!("blob read: {e}"))
|
||||
})?;
|
||||
written += chunk.len() as u64;
|
||||
if written > max_bytes {
|
||||
return Err(DomainError::internal_error(
|
||||
"VideoFrame",
|
||||
format!("video exceeds {max_bytes}-byte thumbnail cap"),
|
||||
));
|
||||
}
|
||||
out.write_all(&chunk).await.map_err(|e| {
|
||||
DomainError::internal_error("VideoFrame", format!("temp write: {e}"))
|
||||
})?;
|
||||
}
|
||||
out.flush()
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("VideoFrame", format!("temp flush: {e}")))?;
|
||||
drop(out); // close our write handle before ffmpeg opens the path
|
||||
Ok(tmp)
|
||||
}
|
||||
|
||||
/// Delete thumbnails for a file.
|
||||
///
|
||||
/// Only invalidates the in-memory moka cache (keyed by file_id).
|
||||
@@ -1205,11 +1361,26 @@ impl ThumbnailService {
|
||||
pub struct ThumbnailRefreshHook {
|
||||
thumbnail: Arc<ThumbnailService>,
|
||||
dedup: Arc<DedupService>,
|
||||
/// Video frame extractor (ffmpeg, or a no-op when unavailable/disabled).
|
||||
video_frame: Arc<dyn VideoFramePort>,
|
||||
/// Max bytes streamed to a temp file for video frame extraction; larger
|
||||
/// videos are skipped (no thumbnail) rather than materialised in full.
|
||||
video_max_bytes: u64,
|
||||
}
|
||||
|
||||
impl ThumbnailRefreshHook {
|
||||
pub fn new(thumbnail: Arc<ThumbnailService>, dedup: Arc<DedupService>) -> Self {
|
||||
Self { thumbnail, dedup }
|
||||
pub fn new(
|
||||
thumbnail: Arc<ThumbnailService>,
|
||||
dedup: Arc<DedupService>,
|
||||
video_frame: Arc<dyn VideoFramePort>,
|
||||
video_max_bytes: u64,
|
||||
) -> Self {
|
||||
Self {
|
||||
thumbnail,
|
||||
dedup,
|
||||
video_frame,
|
||||
video_max_bytes,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1222,16 +1393,28 @@ impl crate::application::ports::file_lifecycle::FileLifecycleHook for ThumbnailR
|
||||
is_new_blob: bool,
|
||||
) {
|
||||
// Blob-hash thumbnail already exists on disk when is_new_blob=false — skip.
|
||||
if !is_new_blob || !ThumbnailService::is_supported_image(content_type) {
|
||||
if !is_new_blob {
|
||||
return;
|
||||
}
|
||||
self.thumbnail
|
||||
.clone()
|
||||
.generate_all_sizes_background_from_blob(
|
||||
if ThumbnailService::is_supported_image(content_type) {
|
||||
self.thumbnail
|
||||
.clone()
|
||||
.generate_all_sizes_background_from_blob(
|
||||
file_id.to_string(),
|
||||
blob_hash.to_string(),
|
||||
self.dedup.clone(),
|
||||
);
|
||||
} else if self.video_frame.is_supported_video(content_type) {
|
||||
// Videos: extract a frame server-side (ffmpeg) and run it through the
|
||||
// same WebP/blob-hash pipeline — eager, off the request path.
|
||||
self.thumbnail.clone().generate_video_thumbnails_background(
|
||||
file_id.to_string(),
|
||||
blob_hash.to_string(),
|
||||
self.dedup.clone(),
|
||||
self.video_frame.clone(),
|
||||
self.video_max_bytes,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn on_file_copied(
|
||||
|
||||
@@ -429,8 +429,11 @@ impl FileHandler {
|
||||
}
|
||||
};
|
||||
|
||||
// Non-image (video, etc.) with no cached thumbnail → 204
|
||||
if !thumbnail_service.is_supported_image(&file.mime_type) {
|
||||
// Images and videos both store blob-hash thumbnails (videos via an
|
||||
// eagerly-extracted frame); anything else has nothing to thumbnail → 204.
|
||||
let is_image = thumbnail_service.is_supported_image(&file.mime_type);
|
||||
let is_video = file.mime_type.starts_with("video/");
|
||||
if !is_image && !is_video {
|
||||
return Response::builder()
|
||||
.status(StatusCode::NO_CONTENT)
|
||||
.header(header::CACHE_CONTROL, "no-store")
|
||||
@@ -470,6 +473,44 @@ impl FileHandler {
|
||||
.into_response();
|
||||
}
|
||||
|
||||
// Videos: thumbnails are produced eagerly server-side (ffmpeg) on upload,
|
||||
// and persisted WebP-only. Serve that WebP regardless of the negotiated
|
||||
// format — a JPEG/`*/*`/no-Accept client still gets it, correctly labelled
|
||||
// via byte-sniffing — otherwise non-WebP clients would 204 forever despite
|
||||
// a valid thumbnail on disk. We never image-decode a video, so a genuine
|
||||
// miss (generation in flight or unavailable) returns 204.
|
||||
if is_video {
|
||||
if let Some(data) = thumbnail_service
|
||||
.get_cached_thumbnail(
|
||||
&id,
|
||||
Some(&blob_hash),
|
||||
thumb_size.into(),
|
||||
ThumbnailFormat::Webp,
|
||||
)
|
||||
.await
|
||||
{
|
||||
return Response::builder()
|
||||
.status(StatusCode::OK)
|
||||
.header(
|
||||
header::CONTENT_TYPE,
|
||||
crate::common::mime_detect::thumbnail_content_type(&data),
|
||||
)
|
||||
.header(header::CONTENT_LENGTH, data.len())
|
||||
.header(header::CACHE_CONTROL, "public, max-age=31536000, immutable")
|
||||
.header(header::ETAG, &etag)
|
||||
.header(header::VARY, header::ACCEPT.as_str())
|
||||
.body(Body::from(data))
|
||||
.unwrap()
|
||||
.into_response();
|
||||
}
|
||||
return Response::builder()
|
||||
.status(StatusCode::NO_CONTENT)
|
||||
.header(header::CACHE_CONTROL, "no-store")
|
||||
.body(Body::empty())
|
||||
.unwrap()
|
||||
.into_response();
|
||||
}
|
||||
|
||||
match thumbnail_service
|
||||
.get_thumbnail_from_blob(
|
||||
&id,
|
||||
|
||||
Reference in New Issue
Block a user