cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

feat(thumbnails): server-side video thumbnails via ffmpeg

Browse Source 
Videos now get a thumbnail generated eagerly server-side on upload, through
the same WebP/blob-hash pipeline as photos — instead of the old browser path
that only ran when the Photos grid first rendered a video tile, re-downloaded
the whole video to seek a frame, and PUT 3 JPEGs back (and produced nothing at
all for HEVC/.mov, which a browser <video> cannot decode).

- New VideoFramePort (application) + FfmpegVideoFrameService / NoopVideoFrameService
  (infrastructure): shell out to the system ffmpeg (no compile-time libav dep),
  extract one representative frame as PNG, bounded by its own semaphore + a
  per-process timeout + kill_on_drop. Noop when ffmpeg is absent/disabled, so
  videos degrade gracefully to no thumbnail.
- ThumbnailRefreshHook.on_file_created routes video/* to
  generate_video_thumbnails_background: stream the (decrypted, reassembled) blob
  to a size- and time-bounded temp file on the data volume, extract a frame, and
  reuse the shared render_and_persist_all_webp helper — so video thumbnails are
  WebP, blob-hash keyed (dedup'd) and content-negotiated, exactly like photos.
- GET thumbnail serves the video's WebP to every client (byte-sniffed
  Content-Type); a genuine miss returns 204.
- Config: OXICLOUD_ENABLE_VIDEO_THUMBNAILS (default true, needs ffmpeg detected
  at startup) + OXICLOUD_FFMPEG_PATH / _CONCURRENCY / _TIMEOUT_SECS / _MAX_MB.
- Dockerfile installs ffmpeg in the runtime image.
- Frontend: drop the client-side generateVideoThumb/frameFromVideo re-download
  path; the server is now the source of truth.

Benchmark (examples/bench_video_thumbnails.rs, needs ffmpeg): 4/4 codecs incl.
HEVC/.mov produce a thumbnail server-side (was 0% for HEVC); ~50-70 ms/frame in
the background; ~3.9 KB preview WebP; up to ~23x less per-first-view transfer on
the test corpus (far more on real multi-MB clips). Methodology in
benches/VIDEO-THUMB.md.

Hardening from an adversarial review: video render holds the decode_semaphore
like the image path; the ffmpeg scale filter bounds both dimensions; the blob
stream has a timeout; the temp file lives on the data volume; the size cap uses
saturating_mul.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DioCrafts
2026-06-21 23:22:39 +02:00
parent f6f4563f82
commit 5722481c4a
14 changed files with 871 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/common/config.rs
+20 -7
View File
@@ -901,6 +901,11 @@ pub struct FeaturesConfig {
/// Expose other OxiCloud users as a read-only "system" address book
/// at GET /api/address-books. Set to false to hide the user directory.
pub expose_system_users: bool,
/// Generate video thumbnails server-side via `ffmpeg` on upload. When true
/// (and ffmpeg is detected at startup) videos get a representative-frame
/// thumbnail through the same WebP pipeline as photos; otherwise videos have
/// no thumbnail. Env: `OXICLOUD_ENABLE_VIDEO_THUMBNAILS`.
pub enable_video_thumbnails: bool,
}
impl Default for FeaturesConfig {
@@ -908,13 +913,14 @@ impl Default for FeaturesConfig {
Self {
enable_auth: true, // Enable authentication by default
enable_user_storage_quotas: false,
enable_file_sharing: true, // Enable file sharing by default
enable_trash: true, // Enable trash feature
enable_search: true, // Enable search feature
enable_music: true, // Enable music feature
enable_places: true, // Photo map (GET /api/photos/geo + Places tab)
enable_faces: false, // People/faces (biometric) — opt-in, off by default
expose_system_users: true, // Expose OxiCloud users as address book by default
enable_file_sharing: true, // Enable file sharing by default
enable_trash: true, // Enable trash feature
enable_search: true, // Enable search feature
enable_music: true, // Enable music feature
enable_places: true, // Photo map (GET /api/photos/geo + Places tab)
enable_faces: false, // People/faces (biometric) — opt-in, off by default
expose_system_users: true, // Expose OxiCloud users as address book by default
enable_video_thumbnails: true, // Video thumbs via ffmpeg (if detected)
}
}
}
@@ -1469,6 +1475,13 @@ impl AppConfig {
config.features.enable_places = val;
}
if let Ok(enable_video_thumbnails) =
env::var("OXICLOUD_ENABLE_VIDEO_THUMBNAILS").map(|v| v.parse::<bool>())
&& let Ok(val) = enable_video_thumbnails
{
config.features.enable_video_thumbnails = val;
}
if let Ok(enable_faces) = env::var("OXICLOUD_ENABLE_FACES").map(|v| v.parse::<bool>())
&& let Ok(val) = enable_faces
{
src/common/di.rs
+59
View File
@@ -46,6 +46,7 @@ use crate::infrastructure::services::search_index::content_index_worker::Content
use crate::infrastructure::services::search_index::tantivy_content_index::TantivyContentIndex;
use crate::infrastructure::services::trash_cleanup_service::TrashCleanupService;
use crate::application::ports::video_frame_ports::VideoFramePort;
use crate::application::services::app_password_service::AppPasswordService;
use crate::application::services::blob_lifecycle_service::BlobLifecycleService;
use crate::application::services::calendar_service::CalendarService;
@@ -66,6 +67,9 @@ use crate::infrastructure::repositories::pg::{
use crate::infrastructure::services::audio_metadata_service::AudioMetadataService;
use crate::infrastructure::services::chunked_upload_service::ChunkedUploadService;
use crate::infrastructure::services::dedup_service::DedupService;
use crate::infrastructure::services::ffmpeg_video_frame_service::{
FfmpegVideoFrameService, NoopVideoFrameService,
};
use crate::infrastructure::services::image_transcode_service::ImageTranscodeService;
use crate::infrastructure::services::jwt_service::JwtTokenService;
use crate::infrastructure::services::media_metadata_service::MediaMetadataService;
@@ -350,9 +354,64 @@ impl AppServiceFactory {
// ThumbnailRefreshHook: handles FileLifecycleHook events (create/update/delete).
// Implemented on ThumbnailRefreshHook (not ThumbnailService) to avoid circular Arc:
// DedupService → BlobLifecycleService → ThumbnailRefreshHook → DedupService.
// Video frame extractor for thumbnails. Detect ffmpeg once at startup so
// the choice (real extractor vs. no-op) is logged here instead of failing
// per upload.
let video_frame: Arc<dyn VideoFramePort> = {
let ffmpeg_path =
std::env::var("OXICLOUD_FFMPEG_PATH").unwrap_or_else(|_| "ffmpeg".to_string());
if self.config.features.enable_video_thumbnails
&& FfmpegVideoFrameService::is_available(&ffmpeg_path)
{
let cpus = std::thread::available_parallelism()
.map(|n| n.get())
.unwrap_or(4);
let concurrency = std::env::var("OXICLOUD_VIDEO_THUMBNAIL_CONCURRENCY")
.ok()
.and_then(|v| v.parse::<usize>().ok())
.unwrap_or((cpus / 2).max(1));
let timeout = std::time::Duration::from_secs(
std::env::var("OXICLOUD_VIDEO_THUMBNAIL_TIMEOUT_SECS")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(30),
);
tracing::info!(
"🎬 Video thumbnails enabled (ffmpeg '{}', concurrency {})",
ffmpeg_path,
concurrency
);
Arc::new(FfmpegVideoFrameService::new(
ffmpeg_path,
concurrency,
timeout,
))
} else {
if self.config.features.enable_video_thumbnails {
tracing::warn!(
"🎬 Video thumbnails enabled but ffmpeg not found at '{}' \
(set OXICLOUD_FFMPEG_PATH) — videos will have no thumbnail",
ffmpeg_path
);
} else {
tracing::info!("🎬 Video thumbnails disabled");
}
Arc::new(NoopVideoFrameService)
}
};
// Cap on bytes streamed to a temp file for frame extraction (default 2 GB).
// saturating_mul so an absurd MB value can't silently wrap to a tiny cap.
let video_max_bytes: u64 = std::env::var("OXICLOUD_VIDEO_THUMBNAIL_MAX_MB")
.ok()
.and_then(|v| v.parse::<u64>().ok())
.unwrap_or(2048)
.saturating_mul(1024 * 1024);
let thumbnail_refresh_hook = Arc::new(ThumbnailRefreshHook::new(
thumbnail_service.clone(),
dedup_service.clone(),
video_frame,
video_max_bytes,
));
// Build the unified FileLifecycleService dispatcher.