feat(thumbnails): server-side video thumbnails via ffmpeg
Videos now get a thumbnail generated eagerly server-side on upload, through the same WebP/blob-hash pipeline as photos — instead of the old browser path that only ran when the Photos grid first rendered a video tile, re-downloaded the whole video to seek a frame, and PUT 3 JPEGs back (and produced nothing at all for HEVC/.mov, which a browser <video> cannot decode). - New VideoFramePort (application) + FfmpegVideoFrameService / NoopVideoFrameService (infrastructure): shell out to the system ffmpeg (no compile-time libav dep), extract one representative frame as PNG, bounded by its own semaphore + a per-process timeout + kill_on_drop. Noop when ffmpeg is absent/disabled, so videos degrade gracefully to no thumbnail. - ThumbnailRefreshHook.on_file_created routes video/* to generate_video_thumbnails_background: stream the (decrypted, reassembled) blob to a size- and time-bounded temp file on the data volume, extract a frame, and reuse the shared render_and_persist_all_webp helper — so video thumbnails are WebP, blob-hash keyed (dedup'd) and content-negotiated, exactly like photos. - GET thumbnail serves the video's WebP to every client (byte-sniffed Content-Type); a genuine miss returns 204. - Config: OXICLOUD_ENABLE_VIDEO_THUMBNAILS (default true, needs ffmpeg detected at startup) + OXICLOUD_FFMPEG_PATH / _CONCURRENCY / _TIMEOUT_SECS / _MAX_MB. - Dockerfile installs ffmpeg in the runtime image. - Frontend: drop the client-side generateVideoThumb/frameFromVideo re-download path; the server is now the source of truth. Benchmark (examples/bench_video_thumbnails.rs, needs ffmpeg): 4/4 codecs incl. HEVC/.mov produce a thumbnail server-side (was 0% for HEVC); ~50-70 ms/frame in the background; ~3.9 KB preview WebP; up to ~23x less per-first-view transfer on the test corpus (far more on real multi-MB clips). Methodology in benches/VIDEO-THUMB.md. Hardening from an adversarial review: video render holds the decode_semaphore like the image path; the ffmpeg scale filter bounds both dimensions; the blob stream has a timeout; the temp file lives on the data volume; the size cap uses saturating_mul. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -429,8 +429,11 @@ impl FileHandler {
|
||||
}
|
||||
};
|
||||
|
||||
// Non-image (video, etc.) with no cached thumbnail → 204
|
||||
if !thumbnail_service.is_supported_image(&file.mime_type) {
|
||||
// Images and videos both store blob-hash thumbnails (videos via an
|
||||
// eagerly-extracted frame); anything else has nothing to thumbnail → 204.
|
||||
let is_image = thumbnail_service.is_supported_image(&file.mime_type);
|
||||
let is_video = file.mime_type.starts_with("video/");
|
||||
if !is_image && !is_video {
|
||||
return Response::builder()
|
||||
.status(StatusCode::NO_CONTENT)
|
||||
.header(header::CACHE_CONTROL, "no-store")
|
||||
@@ -470,6 +473,44 @@ impl FileHandler {
|
||||
.into_response();
|
||||
}
|
||||
|
||||
// Videos: thumbnails are produced eagerly server-side (ffmpeg) on upload,
|
||||
// and persisted WebP-only. Serve that WebP regardless of the negotiated
|
||||
// format — a JPEG/`*/*`/no-Accept client still gets it, correctly labelled
|
||||
// via byte-sniffing — otherwise non-WebP clients would 204 forever despite
|
||||
// a valid thumbnail on disk. We never image-decode a video, so a genuine
|
||||
// miss (generation in flight or unavailable) returns 204.
|
||||
if is_video {
|
||||
if let Some(data) = thumbnail_service
|
||||
.get_cached_thumbnail(
|
||||
&id,
|
||||
Some(&blob_hash),
|
||||
thumb_size.into(),
|
||||
ThumbnailFormat::Webp,
|
||||
)
|
||||
.await
|
||||
{
|
||||
return Response::builder()
|
||||
.status(StatusCode::OK)
|
||||
.header(
|
||||
header::CONTENT_TYPE,
|
||||
crate::common::mime_detect::thumbnail_content_type(&data),
|
||||
)
|
||||
.header(header::CONTENT_LENGTH, data.len())
|
||||
.header(header::CACHE_CONTROL, "public, max-age=31536000, immutable")
|
||||
.header(header::ETAG, &etag)
|
||||
.header(header::VARY, header::ACCEPT.as_str())
|
||||
.body(Body::from(data))
|
||||
.unwrap()
|
||||
.into_response();
|
||||
}
|
||||
return Response::builder()
|
||||
.status(StatusCode::NO_CONTENT)
|
||||
.header(header::CACHE_CONTROL, "no-store")
|
||||
.body(Body::empty())
|
||||
.unwrap()
|
||||
.into_response();
|
||||
}
|
||||
|
||||
match thumbnail_service
|
||||
.get_thumbnail_from_blob(
|
||||
&id,
|
||||
|
||||
Reference in New Issue
Block a user