feat(thumbnails): server-side video thumbnails via ffmpeg
Videos now get a thumbnail generated eagerly server-side on upload, through the same WebP/blob-hash pipeline as photos — instead of the old browser path that only ran when the Photos grid first rendered a video tile, re-downloaded the whole video to seek a frame, and PUT 3 JPEGs back (and produced nothing at all for HEVC/.mov, which a browser <video> cannot decode). - New VideoFramePort (application) + FfmpegVideoFrameService / NoopVideoFrameService (infrastructure): shell out to the system ffmpeg (no compile-time libav dep), extract one representative frame as PNG, bounded by its own semaphore + a per-process timeout + kill_on_drop. Noop when ffmpeg is absent/disabled, so videos degrade gracefully to no thumbnail. - ThumbnailRefreshHook.on_file_created routes video/* to generate_video_thumbnails_background: stream the (decrypted, reassembled) blob to a size- and time-bounded temp file on the data volume, extract a frame, and reuse the shared render_and_persist_all_webp helper — so video thumbnails are WebP, blob-hash keyed (dedup'd) and content-negotiated, exactly like photos. - GET thumbnail serves the video's WebP to every client (byte-sniffed Content-Type); a genuine miss returns 204. - Config: OXICLOUD_ENABLE_VIDEO_THUMBNAILS (default true, needs ffmpeg detected at startup) + OXICLOUD_FFMPEG_PATH / _CONCURRENCY / _TIMEOUT_SECS / _MAX_MB. - Dockerfile installs ffmpeg in the runtime image. - Frontend: drop the client-side generateVideoThumb/frameFromVideo re-download path; the server is now the source of truth. Benchmark (examples/bench_video_thumbnails.rs, needs ffmpeg): 4/4 codecs incl. HEVC/.mov produce a thumbnail server-side (was 0% for HEVC); ~50-70 ms/frame in the background; ~3.9 KB preview WebP; up to ~23x less per-first-view transfer on the test corpus (far more on real multi-MB clips). Methodology in benches/VIDEO-THUMB.md. Hardening from an adversarial review: video render holds the decode_semaphore like the image path; the ffmpeg scale filter bounds both dimensions; the blob stream has a timeout; the temp file lives on the data volume; the size cap uses saturating_mul. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -155,6 +155,13 @@ name = "bench_thumbnails_mem"
|
|||||||
path = "examples/bench_thumbnails_mem.rs"
|
path = "examples/bench_thumbnails_mem.rs"
|
||||||
required-features = ["bench"]
|
required-features = ["bench"]
|
||||||
|
|
||||||
|
# Video thumbnail benchmark — Option B (server-side ffmpeg frame → WebP). Needs
|
||||||
|
# `ffmpeg` on PATH (libx264/libx265/libvpx-vp9 to synthesize the test corpus).
|
||||||
|
[[example]]
|
||||||
|
name = "bench_video_thumbnails"
|
||||||
|
path = "examples/bench_video_thumbnails.rs"
|
||||||
|
required-features = ["bench"]
|
||||||
|
|
||||||
# DB connection-pool tail-latency benchmark (needs the dev Postgres up).
|
# DB connection-pool tail-latency benchmark (needs the dev Postgres up).
|
||||||
[[example]]
|
[[example]]
|
||||||
name = "bench_db_pool"
|
name = "bench_db_pool"
|
||||||
|
|||||||
+3
-1
@@ -67,9 +67,11 @@ LABEL org.opencontainers.image.title="OxiCloud" \
|
|||||||
|
|
||||||
# Install only necessary runtime dependencies and update packages
|
# Install only necessary runtime dependencies and update packages
|
||||||
# su-exec is needed by the entrypoint to drop privileges after fixing volume permissions.
|
# su-exec is needed by the entrypoint to drop privileges after fixing volume permissions.
|
||||||
|
# ffmpeg powers server-side video thumbnail extraction (one frame → WebP pipeline);
|
||||||
|
# without it videos simply have no thumbnail (OXICLOUD_ENABLE_VIDEO_THUMBNAILS).
|
||||||
# No libpq: the pure-Rust sqlx postgres driver never links it.
|
# No libpq: the pure-Rust sqlx postgres driver never links it.
|
||||||
RUN apk --no-cache upgrade && \
|
RUN apk --no-cache upgrade && \
|
||||||
apk add --no-cache libgcc ca-certificates tzdata su-exec && \
|
apk add --no-cache libgcc ca-certificates tzdata su-exec ffmpeg && \
|
||||||
addgroup -g 1001 -S oxicloud && \
|
addgroup -g 1001 -S oxicloud && \
|
||||||
adduser -u 1001 -S oxicloud -G oxicloud
|
adduser -u 1001 -S oxicloud -G oxicloud
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,88 @@
|
|||||||
|
# Video thumbnails — server-side ffmpeg (Option B)
|
||||||
|
|
||||||
|
Video thumbnails are now generated **server-side on upload**: a lifecycle hook
|
||||||
|
streams the (decrypted) blob to a temp file, `ffmpeg` extracts one representative
|
||||||
|
frame, and that frame goes through the **same WebP pipeline as photos** — so
|
||||||
|
video thumbnails are WebP, blob-hash keyed (dedup'd), and content-negotiated,
|
||||||
|
exactly like images.
|
||||||
|
|
||||||
|
This replaces the old browser path, which only generated a thumbnail when the
|
||||||
|
Photos grid first rendered a video tile, the `<img>` 404'd, and the browser
|
||||||
|
**re-downloaded the video** to seek a frame and PUT 3 JPEGs back.
|
||||||
|
|
||||||
|
## What it buys
|
||||||
|
|
||||||
|
1. **Coverage incl. HEVC/iPhone.** The browser `<video>` element cannot decode
|
||||||
|
HEVC/H.265 (`.mov` from iPhones), ProRes, many mkv/avi — so the old path
|
||||||
|
produced **no** thumbnail for them. ffmpeg decodes all of them.
|
||||||
|
2. **No client re-download.** The frame is taken server-side from the blob the
|
||||||
|
server already has — the browser never pulls the video back.
|
||||||
|
3. **Eager.** Thumbnails are ready before the gallery asks; tiles paint
|
||||||
|
immediately instead of waiting on a failed `<img>` + re-download cascade.
|
||||||
|
|
||||||
|
## Reproduce
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run --release --features bench --example bench_video_thumbnails
|
||||||
|
```
|
||||||
|
|
||||||
|
Requires `ffmpeg` on PATH (with libx264/libx265/libvpx-vp9 to synthesize the
|
||||||
|
test corpus — written to `benches/corpus/`, git-ignored). The corpus is a
|
||||||
|
`testsrc` pattern at several codecs/resolutions, incl. an HEVC `.mov`.
|
||||||
|
|
||||||
|
## Results (14 cores, ffmpeg 8.1)
|
||||||
|
|
||||||
|
| case | video KB | extract ms | frame KB | icon | preview | large (WebP B) | ok |
|
||||||
|
|-----------------|---------:|-----------:|---------:|-----:|--------:|---------------:|----|
|
||||||
|
| h264 720p | 46.9 | 49.2 | 44.1 | 1818 | 3836 | 6844 | ✅ |
|
||||||
|
| h264 1080p | 76.1 | 64.9 | 37.9 | 1844 | 3888 | 6870 | ✅ |
|
||||||
|
| HEVC 1080p .mov | 38.6 | 69.5 | 51.9 | 1858 | 3928 | 6976 | ✅ |
|
||||||
|
| VP9 720p .webm | 187.8 | 67.8 | 14.6 | 1824 | 3814 | 6650 | ✅ |
|
||||||
|
|
||||||
|
- **Coverage: 4/4 codecs, including HEVC/.mov** — the browser path produced 0 for
|
||||||
|
HEVC. Going from "no thumbnail" to "a thumbnail" is the real headline for
|
||||||
|
iPhone footage.
|
||||||
|
- **Extraction: ~50–70 ms/frame** for 720p–1080p. Paid once, in a background task,
|
||||||
|
per unique blob — never on the request path. Bounded by a per-process timeout
|
||||||
|
and a dedicated concurrency semaphore.
|
||||||
|
- **Served bytes per tile: ~3.8 KB (preview WebP)** — same compact WebP as photos.
|
||||||
|
|
||||||
|
### Transfer per first view (the bandwidth win)
|
||||||
|
|
||||||
|
```
|
||||||
|
OLD (browser re-downloads the video, worst case): 349 KB for 4 tiles + 3 JPEG PUTs/video
|
||||||
|
NEW (fetch the server WebP preview): 15.1 KB + 0 client decode
|
||||||
|
→ ~23× less data on this corpus.
|
||||||
|
```
|
||||||
|
|
||||||
|
> ⚠️ The test clips are tiny (3 s `testsrc`, 38–188 KB), which **understates** the
|
||||||
|
> win enormously. Real phone videos are 10–100+ MB; the old path re-downloaded a
|
||||||
|
> large fraction of that per first view, vs ~4 KB now — i.e. thousands-fold less
|
||||||
|
> for a 50 MB clip, plus it works for HEVC at all.
|
||||||
|
|
||||||
|
## How it's wired
|
||||||
|
|
||||||
|
- `application/ports/video_frame_ports.rs` — `VideoFramePort` (extract one PNG
|
||||||
|
frame from a video file).
|
||||||
|
- `infrastructure/services/ffmpeg_video_frame_service.rs` — shells out to the
|
||||||
|
system `ffmpeg` (no compile-time libav dep); `NoopVideoFrameService` when
|
||||||
|
ffmpeg is absent/disabled, so videos degrade to "no thumbnail" gracefully.
|
||||||
|
- `ThumbnailRefreshHook::on_file_created` routes `video/*` to
|
||||||
|
`generate_video_thumbnails_background`, which streams the blob to a temp file
|
||||||
|
(capped, decrypting), extracts a frame, and reuses `render_and_persist_all_webp`.
|
||||||
|
- GET `/api/files/{id}/thumbnail/{size}` serves the blob-hash WebP for videos
|
||||||
|
too; a miss returns 204 (generation in flight / unavailable).
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
| Env | Default | Meaning |
|
||||||
|
|---|---|---|
|
||||||
|
| `OXICLOUD_ENABLE_VIDEO_THUMBNAILS` | `true` | Master switch (also needs ffmpeg present). |
|
||||||
|
| `OXICLOUD_FFMPEG_PATH` | `ffmpeg` | Path to the ffmpeg binary. |
|
||||||
|
| `OXICLOUD_VIDEO_THUMBNAIL_CONCURRENCY` | `cpus/2` | Max concurrent ffmpeg processes. |
|
||||||
|
| `OXICLOUD_VIDEO_THUMBNAIL_TIMEOUT_SECS` | `30` | Per-extraction wall-clock cap. |
|
||||||
|
| `OXICLOUD_VIDEO_THUMBNAIL_MAX_MB` | `2048` | Skip videos larger than this (no temp materialise). |
|
||||||
|
|
||||||
|
The Docker runtime image installs `ffmpeg`. Existing videos uploaded before this
|
||||||
|
change get a thumbnail the next time their blob is (re)created; a backfill task
|
||||||
|
is a possible follow-up.
|
||||||
@@ -0,0 +1,232 @@
|
|||||||
|
//! Video thumbnail benchmark — Option B (server-side ffmpeg frame → WebP).
|
||||||
|
//!
|
||||||
|
//! Measures the "after" of moving video thumbnail generation off the browser
|
||||||
|
//! and onto the server:
|
||||||
|
//! * extraction time per codec/resolution (the new server cost),
|
||||||
|
//! * the WebP thumbnail bytes actually served per tile (the new transfer),
|
||||||
|
//! * codec coverage incl. HEVC/MOV — the iPhone case a browser `<video>`
|
||||||
|
//! cannot decode, so the old client-side path produced *no* thumbnail.
|
||||||
|
//!
|
||||||
|
//! vs the OLD client-side path, whose first view of each video tile re-downloaded
|
||||||
|
//! the video from the server (metadata + byte-ranges, up to the whole file) and
|
||||||
|
//! PUT 3 JPEGs back.
|
||||||
|
//!
|
||||||
|
//! Requires `ffmpeg` on PATH (with libx264/libx265/libvpx-vp9 to generate the
|
||||||
|
//! corpus). Run: `cargo run --release --features bench --example bench_video_thumbnails`.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
use oxicloud::application::ports::thumbnail_ports::ThumbnailFormat;
|
||||||
|
use oxicloud::application::ports::video_frame_ports::VideoFramePort;
|
||||||
|
use oxicloud::infrastructure::services::ffmpeg_video_frame_service::FfmpegVideoFrameService;
|
||||||
|
use oxicloud::infrastructure::services::thumbnail_service::{ThumbnailService, ThumbnailSize};
|
||||||
|
|
||||||
|
/// One synthetic test video: a label, output filename, and the ffmpeg encode
|
||||||
|
/// args (a `testsrc` pattern keeps it license-free and deterministic enough).
|
||||||
|
struct VideoSpec {
|
||||||
|
name: &'static str,
|
||||||
|
filename: &'static str,
|
||||||
|
encode_args: &'static [&'static str],
|
||||||
|
}
|
||||||
|
|
||||||
|
const SPECS: &[VideoSpec] = &[
|
||||||
|
VideoSpec {
|
||||||
|
name: "h264 720p",
|
||||||
|
filename: "video_h264_720p.mp4",
|
||||||
|
encode_args: &[
|
||||||
|
"-f",
|
||||||
|
"lavfi",
|
||||||
|
"-i",
|
||||||
|
"testsrc=duration=3:size=1280x720:rate=30",
|
||||||
|
"-c:v",
|
||||||
|
"libx264",
|
||||||
|
"-pix_fmt",
|
||||||
|
"yuv420p",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
VideoSpec {
|
||||||
|
name: "h264 1080p",
|
||||||
|
filename: "video_h264_1080p.mp4",
|
||||||
|
encode_args: &[
|
||||||
|
"-f",
|
||||||
|
"lavfi",
|
||||||
|
"-i",
|
||||||
|
"testsrc=duration=3:size=1920x1080:rate=30",
|
||||||
|
"-c:v",
|
||||||
|
"libx264",
|
||||||
|
"-pix_fmt",
|
||||||
|
"yuv420p",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
VideoSpec {
|
||||||
|
// The iPhone case: HEVC/H.265 in a QuickTime .mov — undecodable by a
|
||||||
|
// browser <video>, so the old client path produced nothing for these.
|
||||||
|
name: "HEVC 1080p .mov",
|
||||||
|
filename: "video_hevc_1080p.mov",
|
||||||
|
encode_args: &[
|
||||||
|
"-f",
|
||||||
|
"lavfi",
|
||||||
|
"-i",
|
||||||
|
"testsrc=duration=3:size=1920x1080:rate=30",
|
||||||
|
"-c:v",
|
||||||
|
"libx265",
|
||||||
|
"-tag:v",
|
||||||
|
"hvc1",
|
||||||
|
"-pix_fmt",
|
||||||
|
"yuv420p",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
VideoSpec {
|
||||||
|
name: "VP9 720p .webm",
|
||||||
|
filename: "video_vp9_720p.webm",
|
||||||
|
encode_args: &[
|
||||||
|
"-f",
|
||||||
|
"lavfi",
|
||||||
|
"-i",
|
||||||
|
"testsrc=duration=3:size=1280x720:rate=30",
|
||||||
|
"-c:v",
|
||||||
|
"libvpx-vp9",
|
||||||
|
"-b:v",
|
||||||
|
"1M",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
fn corpus_dir() -> PathBuf {
|
||||||
|
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||||
|
.join("benches")
|
||||||
|
.join("corpus")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generate a test video with ffmpeg if it isn't already on disk.
|
||||||
|
fn ensure_video(ffmpeg: &str, spec: &VideoSpec, path: &Path) {
|
||||||
|
if path.exists() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let mut cmd = std::process::Command::new(ffmpeg);
|
||||||
|
cmd.arg("-y")
|
||||||
|
.arg("-hide_banner")
|
||||||
|
.arg("-loglevel")
|
||||||
|
.arg("error");
|
||||||
|
cmd.args(spec.encode_args);
|
||||||
|
cmd.arg(path);
|
||||||
|
match cmd.status() {
|
||||||
|
Ok(s) if s.success() => {}
|
||||||
|
Ok(s) => eprintln!("ffmpeg gen {} exited {s}", spec.name),
|
||||||
|
Err(e) => eprintln!("ffmpeg gen {} failed: {e}", spec.name),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::main]
|
||||||
|
async fn main() {
|
||||||
|
let ffmpeg = std::env::var("OXICLOUD_FFMPEG_PATH").unwrap_or_else(|_| "ffmpeg".to_string());
|
||||||
|
if !FfmpegVideoFrameService::is_available(&ffmpeg) {
|
||||||
|
eprintln!("ffmpeg not found (set OXICLOUD_FFMPEG_PATH) — cannot run video bench");
|
||||||
|
std::process::exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
let dir = corpus_dir();
|
||||||
|
let _ = std::fs::create_dir_all(&dir);
|
||||||
|
for spec in SPECS {
|
||||||
|
ensure_video(&ffmpeg, spec, &dir.join(spec.filename));
|
||||||
|
}
|
||||||
|
|
||||||
|
let svc = FfmpegVideoFrameService::new(ffmpeg.clone(), 4, Duration::from_secs(60));
|
||||||
|
|
||||||
|
println!("== Video thumbnails: server-side ffmpeg frame → WebP pipeline (Option B) ==");
|
||||||
|
println!(
|
||||||
|
"| {:<16} | {:>9} | {:>10} | {:>9} | {:>6} | {:>8} | {:>7} | {:<4} |",
|
||||||
|
"case", "video KB", "extract ms", "frame KB", "icon B", "prev B", "large B", "ok"
|
||||||
|
);
|
||||||
|
println!(
|
||||||
|
"|{:-<18}|{:-<11}|{:-<12}|{:-<11}|{:-<8}|{:-<10}|{:-<9}|{:-<6}|",
|
||||||
|
"", "", "", "", "", "", "", ""
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut old_transfer_kb = 0f64; // re-download the video on first view (worst case)
|
||||||
|
let mut new_transfer_b = 0u64; // fetch the preview WebP thumbnail
|
||||||
|
let mut covered = 0usize;
|
||||||
|
|
||||||
|
for spec in SPECS {
|
||||||
|
let path = dir.join(spec.filename);
|
||||||
|
let video_kb = std::fs::metadata(&path).map(|m| m.len()).unwrap_or(0) as f64 / 1024.0;
|
||||||
|
|
||||||
|
// Best-of-3 extraction time.
|
||||||
|
let mut best = Duration::MAX;
|
||||||
|
let mut frame = bytes::Bytes::new();
|
||||||
|
let mut ok = true;
|
||||||
|
for _ in 0..3 {
|
||||||
|
let t = Instant::now();
|
||||||
|
match svc.extract_frame(&path).await {
|
||||||
|
Ok(f) => {
|
||||||
|
best = best.min(t.elapsed());
|
||||||
|
frame = f;
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
println!("| {:<16} | extract failed: {e}", spec.name);
|
||||||
|
ok = false;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !ok || frame.is_empty() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
let thumbs = match ThumbnailService::bench_render_all_fmt(&frame, ThumbnailFormat::Webp) {
|
||||||
|
Ok(t) => t,
|
||||||
|
Err(e) => {
|
||||||
|
println!("| {:<16} | webp render failed: {e}", spec.name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let sz = |want: ThumbnailSize| {
|
||||||
|
thumbs
|
||||||
|
.iter()
|
||||||
|
.find(|(s, _)| *s == want)
|
||||||
|
.map(|(_, b)| *b)
|
||||||
|
.unwrap_or(0)
|
||||||
|
};
|
||||||
|
let (icon, preview, large) = (
|
||||||
|
sz(ThumbnailSize::Icon),
|
||||||
|
sz(ThumbnailSize::Preview),
|
||||||
|
sz(ThumbnailSize::Large),
|
||||||
|
);
|
||||||
|
|
||||||
|
covered += 1;
|
||||||
|
old_transfer_kb += video_kb;
|
||||||
|
new_transfer_b += preview as u64;
|
||||||
|
|
||||||
|
println!(
|
||||||
|
"| {:<16} | {:>9.1} | {:>10.1} | {:>9.1} | {:>6} | {:>8} | {:>7} | {:<4} |",
|
||||||
|
spec.name,
|
||||||
|
video_kb,
|
||||||
|
best.as_secs_f64() * 1000.0,
|
||||||
|
frame.len() as f64 / 1024.0,
|
||||||
|
icon,
|
||||||
|
preview,
|
||||||
|
large,
|
||||||
|
"yes"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
println!(
|
||||||
|
"\n Coverage: {}/{} codecs produced a thumbnail server-side (incl. HEVC/.mov — the \
|
||||||
|
browser <video> path produced 0 for HEVC).",
|
||||||
|
covered,
|
||||||
|
SPECS.len()
|
||||||
|
);
|
||||||
|
let new_kb = new_transfer_b as f64 / 1024.0;
|
||||||
|
println!(
|
||||||
|
" Per-first-view transfer to show {} video tiles:\n OLD (client re-downloads the video, worst case): {:.0} KB + 3 JPEG PUTs/video\n NEW (fetch the server WebP preview): {:.1} KB + 0 client decode\n → up to {:.0}× less data, and it is eager (ready before the gallery asks).",
|
||||||
|
covered,
|
||||||
|
old_transfer_kb,
|
||||||
|
new_kb,
|
||||||
|
if new_kb > 0.0 {
|
||||||
|
old_transfer_kb / new_kb
|
||||||
|
} else {
|
||||||
|
0.0
|
||||||
|
}
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -114,21 +114,3 @@ export async function batchTrash(fileIds: string[]): Promise<Set<string>> {
|
|||||||
}
|
}
|
||||||
return trashed;
|
return trashed;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Upload a generated thumbnail blob for a file at a given size. Used by the
|
|
||||||
* photos grid to persist client-generated video frames server-side.
|
|
||||||
*/
|
|
||||||
export async function uploadThumbnail(
|
|
||||||
fileId: string,
|
|
||||||
size: 'icon' | 'preview' | 'large',
|
|
||||||
blob: Blob,
|
|
||||||
contentType = 'image/jpeg'
|
|
||||||
): Promise<void> {
|
|
||||||
await apiFetch(`/api/files/${fileId}/thumbnail/${size}`, {
|
|
||||||
method: 'PUT',
|
|
||||||
credentials: 'same-origin',
|
|
||||||
headers: { ...getCsrfHeaders(), 'Content-Type': contentType },
|
|
||||||
body: blob
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -6,12 +6,7 @@
|
|||||||
import { useSelection } from '$lib/composables/useSelection.svelte';
|
import { useSelection } from '$lib/composables/useSelection.svelte';
|
||||||
import { errorToast } from '$lib/utils/errors';
|
import { errorToast } from '$lib/utils/errors';
|
||||||
import { onMount } from 'svelte';
|
import { onMount } from 'svelte';
|
||||||
import {
|
import { batchTrash, fetchPhotos, type PhotoItem } from '$lib/api/endpoints/photos';
|
||||||
batchTrash,
|
|
||||||
fetchPhotos,
|
|
||||||
uploadThumbnail,
|
|
||||||
type PhotoItem
|
|
||||||
} from '$lib/api/endpoints/photos';
|
|
||||||
import { peopleEnabled } from '$lib/api/endpoints/people';
|
import { peopleEnabled } from '$lib/api/endpoints/people';
|
||||||
import { fileDownloadUrl, fileThumbnailUrl } from '$lib/api/endpoints/files';
|
import { fileDownloadUrl, fileThumbnailUrl } from '$lib/api/endpoints/files';
|
||||||
import Icon from '$lib/icons/Icon.svelte';
|
import Icon from '$lib/icons/Icon.svelte';
|
||||||
@@ -55,9 +50,6 @@
|
|||||||
else if (tab === 'people') void peopleView.load();
|
else if (tab === 'people') void peopleView.load();
|
||||||
});
|
});
|
||||||
|
|
||||||
/** Client-generated video frame thumbnails (file id → data/URL). */
|
|
||||||
let videoThumbs = $state<Record<string, string>>({});
|
|
||||||
|
|
||||||
/** EXIF-aware timestamp (seconds → ms), matching the OLD grouping logic. */
|
/** EXIF-aware timestamp (seconds → ms), matching the OLD grouping logic. */
|
||||||
function bucketKey(d: Date): string {
|
function bucketKey(d: Date): string {
|
||||||
const y = d.getFullYear();
|
const y = d.getFullYear();
|
||||||
@@ -290,95 +282,17 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* A tile thumbnail failed to load (no server thumbnail — e.g. SVGs, which the
|
* A tile thumbnail failed to load — no server thumbnail yet (SVGs the backend
|
||||||
* backend can't rasterise — or a transient error). Hide the broken <img> so
|
* can't rasterise; a video whose server-side frame extraction is still running
|
||||||
* the always-present placeholder shows through, and for videos kick off
|
* or unavailable) or a transient error. Hide the broken <img> so the
|
||||||
* client-side frame extraction (which, on success, re-renders the tile from
|
* always-present placeholder (and, for videos, the play badge) shows through.
|
||||||
* `videoThumbs`).
|
*
|
||||||
|
* Video thumbnails are now produced server-side (ffmpeg) on upload through the
|
||||||
|
* same WebP pipeline as photos — the browser no longer re-downloads the video
|
||||||
|
* to extract a frame.
|
||||||
*/
|
*/
|
||||||
function onThumbError(e: Event, photo: PhotoItem) {
|
function onThumbError(e: Event) {
|
||||||
(e.currentTarget as HTMLImageElement).style.display = 'none';
|
(e.currentTarget as HTMLImageElement).style.display = 'none';
|
||||||
if (isVideo(photo)) void generateVideoThumb(photo);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Client-side video thumbnail generation ──────────────────────────────
|
|
||||||
// When the server has no thumbnail for a video tile the <img> errors; we
|
|
||||||
// then extract a frame with the browser's native decoder and upload it.
|
|
||||||
|
|
||||||
async function generateVideoThumb(file: PhotoItem) {
|
|
||||||
if (videoThumbs[file.id]) return;
|
|
||||||
try {
|
|
||||||
const bitmap = await frameFromVideo(`/api/files/${file.id}?inline=true`);
|
|
||||||
const SIZES: Array<['icon' | 'preview' | 'large', number, number]> = [
|
|
||||||
['icon', 150, 150],
|
|
||||||
['preview', 400, 400],
|
|
||||||
['large', 800, 800]
|
|
||||||
];
|
|
||||||
let previewData = '';
|
|
||||||
// Render the blobs and push all three sizes in parallel; `previewData`
|
|
||||||
// is captured before its upload so the local preview shows even if that
|
|
||||||
// upload fails (allSettled swallows per-size failures, as before).
|
|
||||||
await Promise.allSettled(
|
|
||||||
SIZES.map(async ([size, w, h]) => {
|
|
||||||
const blob = await bitmapToBlob(bitmap, w, h);
|
|
||||||
if (size === 'preview') previewData = await blobToDataUrl(blob);
|
|
||||||
await uploadThumbnail(file.id, size, blob);
|
|
||||||
})
|
|
||||||
);
|
|
||||||
if (previewData) videoThumbs = { ...videoThumbs, [file.id]: previewData };
|
|
||||||
} catch {
|
|
||||||
// Keep the generic play badge on failure.
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function frameFromVideo(src: string): Promise<ImageBitmap> {
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const video = document.createElement('video');
|
|
||||||
video.src = src;
|
|
||||||
video.muted = true;
|
|
||||||
video.preload = 'metadata';
|
|
||||||
video.onloadedmetadata = () => {
|
|
||||||
video.currentTime = (video.duration || 3) / 3;
|
|
||||||
};
|
|
||||||
video.onseeked = async () => {
|
|
||||||
try {
|
|
||||||
const bitmap = await createImageBitmap(video);
|
|
||||||
video.removeAttribute('src');
|
|
||||||
video.load();
|
|
||||||
resolve(bitmap);
|
|
||||||
} catch (e) {
|
|
||||||
reject(e instanceof Error ? e : new Error(String(e)));
|
|
||||||
}
|
|
||||||
};
|
|
||||||
video.onerror = () => reject(new Error('video frame extraction failed'));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async function bitmapToBlob(bitmap: ImageBitmap, tw: number, th: number): Promise<Blob> {
|
|
||||||
const ratio = bitmap.width / bitmap.height;
|
|
||||||
const target = tw / th;
|
|
||||||
const w = ratio > target ? tw : Math.round(th * ratio);
|
|
||||||
const h = ratio > target ? Math.round(tw / ratio) : th;
|
|
||||||
const canvas = document.createElement('canvas');
|
|
||||||
canvas.width = w;
|
|
||||||
canvas.height = h;
|
|
||||||
canvas.getContext('2d')?.drawImage(bitmap, 0, 0, w, h);
|
|
||||||
return new Promise<Blob>((resolve, reject) => {
|
|
||||||
canvas.toBlob(
|
|
||||||
(b) => (b ? resolve(b) : reject(new Error('canvas toBlob failed'))),
|
|
||||||
'image/jpeg',
|
|
||||||
0.8
|
|
||||||
);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function blobToDataUrl(blob: Blob): Promise<string> {
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const reader = new FileReader();
|
|
||||||
reader.onload = () => resolve(String(reader.result));
|
|
||||||
reader.onerror = () => reject(new Error('blob read failed'));
|
|
||||||
reader.readAsDataURL(blob);
|
|
||||||
});
|
|
||||||
}
|
}
|
||||||
|
|
||||||
onMount(() => {
|
onMount(() => {
|
||||||
@@ -540,19 +454,15 @@
|
|||||||
it can't load (no server thumbnail, e.g. SVG), hides itself to reveal
|
it can't load (no server thumbnail, e.g. SVG), hides itself to reveal
|
||||||
this default rather than the browser's broken-image glyph. -->
|
this default rather than the browser's broken-image glyph. -->
|
||||||
<span class="photo-tile__placeholder" aria-hidden="true"><Icon name="file-image" /></span>
|
<span class="photo-tile__placeholder" aria-hidden="true"><Icon name="file-image" /></span>
|
||||||
{#if videoThumbs[photo.id]}
|
<img
|
||||||
<img src={videoThumbs[photo.id]} alt={photo.name} loading="lazy" decoding="async" />
|
src={fileThumbnailUrl(photo.id, 'preview')}
|
||||||
{:else}
|
srcset={`${fileThumbnailUrl(photo.id, 'icon')} 150w, ${fileThumbnailUrl(photo.id, 'preview')} 400w, ${fileThumbnailUrl(photo.id, 'large')} 800w`}
|
||||||
<img
|
sizes="(max-width: 768px) 33vw, 200px"
|
||||||
src={fileThumbnailUrl(photo.id, 'preview')}
|
alt={photo.name}
|
||||||
srcset={`${fileThumbnailUrl(photo.id, 'icon')} 150w, ${fileThumbnailUrl(photo.id, 'preview')} 400w, ${fileThumbnailUrl(photo.id, 'large')} 800w`}
|
loading="lazy"
|
||||||
sizes="(max-width: 768px) 33vw, 200px"
|
decoding="async"
|
||||||
alt={photo.name}
|
onerror={onThumbError}
|
||||||
loading="lazy"
|
/>
|
||||||
decoding="async"
|
|
||||||
onerror={(e) => onThumbError(e, photo)}
|
|
||||||
/>
|
|
||||||
{/if}
|
|
||||||
{#if isVideo(photo)}
|
{#if isVideo(photo)}
|
||||||
<span class="photo-tile__video-badge" aria-hidden="true"><Icon name="play" /></span>
|
<span class="photo-tile__video-badge" aria-hidden="true"><Icon name="play" /></span>
|
||||||
{/if}
|
{/if}
|
||||||
|
|||||||
@@ -26,4 +26,5 @@ pub mod thumbnail_ports;
|
|||||||
pub mod transcode_ports;
|
pub mod transcode_ports;
|
||||||
pub mod trash_ports;
|
pub mod trash_ports;
|
||||||
pub mod user_lifecycle;
|
pub mod user_lifecycle;
|
||||||
|
pub mod video_frame_ports;
|
||||||
pub mod zip_ports;
|
pub mod zip_ports;
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
//! Video frame extraction port.
|
||||||
|
//!
|
||||||
|
//! Pulls a single representative still frame out of a video so the existing
|
||||||
|
//! image thumbnail pipeline (shrink-on-load → SIMD resize → WebP → blob-hash
|
||||||
|
//! storage → HTTP content negotiation) can treat videos exactly like photos —
|
||||||
|
//! eagerly, server-side, on upload. Keeping this behind a port lets the
|
||||||
|
//! composition root swap in a no-op when `ffmpeg` is absent or the feature is
|
||||||
|
//! off, so video uploads degrade gracefully (no thumbnail) instead of erroring.
|
||||||
|
|
||||||
|
use crate::common::errors::DomainError;
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use bytes::Bytes;
|
||||||
|
use std::path::Path;
|
||||||
|
|
||||||
|
/// Extracts a representative still frame from a video file.
|
||||||
|
///
|
||||||
|
/// Implementations:
|
||||||
|
/// - `FfmpegVideoFrameService` — shells out to the system `ffmpeg`, covering
|
||||||
|
/// every container/codec (incl. HEVC/MOV, which a browser `<video>` cannot
|
||||||
|
/// decode). Bounds its own process concurrency and per-call timeout.
|
||||||
|
/// - `NoopVideoFrameService` — registered when `ffmpeg` is unavailable or the
|
||||||
|
/// feature is disabled; `is_supported_video` returns false so the lifecycle
|
||||||
|
/// hook never attempts video thumbnails.
|
||||||
|
#[async_trait]
|
||||||
|
pub trait VideoFramePort: Send + Sync + 'static {
|
||||||
|
/// Whether `mime_type` is a video this extractor will attempt to thumbnail.
|
||||||
|
/// The no-op implementation always returns false.
|
||||||
|
fn is_supported_video(&self, mime_type: &str) -> bool;
|
||||||
|
|
||||||
|
/// Extract one representative frame from the video file at `path`, returning
|
||||||
|
/// encoded **PNG** bytes ready to feed into the image thumbnail renderer.
|
||||||
|
/// `path` must point at the decoded (decrypted, reassembled) video on disk.
|
||||||
|
async fn extract_frame(&self, path: &Path) -> Result<Bytes, DomainError>;
|
||||||
|
}
|
||||||
+20
-7
@@ -901,6 +901,11 @@ pub struct FeaturesConfig {
|
|||||||
/// Expose other OxiCloud users as a read-only "system" address book
|
/// Expose other OxiCloud users as a read-only "system" address book
|
||||||
/// at GET /api/address-books. Set to false to hide the user directory.
|
/// at GET /api/address-books. Set to false to hide the user directory.
|
||||||
pub expose_system_users: bool,
|
pub expose_system_users: bool,
|
||||||
|
/// Generate video thumbnails server-side via `ffmpeg` on upload. When true
|
||||||
|
/// (and ffmpeg is detected at startup) videos get a representative-frame
|
||||||
|
/// thumbnail through the same WebP pipeline as photos; otherwise videos have
|
||||||
|
/// no thumbnail. Env: `OXICLOUD_ENABLE_VIDEO_THUMBNAILS`.
|
||||||
|
pub enable_video_thumbnails: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Default for FeaturesConfig {
|
impl Default for FeaturesConfig {
|
||||||
@@ -908,13 +913,14 @@ impl Default for FeaturesConfig {
|
|||||||
Self {
|
Self {
|
||||||
enable_auth: true, // Enable authentication by default
|
enable_auth: true, // Enable authentication by default
|
||||||
enable_user_storage_quotas: false,
|
enable_user_storage_quotas: false,
|
||||||
enable_file_sharing: true, // Enable file sharing by default
|
enable_file_sharing: true, // Enable file sharing by default
|
||||||
enable_trash: true, // Enable trash feature
|
enable_trash: true, // Enable trash feature
|
||||||
enable_search: true, // Enable search feature
|
enable_search: true, // Enable search feature
|
||||||
enable_music: true, // Enable music feature
|
enable_music: true, // Enable music feature
|
||||||
enable_places: true, // Photo map (GET /api/photos/geo + Places tab)
|
enable_places: true, // Photo map (GET /api/photos/geo + Places tab)
|
||||||
enable_faces: false, // People/faces (biometric) — opt-in, off by default
|
enable_faces: false, // People/faces (biometric) — opt-in, off by default
|
||||||
expose_system_users: true, // Expose OxiCloud users as address book by default
|
expose_system_users: true, // Expose OxiCloud users as address book by default
|
||||||
|
enable_video_thumbnails: true, // Video thumbs via ffmpeg (if detected)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1469,6 +1475,13 @@ impl AppConfig {
|
|||||||
config.features.enable_places = val;
|
config.features.enable_places = val;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if let Ok(enable_video_thumbnails) =
|
||||||
|
env::var("OXICLOUD_ENABLE_VIDEO_THUMBNAILS").map(|v| v.parse::<bool>())
|
||||||
|
&& let Ok(val) = enable_video_thumbnails
|
||||||
|
{
|
||||||
|
config.features.enable_video_thumbnails = val;
|
||||||
|
}
|
||||||
|
|
||||||
if let Ok(enable_faces) = env::var("OXICLOUD_ENABLE_FACES").map(|v| v.parse::<bool>())
|
if let Ok(enable_faces) = env::var("OXICLOUD_ENABLE_FACES").map(|v| v.parse::<bool>())
|
||||||
&& let Ok(val) = enable_faces
|
&& let Ok(val) = enable_faces
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -46,6 +46,7 @@ use crate::infrastructure::services::search_index::content_index_worker::Content
|
|||||||
use crate::infrastructure::services::search_index::tantivy_content_index::TantivyContentIndex;
|
use crate::infrastructure::services::search_index::tantivy_content_index::TantivyContentIndex;
|
||||||
use crate::infrastructure::services::trash_cleanup_service::TrashCleanupService;
|
use crate::infrastructure::services::trash_cleanup_service::TrashCleanupService;
|
||||||
|
|
||||||
|
use crate::application::ports::video_frame_ports::VideoFramePort;
|
||||||
use crate::application::services::app_password_service::AppPasswordService;
|
use crate::application::services::app_password_service::AppPasswordService;
|
||||||
use crate::application::services::blob_lifecycle_service::BlobLifecycleService;
|
use crate::application::services::blob_lifecycle_service::BlobLifecycleService;
|
||||||
use crate::application::services::calendar_service::CalendarService;
|
use crate::application::services::calendar_service::CalendarService;
|
||||||
@@ -66,6 +67,9 @@ use crate::infrastructure::repositories::pg::{
|
|||||||
use crate::infrastructure::services::audio_metadata_service::AudioMetadataService;
|
use crate::infrastructure::services::audio_metadata_service::AudioMetadataService;
|
||||||
use crate::infrastructure::services::chunked_upload_service::ChunkedUploadService;
|
use crate::infrastructure::services::chunked_upload_service::ChunkedUploadService;
|
||||||
use crate::infrastructure::services::dedup_service::DedupService;
|
use crate::infrastructure::services::dedup_service::DedupService;
|
||||||
|
use crate::infrastructure::services::ffmpeg_video_frame_service::{
|
||||||
|
FfmpegVideoFrameService, NoopVideoFrameService,
|
||||||
|
};
|
||||||
use crate::infrastructure::services::image_transcode_service::ImageTranscodeService;
|
use crate::infrastructure::services::image_transcode_service::ImageTranscodeService;
|
||||||
use crate::infrastructure::services::jwt_service::JwtTokenService;
|
use crate::infrastructure::services::jwt_service::JwtTokenService;
|
||||||
use crate::infrastructure::services::media_metadata_service::MediaMetadataService;
|
use crate::infrastructure::services::media_metadata_service::MediaMetadataService;
|
||||||
@@ -350,9 +354,64 @@ impl AppServiceFactory {
|
|||||||
// ThumbnailRefreshHook: handles FileLifecycleHook events (create/update/delete).
|
// ThumbnailRefreshHook: handles FileLifecycleHook events (create/update/delete).
|
||||||
// Implemented on ThumbnailRefreshHook (not ThumbnailService) to avoid circular Arc:
|
// Implemented on ThumbnailRefreshHook (not ThumbnailService) to avoid circular Arc:
|
||||||
// DedupService → BlobLifecycleService → ThumbnailRefreshHook → DedupService.
|
// DedupService → BlobLifecycleService → ThumbnailRefreshHook → DedupService.
|
||||||
|
// Video frame extractor for thumbnails. Detect ffmpeg once at startup so
|
||||||
|
// the choice (real extractor vs. no-op) is logged here instead of failing
|
||||||
|
// per upload.
|
||||||
|
let video_frame: Arc<dyn VideoFramePort> = {
|
||||||
|
let ffmpeg_path =
|
||||||
|
std::env::var("OXICLOUD_FFMPEG_PATH").unwrap_or_else(|_| "ffmpeg".to_string());
|
||||||
|
if self.config.features.enable_video_thumbnails
|
||||||
|
&& FfmpegVideoFrameService::is_available(&ffmpeg_path)
|
||||||
|
{
|
||||||
|
let cpus = std::thread::available_parallelism()
|
||||||
|
.map(|n| n.get())
|
||||||
|
.unwrap_or(4);
|
||||||
|
let concurrency = std::env::var("OXICLOUD_VIDEO_THUMBNAIL_CONCURRENCY")
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.parse::<usize>().ok())
|
||||||
|
.unwrap_or((cpus / 2).max(1));
|
||||||
|
let timeout = std::time::Duration::from_secs(
|
||||||
|
std::env::var("OXICLOUD_VIDEO_THUMBNAIL_TIMEOUT_SECS")
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.parse().ok())
|
||||||
|
.unwrap_or(30),
|
||||||
|
);
|
||||||
|
tracing::info!(
|
||||||
|
"🎬 Video thumbnails enabled (ffmpeg '{}', concurrency {})",
|
||||||
|
ffmpeg_path,
|
||||||
|
concurrency
|
||||||
|
);
|
||||||
|
Arc::new(FfmpegVideoFrameService::new(
|
||||||
|
ffmpeg_path,
|
||||||
|
concurrency,
|
||||||
|
timeout,
|
||||||
|
))
|
||||||
|
} else {
|
||||||
|
if self.config.features.enable_video_thumbnails {
|
||||||
|
tracing::warn!(
|
||||||
|
"🎬 Video thumbnails enabled but ffmpeg not found at '{}' \
|
||||||
|
(set OXICLOUD_FFMPEG_PATH) — videos will have no thumbnail",
|
||||||
|
ffmpeg_path
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
tracing::info!("🎬 Video thumbnails disabled");
|
||||||
|
}
|
||||||
|
Arc::new(NoopVideoFrameService)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// Cap on bytes streamed to a temp file for frame extraction (default 2 GB).
|
||||||
|
// saturating_mul so an absurd MB value can't silently wrap to a tiny cap.
|
||||||
|
let video_max_bytes: u64 = std::env::var("OXICLOUD_VIDEO_THUMBNAIL_MAX_MB")
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.parse::<u64>().ok())
|
||||||
|
.unwrap_or(2048)
|
||||||
|
.saturating_mul(1024 * 1024);
|
||||||
|
|
||||||
let thumbnail_refresh_hook = Arc::new(ThumbnailRefreshHook::new(
|
let thumbnail_refresh_hook = Arc::new(ThumbnailRefreshHook::new(
|
||||||
thumbnail_service.clone(),
|
thumbnail_service.clone(),
|
||||||
dedup_service.clone(),
|
dedup_service.clone(),
|
||||||
|
video_frame,
|
||||||
|
video_max_bytes,
|
||||||
));
|
));
|
||||||
|
|
||||||
// Build the unified FileLifecycleService dispatcher.
|
// Build the unified FileLifecycleService dispatcher.
|
||||||
|
|||||||
@@ -0,0 +1,150 @@
|
|||||||
|
//! ffmpeg-backed video frame extractor (and a no-op fallback).
|
||||||
|
//!
|
||||||
|
//! Shells out to the system `ffmpeg` binary rather than linking libav*: no
|
||||||
|
//! compile-time dependency, no binary bloat, and it decodes every container the
|
||||||
|
//! browser `<video>` element cannot (HEVC/MOV, ProRes, mkv/avi/wmv…). The
|
||||||
|
//! extracted still frame is handed to the existing image thumbnail pipeline, so
|
||||||
|
//! video thumbnails become first-class: WebP, blob-hash keyed (dedup'd), and
|
||||||
|
//! served through the same content negotiation as photos.
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use bytes::Bytes;
|
||||||
|
use std::path::Path;
|
||||||
|
use std::process::Stdio;
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::time::Duration;
|
||||||
|
use tokio::process::Command;
|
||||||
|
use tokio::sync::Semaphore;
|
||||||
|
use tokio::time::timeout;
|
||||||
|
|
||||||
|
use crate::application::ports::video_frame_ports::VideoFramePort;
|
||||||
|
use crate::common::errors::DomainError;
|
||||||
|
|
||||||
|
/// PNG file signature — guards against feeding a non-image (or empty) ffmpeg
|
||||||
|
/// output into the renderer.
|
||||||
|
const PNG_MAGIC: &[u8; 8] = b"\x89PNG\r\n\x1a\n";
|
||||||
|
|
||||||
|
/// Extracts a representative frame by invoking `ffmpeg`.
|
||||||
|
pub struct FfmpegVideoFrameService {
|
||||||
|
ffmpeg_path: String,
|
||||||
|
/// Bounds concurrent ffmpeg processes — video decode is CPU-heavy and runs
|
||||||
|
/// outside the async runtime, so it gets its own (smaller) limit rather than
|
||||||
|
/// sharing the image decode semaphore.
|
||||||
|
semaphore: Arc<Semaphore>,
|
||||||
|
/// Per-extraction wall-clock cap; the child is killed on overrun.
|
||||||
|
timeout: Duration,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FfmpegVideoFrameService {
|
||||||
|
pub fn new(ffmpeg_path: String, concurrency: usize, timeout: Duration) -> Self {
|
||||||
|
Self {
|
||||||
|
ffmpeg_path,
|
||||||
|
semaphore: Arc::new(Semaphore::new(concurrency.max(1))),
|
||||||
|
timeout,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Best-effort startup probe: true if `<ffmpeg_path> -version` runs and exits
|
||||||
|
/// 0. Synchronous so the composition root can decide — register the real
|
||||||
|
/// extractor or fall back to [`NoopVideoFrameService`] — without an async
|
||||||
|
/// context, and so a misconfigured path is logged once at boot instead of
|
||||||
|
/// failing per upload.
|
||||||
|
pub fn is_available(ffmpeg_path: &str) -> bool {
|
||||||
|
std::process::Command::new(ffmpeg_path)
|
||||||
|
.arg("-version")
|
||||||
|
.stdin(Stdio::null())
|
||||||
|
.stdout(Stdio::null())
|
||||||
|
.stderr(Stdio::null())
|
||||||
|
.status()
|
||||||
|
.map(|s| s.success())
|
||||||
|
.unwrap_or(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl VideoFramePort for FfmpegVideoFrameService {
|
||||||
|
fn is_supported_video(&self, mime_type: &str) -> bool {
|
||||||
|
// ffmpeg is the arbiter of what actually decodes; gate broadly on video/*
|
||||||
|
// and let extraction fail gracefully for the rare unsupported container.
|
||||||
|
mime_type.starts_with("video/")
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn extract_frame(&self, path: &Path) -> Result<Bytes, DomainError> {
|
||||||
|
let _permit =
|
||||||
|
self.semaphore.acquire().await.map_err(|_| {
|
||||||
|
DomainError::internal_error("VideoFrame", "extractor semaphore closed")
|
||||||
|
})?;
|
||||||
|
|
||||||
|
// One representative still → PNG on stdout. The `thumbnail` filter scans a
|
||||||
|
// window of frames and picks the most representative one (skipping black
|
||||||
|
// intros) without needing a separate duration probe. Fit within a
|
||||||
|
// 1024×1024 box (preserving aspect) — enough for the 800px `large`
|
||||||
|
// thumbnail, and bounding BOTH dimensions caps the emitted PNG size so a
|
||||||
|
// hostile/extreme geometry can't balloon the buffered output. Arguments
|
||||||
|
// are passed individually (never through a shell), so a hostile file name
|
||||||
|
// cannot inject anything.
|
||||||
|
let run = Command::new(&self.ffmpeg_path)
|
||||||
|
.arg("-nostdin")
|
||||||
|
.arg("-loglevel")
|
||||||
|
.arg("error")
|
||||||
|
.arg("-i")
|
||||||
|
.arg(path)
|
||||||
|
.arg("-vf")
|
||||||
|
.arg("thumbnail,scale=w='min(1024,iw)':h='min(1024,ih)':force_original_aspect_ratio=decrease")
|
||||||
|
.arg("-frames:v")
|
||||||
|
.arg("1")
|
||||||
|
.arg("-f")
|
||||||
|
.arg("image2pipe")
|
||||||
|
.arg("-vcodec")
|
||||||
|
.arg("png")
|
||||||
|
.arg("pipe:1")
|
||||||
|
.stdin(Stdio::null())
|
||||||
|
.stdout(Stdio::piped())
|
||||||
|
.stderr(Stdio::piped())
|
||||||
|
.kill_on_drop(true)
|
||||||
|
.output();
|
||||||
|
|
||||||
|
let output = timeout(self.timeout, run)
|
||||||
|
.await
|
||||||
|
.map_err(|_| DomainError::internal_error("VideoFrame", "ffmpeg timed out"))?
|
||||||
|
.map_err(|e| {
|
||||||
|
DomainError::internal_error("VideoFrame", format!("ffmpeg spawn failed: {e}"))
|
||||||
|
})?;
|
||||||
|
|
||||||
|
if !output.status.success() {
|
||||||
|
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||||
|
return Err(DomainError::internal_error(
|
||||||
|
"VideoFrame",
|
||||||
|
format!("ffmpeg exited {}: {}", output.status, stderr.trim()),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
let png = output.stdout;
|
||||||
|
if png.len() < PNG_MAGIC.len() || &png[..PNG_MAGIC.len()] != PNG_MAGIC {
|
||||||
|
return Err(DomainError::internal_error(
|
||||||
|
"VideoFrame",
|
||||||
|
"ffmpeg produced no decodable frame",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(Bytes::from(png))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// No-op extractor used when `ffmpeg` is unavailable or video thumbnails are
|
||||||
|
/// disabled. `is_supported_video` returns false so the lifecycle hook never
|
||||||
|
/// attempts generation; videos simply have no thumbnail (the prior behaviour).
|
||||||
|
pub struct NoopVideoFrameService;
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl VideoFramePort for NoopVideoFrameService {
|
||||||
|
fn is_supported_video(&self, _mime_type: &str) -> bool {
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn extract_frame(&self, _path: &Path) -> Result<Bytes, DomainError> {
|
||||||
|
Err(DomainError::internal_error(
|
||||||
|
"VideoFrame",
|
||||||
|
"video thumbnail extraction is disabled (no ffmpeg)",
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -9,6 +9,7 @@ pub mod encrypted_blob_backend;
|
|||||||
pub mod exif_service;
|
pub mod exif_service;
|
||||||
pub mod face_geometry;
|
pub mod face_geometry;
|
||||||
pub mod face_indexing_service;
|
pub mod face_indexing_service;
|
||||||
|
pub mod ffmpeg_video_frame_service;
|
||||||
pub mod file_content_cache;
|
pub mod file_content_cache;
|
||||||
pub mod file_system_i18n_service;
|
pub mod file_system_i18n_service;
|
||||||
pub mod image_transcode_service;
|
pub mod image_transcode_service;
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ use tokio::time::timeout;
|
|||||||
use crate::application::ports::thumbnail_ports::{
|
use crate::application::ports::thumbnail_ports::{
|
||||||
ThumbnailFormat, ThumbnailPort, ThumbnailSize as PortThumbnailSize, ThumbnailStatsDto,
|
ThumbnailFormat, ThumbnailPort, ThumbnailSize as PortThumbnailSize, ThumbnailStatsDto,
|
||||||
};
|
};
|
||||||
|
use crate::application::ports::video_frame_ports::VideoFramePort;
|
||||||
use crate::domain::errors::{DomainError, ErrorKind};
|
use crate::domain::errors::{DomainError, ErrorKind};
|
||||||
use crate::infrastructure::services::dedup_service::DedupService;
|
use crate::infrastructure::services::dedup_service::DedupService;
|
||||||
|
|
||||||
@@ -94,6 +95,11 @@ const WEBP_QUALITY: f32 = 82.0;
|
|||||||
/// Environment override for the decode-concurrency cap (ops tuning).
|
/// Environment override for the decode-concurrency cap (ops tuning).
|
||||||
const DECODE_CONCURRENCY_ENV: &str = "OXICLOUD_THUMBNAIL_DECODE_CONCURRENCY";
|
const DECODE_CONCURRENCY_ENV: &str = "OXICLOUD_THUMBNAIL_DECODE_CONCURRENCY";
|
||||||
|
|
||||||
|
/// Wall-clock cap for streaming a video blob to a temp file before frame
|
||||||
|
/// extraction. Bounds a stalled remote object-store read so the background task
|
||||||
|
/// (and its temp file) can't hang forever.
|
||||||
|
const STREAM_TO_TEMP_TIMEOUT: Duration = Duration::from_secs(120);
|
||||||
|
|
||||||
/// Compute max concurrent thumbnail decode operations at runtime.
|
/// Compute max concurrent thumbnail decode operations at runtime.
|
||||||
///
|
///
|
||||||
/// Uses all available CPUs (min 2). Before shrink-on-load each decode
|
/// Uses all available CPUs (min 2). Before shrink-on-load each decode
|
||||||
@@ -1094,45 +1100,195 @@ impl ThumbnailService {
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
let results = tokio::task::spawn_blocking(move || {
|
self.render_and_persist_all_webp(&file_id, &blob_hash, original_data)
|
||||||
Self::render_all_thumbnails_from_data(original_data.as_ref(), ThumbnailFormat::Webp)
|
.await;
|
||||||
})
|
|
||||||
.await;
|
|
||||||
|
|
||||||
let thumbnails = match results {
|
tracing::info!("✅ Background thumbnail generation complete: {}", file_id);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Render every size as WebP from a decoded source image and persist them by
|
||||||
|
/// blob_hash (disk `{hash}.webp` + moka). Shared by the image upload path and
|
||||||
|
/// the video path (which passes the extracted frame as the source), so both
|
||||||
|
/// produce identical, dedup-able, content-negotiable thumbnails.
|
||||||
|
async fn render_and_persist_all_webp(&self, file_id: &str, blob_hash: &str, source: Bytes) {
|
||||||
|
let results = tokio::task::spawn_blocking(move || {
|
||||||
|
Self::render_all_thumbnails_from_data(source.as_ref(), ThumbnailFormat::Webp)
|
||||||
|
})
|
||||||
|
.await;
|
||||||
|
|
||||||
|
let thumbnails = match results {
|
||||||
|
Ok(Ok(t)) => t,
|
||||||
|
Ok(Err(e)) => {
|
||||||
|
tracing::warn!("Thumbnail generation failed for {}: {}", file_id, e);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
tracing::warn!("Thumbnail task panicked for {}: {}", file_id, e);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
for (size, bytes) in thumbnails {
|
||||||
|
let thumb_path = self.get_thumbnail_path(blob_hash, size, ThumbnailFormat::Webp);
|
||||||
|
if let Some(parent) = thumb_path.parent() {
|
||||||
|
let _ = fs::create_dir_all(parent).await;
|
||||||
|
}
|
||||||
|
if let Err(e) = fs::write(&thumb_path, &bytes).await {
|
||||||
|
tracing::warn!("Failed to save thumbnail {} {:?}: {}", file_id, size, e);
|
||||||
|
} else {
|
||||||
|
let cache_key = ThumbnailCacheKey {
|
||||||
|
file_id: file_id.to_string(),
|
||||||
|
size,
|
||||||
|
format: ThumbnailFormat::Webp,
|
||||||
|
};
|
||||||
|
self.cache.insert(cache_key, bytes).await;
|
||||||
|
tracing::debug!("✅ Generated thumbnail: {} {:?}", file_id, size);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Eagerly generate WebP thumbnails for a freshly-uploaded video.
|
||||||
|
///
|
||||||
|
/// Streams the (decrypted, reassembled) blob to a temp file — bounded by
|
||||||
|
/// `max_bytes` so a giant upload never materialises in full — extracts one
|
||||||
|
/// representative frame via `video_frame`, then runs it through the same
|
||||||
|
/// WebP/blob-hash pipeline as photos. Any miss or failure leaves the video
|
||||||
|
/// without a thumbnail (the prior behaviour), never an error to the user.
|
||||||
|
pub fn generate_video_thumbnails_background(
|
||||||
|
self: Arc<Self>,
|
||||||
|
file_id: String,
|
||||||
|
blob_hash: String,
|
||||||
|
dedup: Arc<DedupService>,
|
||||||
|
video_frame: Arc<dyn VideoFramePort>,
|
||||||
|
max_bytes: u64,
|
||||||
|
) {
|
||||||
|
tokio::spawn(async move {
|
||||||
|
// Blob deleted before this task ran → nothing to do.
|
||||||
|
if !dedup.blob_exists(&blob_hash).await {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dedup hit: thumbnails for this content already exist on disk — just
|
||||||
|
// warm the moka cache (zero CPU), mirroring the image path.
|
||||||
|
let all_exist = {
|
||||||
|
let mut ok = true;
|
||||||
|
for size in ThumbnailSize::all() {
|
||||||
|
let p = self.get_thumbnail_path(&blob_hash, *size, ThumbnailFormat::Webp);
|
||||||
|
if fs::metadata(&p).await.is_err() {
|
||||||
|
ok = false;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ok
|
||||||
|
};
|
||||||
|
if all_exist {
|
||||||
|
for size in ThumbnailSize::all() {
|
||||||
|
let p = self.get_thumbnail_path(&blob_hash, *size, ThumbnailFormat::Webp);
|
||||||
|
if let Ok(data) = fs::read(&p).await {
|
||||||
|
let key = ThumbnailCacheKey {
|
||||||
|
file_id: file_id.clone(),
|
||||||
|
size: *size,
|
||||||
|
format: ThumbnailFormat::Webp,
|
||||||
|
};
|
||||||
|
self.cache.insert(key, Bytes::from(data)).await;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ffmpeg needs a seekable file, and the blob may be encrypted/chunked
|
||||||
|
// on disk — stream through the normal decrypting read path into a
|
||||||
|
// bounded temp file (on the data volume) rather than reading the raw
|
||||||
|
// blob path. Bounded by size (max_bytes) AND time, so a stalled remote
|
||||||
|
// backend can't hang the task or leak its temp file forever.
|
||||||
|
let tmp = match tokio::time::timeout(
|
||||||
|
STREAM_TO_TEMP_TIMEOUT,
|
||||||
|
Self::stream_blob_to_temp(&dedup, &blob_hash, max_bytes, &self.thumbnails_root),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
{
|
||||||
Ok(Ok(t)) => t,
|
Ok(Ok(t)) => t,
|
||||||
Ok(Err(e)) => {
|
Ok(Err(e)) => {
|
||||||
tracing::warn!("Thumbnail generation failed for {}: {}", file_id, e);
|
tracing::warn!("🎬 video thumb: stream {} failed: {}", file_id, e);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Err(_) => {
|
||||||
tracing::warn!("Thumbnail task panicked for {}: {}", file_id, e);
|
tracing::warn!("🎬 video thumb: stream {} timed out", file_id);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
for (size, bytes) in thumbnails {
|
let frame = match video_frame.extract_frame(tmp.path()).await {
|
||||||
let thumb_path = self.get_thumbnail_path(&blob_hash, size, ThumbnailFormat::Webp);
|
Ok(f) => f,
|
||||||
if let Some(parent) = thumb_path.parent() {
|
Err(e) => {
|
||||||
let _ = fs::create_dir_all(parent).await;
|
tracing::info!("🎬 video thumb: extract {} skipped: {}", file_id, e);
|
||||||
|
return;
|
||||||
}
|
}
|
||||||
if let Err(e) = fs::write(&thumb_path, &bytes).await {
|
};
|
||||||
tracing::warn!("Failed to save thumbnail {} {:?}: {}", file_id, size, e);
|
drop(tmp); // remove the temp video promptly; we have the frame bytes
|
||||||
} else {
|
|
||||||
let cache_key = ThumbnailCacheKey {
|
|
||||||
file_id: file_id.clone(),
|
|
||||||
size,
|
|
||||||
format: ThumbnailFormat::Webp,
|
|
||||||
};
|
|
||||||
self.cache.insert(cache_key, bytes).await;
|
|
||||||
tracing::debug!("✅ Generated thumbnail: {} {:?}", file_id, size);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
tracing::info!("✅ Background thumbnail generation complete: {}", file_id);
|
// Bound the CPU-bound decode+resize with the same decode_semaphore the
|
||||||
|
// image path uses (the ffmpeg extractor's own permit covered only
|
||||||
|
// extraction and is already released), so concurrent video uploads
|
||||||
|
// can't spawn unbounded render jobs.
|
||||||
|
let _permit = match self.decode_semaphore.acquire().await {
|
||||||
|
Ok(p) => p,
|
||||||
|
Err(_) => return,
|
||||||
|
};
|
||||||
|
self.render_and_persist_all_webp(&file_id, &blob_hash, frame)
|
||||||
|
.await;
|
||||||
|
tracing::info!("✅ Video thumbnail generation complete: {}", file_id);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Stream a blob through the decrypting/CDC-aware read path into a temp file,
|
||||||
|
/// aborting if it exceeds `max_bytes`. The returned handle deletes the file
|
||||||
|
/// on drop. Bounded memory: chunks are written straight to disk, never
|
||||||
|
/// buffered whole.
|
||||||
|
async fn stream_blob_to_temp(
|
||||||
|
dedup: &DedupService,
|
||||||
|
blob_hash: &str,
|
||||||
|
max_bytes: u64,
|
||||||
|
temp_dir: &Path,
|
||||||
|
) -> Result<tempfile::NamedTempFile, DomainError> {
|
||||||
|
use futures::StreamExt;
|
||||||
|
use tokio::io::AsyncWriteExt;
|
||||||
|
|
||||||
|
let mut stream = dedup.read_blob_stream(blob_hash).await?;
|
||||||
|
// Colocate the temp video with the data volume (sized for blobs) instead
|
||||||
|
// of the OS temp dir, which can be small or a tmpfs in containers.
|
||||||
|
let tmp = tempfile::Builder::new()
|
||||||
|
.prefix("oxithumb-")
|
||||||
|
.tempfile_in(temp_dir)
|
||||||
|
.map_err(|e| DomainError::internal_error("VideoFrame", format!("temp file: {e}")))?;
|
||||||
|
let mut out = tokio::fs::File::create(tmp.path())
|
||||||
|
.await
|
||||||
|
.map_err(|e| DomainError::internal_error("VideoFrame", format!("temp open: {e}")))?;
|
||||||
|
|
||||||
|
let mut written: u64 = 0;
|
||||||
|
while let Some(chunk) = stream.next().await {
|
||||||
|
let chunk = chunk.map_err(|e| {
|
||||||
|
DomainError::internal_error("VideoFrame", format!("blob read: {e}"))
|
||||||
|
})?;
|
||||||
|
written += chunk.len() as u64;
|
||||||
|
if written > max_bytes {
|
||||||
|
return Err(DomainError::internal_error(
|
||||||
|
"VideoFrame",
|
||||||
|
format!("video exceeds {max_bytes}-byte thumbnail cap"),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
out.write_all(&chunk).await.map_err(|e| {
|
||||||
|
DomainError::internal_error("VideoFrame", format!("temp write: {e}"))
|
||||||
|
})?;
|
||||||
|
}
|
||||||
|
out.flush()
|
||||||
|
.await
|
||||||
|
.map_err(|e| DomainError::internal_error("VideoFrame", format!("temp flush: {e}")))?;
|
||||||
|
drop(out); // close our write handle before ffmpeg opens the path
|
||||||
|
Ok(tmp)
|
||||||
|
}
|
||||||
|
|
||||||
/// Delete thumbnails for a file.
|
/// Delete thumbnails for a file.
|
||||||
///
|
///
|
||||||
/// Only invalidates the in-memory moka cache (keyed by file_id).
|
/// Only invalidates the in-memory moka cache (keyed by file_id).
|
||||||
@@ -1205,11 +1361,26 @@ impl ThumbnailService {
|
|||||||
pub struct ThumbnailRefreshHook {
|
pub struct ThumbnailRefreshHook {
|
||||||
thumbnail: Arc<ThumbnailService>,
|
thumbnail: Arc<ThumbnailService>,
|
||||||
dedup: Arc<DedupService>,
|
dedup: Arc<DedupService>,
|
||||||
|
/// Video frame extractor (ffmpeg, or a no-op when unavailable/disabled).
|
||||||
|
video_frame: Arc<dyn VideoFramePort>,
|
||||||
|
/// Max bytes streamed to a temp file for video frame extraction; larger
|
||||||
|
/// videos are skipped (no thumbnail) rather than materialised in full.
|
||||||
|
video_max_bytes: u64,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl ThumbnailRefreshHook {
|
impl ThumbnailRefreshHook {
|
||||||
pub fn new(thumbnail: Arc<ThumbnailService>, dedup: Arc<DedupService>) -> Self {
|
pub fn new(
|
||||||
Self { thumbnail, dedup }
|
thumbnail: Arc<ThumbnailService>,
|
||||||
|
dedup: Arc<DedupService>,
|
||||||
|
video_frame: Arc<dyn VideoFramePort>,
|
||||||
|
video_max_bytes: u64,
|
||||||
|
) -> Self {
|
||||||
|
Self {
|
||||||
|
thumbnail,
|
||||||
|
dedup,
|
||||||
|
video_frame,
|
||||||
|
video_max_bytes,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1222,16 +1393,28 @@ impl crate::application::ports::file_lifecycle::FileLifecycleHook for ThumbnailR
|
|||||||
is_new_blob: bool,
|
is_new_blob: bool,
|
||||||
) {
|
) {
|
||||||
// Blob-hash thumbnail already exists on disk when is_new_blob=false — skip.
|
// Blob-hash thumbnail already exists on disk when is_new_blob=false — skip.
|
||||||
if !is_new_blob || !ThumbnailService::is_supported_image(content_type) {
|
if !is_new_blob {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
self.thumbnail
|
if ThumbnailService::is_supported_image(content_type) {
|
||||||
.clone()
|
self.thumbnail
|
||||||
.generate_all_sizes_background_from_blob(
|
.clone()
|
||||||
|
.generate_all_sizes_background_from_blob(
|
||||||
|
file_id.to_string(),
|
||||||
|
blob_hash.to_string(),
|
||||||
|
self.dedup.clone(),
|
||||||
|
);
|
||||||
|
} else if self.video_frame.is_supported_video(content_type) {
|
||||||
|
// Videos: extract a frame server-side (ffmpeg) and run it through the
|
||||||
|
// same WebP/blob-hash pipeline — eager, off the request path.
|
||||||
|
self.thumbnail.clone().generate_video_thumbnails_background(
|
||||||
file_id.to_string(),
|
file_id.to_string(),
|
||||||
blob_hash.to_string(),
|
blob_hash.to_string(),
|
||||||
self.dedup.clone(),
|
self.dedup.clone(),
|
||||||
|
self.video_frame.clone(),
|
||||||
|
self.video_max_bytes,
|
||||||
);
|
);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn on_file_copied(
|
fn on_file_copied(
|
||||||
|
|||||||
@@ -429,8 +429,11 @@ impl FileHandler {
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
// Non-image (video, etc.) with no cached thumbnail → 204
|
// Images and videos both store blob-hash thumbnails (videos via an
|
||||||
if !thumbnail_service.is_supported_image(&file.mime_type) {
|
// eagerly-extracted frame); anything else has nothing to thumbnail → 204.
|
||||||
|
let is_image = thumbnail_service.is_supported_image(&file.mime_type);
|
||||||
|
let is_video = file.mime_type.starts_with("video/");
|
||||||
|
if !is_image && !is_video {
|
||||||
return Response::builder()
|
return Response::builder()
|
||||||
.status(StatusCode::NO_CONTENT)
|
.status(StatusCode::NO_CONTENT)
|
||||||
.header(header::CACHE_CONTROL, "no-store")
|
.header(header::CACHE_CONTROL, "no-store")
|
||||||
@@ -470,6 +473,44 @@ impl FileHandler {
|
|||||||
.into_response();
|
.into_response();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Videos: thumbnails are produced eagerly server-side (ffmpeg) on upload,
|
||||||
|
// and persisted WebP-only. Serve that WebP regardless of the negotiated
|
||||||
|
// format — a JPEG/`*/*`/no-Accept client still gets it, correctly labelled
|
||||||
|
// via byte-sniffing — otherwise non-WebP clients would 204 forever despite
|
||||||
|
// a valid thumbnail on disk. We never image-decode a video, so a genuine
|
||||||
|
// miss (generation in flight or unavailable) returns 204.
|
||||||
|
if is_video {
|
||||||
|
if let Some(data) = thumbnail_service
|
||||||
|
.get_cached_thumbnail(
|
||||||
|
&id,
|
||||||
|
Some(&blob_hash),
|
||||||
|
thumb_size.into(),
|
||||||
|
ThumbnailFormat::Webp,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
return Response::builder()
|
||||||
|
.status(StatusCode::OK)
|
||||||
|
.header(
|
||||||
|
header::CONTENT_TYPE,
|
||||||
|
crate::common::mime_detect::thumbnail_content_type(&data),
|
||||||
|
)
|
||||||
|
.header(header::CONTENT_LENGTH, data.len())
|
||||||
|
.header(header::CACHE_CONTROL, "public, max-age=31536000, immutable")
|
||||||
|
.header(header::ETAG, &etag)
|
||||||
|
.header(header::VARY, header::ACCEPT.as_str())
|
||||||
|
.body(Body::from(data))
|
||||||
|
.unwrap()
|
||||||
|
.into_response();
|
||||||
|
}
|
||||||
|
return Response::builder()
|
||||||
|
.status(StatusCode::NO_CONTENT)
|
||||||
|
.header(header::CACHE_CONTROL, "no-store")
|
||||||
|
.body(Body::empty())
|
||||||
|
.unwrap()
|
||||||
|
.into_response();
|
||||||
|
}
|
||||||
|
|
||||||
match thumbnail_service
|
match thumbnail_service
|
||||||
.get_thumbnail_from_blob(
|
.get_thumbnail_from_blob(
|
||||||
&id,
|
&id,
|
||||||
|
|||||||
Reference in New Issue
Block a user