perf(thumbnails): hold a decode permit before reading source blobs
Request-path thumbnail generation (REST get_thumbnail and NC preview) read the full source blob into RAM and decoded it with no concurrency bound — a first-view gallery of K images stacked K full-size buffers and K parallel decodes on the blocking pool. The background hook had the inverse ordering problem: it read the blob eagerly and only then queued on the decode semaphore, so N concurrent uploads held N originals in memory while waiting. All three paths now acquire the decode semaphore first and read the blob under the permit, capping peak RAM at permits x image size: - new ThumbnailService::get_thumbnail_from_blob defers the blob read into the moka init closure (cache and disk hits never touch the blob); both handlers use it and no longer pre-read. - get_thumbnail_from_bytes (direct-bytes variant) now also takes a permit before decoding; shared generate_and_persist core keeps the two entrypoints duplicate-free. - generate_all_sizes_background_from_blob (renamed from _from_bytes) checks blob existence and disk-dedup state, then acquires the permit, then reads; the redundant pre-read spawn wrapper in ThumbnailRefreshHook is gone. https://claude.ai/code/session_01QxwJDHqQhbMkHK333QtMme
This commit is contained in:
@@ -157,26 +157,18 @@ pub async fn handle_preview(
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
let original_bytes = match state.core.dedup_service.read_blob_bytes(&blob_hash).await {
|
||||
Ok(bytes) => bytes,
|
||||
Err(err) => {
|
||||
tracing::error!(
|
||||
"Failed to load source image for preview {}: {}",
|
||||
object_id,
|
||||
err
|
||||
);
|
||||
return Response::builder()
|
||||
.status(StatusCode::INTERNAL_SERVER_ERROR)
|
||||
.body(Body::from("Failed to load preview source"))
|
||||
.unwrap();
|
||||
}
|
||||
};
|
||||
|
||||
// Generate/get thumbnail
|
||||
// Generate/get thumbnail — the blob is read inside the service once a
|
||||
// decode permit is held, so preview stampedes cannot stack source
|
||||
// images in RAM.
|
||||
match state
|
||||
.core
|
||||
.thumbnail_service
|
||||
.get_thumbnail_from_bytes(&object_id, &blob_hash, thumb_size.into(), original_bytes)
|
||||
.get_thumbnail_from_blob(
|
||||
&object_id,
|
||||
&blob_hash,
|
||||
thumb_size.into(),
|
||||
state.core.dedup_service.clone(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(data) => {
|
||||
|
||||
Reference in New Issue
Block a user