Claude
18781022be
This reverts the image-pool migration (commit 5629ba6). The bench
(bench_pool_concurrency / POOL-CONCURRENCY.md) measured the one pool it could
isolate — the thumbnail decode semaphore — and found flat throughput, p99 AND
peak RSS (137 MiB) from K=1..16: shrink-on-load already makes each decode
RAM-cheap, so sizing it to the CFS quota gains nothing measurable. Adding code
without a measured benefit isn't worth it.
Kept: the effective_parallelism() helper (it has a *measured* win in the Tokio
runtime — benches/RUNTIME.md) and the benchmark itself (reusable). The ffmpeg
video fan-out has a plausible a-priori case (one OS process per permit) but is
left as a future, deliberately-measured change rather than shipped on
speculation. Doc updated to record the decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JG5yYZ9s868mJwqT2Qz7ez
78 lines
3.8 KiB
Markdown
78 lines
3.8 KiB
Markdown
# CPU pool concurrency benchmark — thumbnail decode under a CPU quota
|
||
|
||
Measures what `effective_parallelism()` changes for the image pools
|
||
(`ThumbnailService::max_concurrent_decodes`, `image_transcode_service`, `di.rs`
|
||
video ffmpeg fan-out): the number of concurrent CPU-heavy renders permitted.
|
||
Those pools used to size from `available_parallelism()`, which ignores the CFS
|
||
quota (`--cpus` / cgroup `cpu.max`), so under a container quota they permit one
|
||
render per *host* core. Drives the **real service path** — `Semaphore(K)` gating
|
||
`spawn_blocking(ThumbnailService::bench_render_all)` with a gallery of concurrent
|
||
callers — and sweeps the permit count K, measuring throughput, p50/p99, and peak
|
||
RSS for K concurrent decodes.
|
||
|
||
## Reproduce
|
||
|
||
```bash
|
||
cargo build --release --features bench --example bench_pool_concurrency
|
||
taskset -c 0,1 ./target/release/examples/bench_pool_concurrency # model a 2-core quota
|
||
# tunables: BENCH_K_LIST=1,2,4,8,16 BENCH_GALLERY=48 BENCH_SECONDS=4
|
||
```
|
||
|
||
## Results (4-core box, pinned to 2 cores; image: synthetic 48 MP JPEG)
|
||
|
||
### [A] Throughput + tail latency (48 concurrent gallery callers)
|
||
|
||
| permits | renders/s | p50 ms | p99 ms |
|
||
|--------:|----------:|-------:|-------:|
|
||
| 1 | 16.5 | 7342 | 10370 |
|
||
| 2 (effective) | 20.0 | 5009 | 5816 |
|
||
| 4 | 20.8 | 4895 | 5536 |
|
||
| 8 | 20.0 | 4784 | 5685 |
|
||
| 16 | 18.0 | 4576 | 6140 |
|
||
|
||
### [B] Peak RSS, K concurrent decodes (one wave)
|
||
|
||
| permits | peak RSS MiB |
|
||
|--------:|-------------:|
|
||
| 1 | 137 |
|
||
| 2 | 137 |
|
||
| 4 | 137 |
|
||
| 8 | 137 |
|
||
| 16 | 137 |
|
||
|
||
## Conclusions
|
||
|
||
1. **The thumbnail-decode pool is not a bottleneck — over-permitting costs
|
||
nothing measurable here.** Throughput is flat from K=2 to K=8 (CPU-bound: two
|
||
cores stay saturated regardless), p99 barely moves, and **peak RSS is flat at
|
||
137 MiB across K=1..16**. K=1 under-utilises (one decode can't fill two cores);
|
||
K=16 is marginally worse on throughput/p99. So sizing this pool to the CFS
|
||
quota neither gains nor loses on this workload.
|
||
|
||
2. **This confirms the codebase's own design.** `thumbnail_service.rs` documents
|
||
that *shrink-on-load* (DCT-scaled decode straight to thumbnail size, ~18–25 MB
|
||
regardless of source resolution) is why the historical concurrency throttle
|
||
was removed — "the RAM ceiling no longer forces throttling and we can saturate
|
||
every core". The flat RSS is exactly that: each concurrent decode's transient
|
||
buffer is small, so 16 in flight cost the same resident memory as 1.
|
||
|
||
3. **Decision: NOT migrated (reverted).** Because the only pool this bench could
|
||
isolate showed zero measured benefit, the `effective_parallelism()` migration
|
||
of the image pools was reverted — adding code without a measured win isn't
|
||
worth it. The `effective_parallelism()` helper stays (it has a *measured*
|
||
benefit in the Tokio runtime — see `RUNTIME`), so a future, deliberately
|
||
measured case can adopt it per-pool.
|
||
The one pool with a plausible a-priori argument is the **ffmpeg video
|
||
fan-out** (one heavyweight OS process per permit — 32 ffmpeg processes for a
|
||
2-core budget on a many-core host is self-evidently wasteful). That was left
|
||
on `available_parallelism()` too, to revisit *with* a measurement if a
|
||
high-host-core / low-quota deployment running video thumbnails ever warrants
|
||
it. The transcode rayon pool over-sizing only costs parked thread stacks
|
||
(negligible).
|
||
|
||
4. **Honest caveat on scale.** This was run at a 2-core quota on a 4-core host
|
||
(K_oversub = 8 ≈ 4×). On a 64-core host under a 2-core quota the host-count
|
||
permit would be 64 (32× over), where even small per-decode costs and scheduler
|
||
pressure add up — the regime this change protects against but which this box
|
||
can't reproduce.
|