9f692f03c3
Implement dual DB pools (primary + maintenance) and wire services
2026-02-24 19:28:00 +01:00
6aa38d0d24
perf: thumbnail semaphore, WOPI streaming, store_bytes guard, spawn_blocking SHA-256
...
- Issue #1 : Add Semaphore(4) + 50MP resolution guard to thumbnail_service
Bounds peak RAM from 4.8GB (50 uploads) to 384MB (4 concurrent decodes)
- Issue #3 : Migrate WOPI PutFile from Bytes to streaming temp file + SHA-256
RAM per WOPI PUT: ~100MB → ~256KB regardless of file size
- Issue #3 : Add 10MB guard in dedup store_bytes (defense-in-depth)
- Issue #5 : Move chunked upload assembly to spawn_blocking (sync I/O)
Frees Tokio workers during SHA-256 hashing (~130ms for 500MB)
- Clean up unused tokio imports (OpenOptions, BufWriter)
2026-02-24 16:11:52 +01:00
962468a1ce
perf(dedup): remove blocking std::fs calls from async context
...
Two fixes in dedup_service.rs:
1. Rename error path (L234): std::fs::remove_file → tokio::fs::remove_file
Restructured from map_err closure to match block since .await
cannot be used inside a sync closure.
2. Integrity verify (L691): fused blocking .exists() + async metadata()
into a single fs::metadata().await call. Eliminates one stat()
syscall per blob AND removes the only remaining blocking I/O from
the verify_integrity hot loop (buffer_unordered × VERIFY_CONCURRENCY).
2026-02-24 13:06:40 +01:00
b628a3a166
perf(issue#4): stream dedup verify_integrity & garbage_collect
...
- Replace fetch_all() with fetch() streaming cursor in verify_integrity
so memory stays O(batch=16) instead of O(total_blobs)
- Replace fetch_all() with fetch() streaming cursor in garbage_collect
so memory stays O(1) instead of O(orphans)
- Add TryStreamExt import for try_next() on cursors
Eliminates OOM risk with millions of blobs — RAM usage is now constant
regardless of table size.
2026-02-24 10:45:38 +01:00
12c6914d5d
perf( #28 ): parallelize verify_integrity() with buffer_unordered(16)
...
Replace sequential blob-by-blob SHA-256 verification with
futures::stream::buffer_unordered(16) to hash up to 16 blobs
concurrently.
Each hash_file() already runs on spawn_blocking, so 16 concurrent
verifications saturate both disk I/O queue and CPU cores.
Before: 10K blobs × 13ms = ~130s (NVMe) — 1 core, 1 I/O in flight
After: 10K blobs / 16 concurrency = ~8s — 16 cores, 16 I/O in flight
2026-02-23 23:43:59 +01:00
85908311dc
perf: findings 6.1, 6.2, 2.6 — async Argon2, moka cache, full streaming migration
...
- 6.1: PasswordHasherPort now async_trait with spawn_blocking for Argon2
- 6.2: OIDC pending maps migrated from std::sync::Mutex to moka::sync::Cache with TTL
- 2.6: All file download paths migrated to 64KB streaming (get_file_stream / read_blob_stream)
- WOPI, dedup, batch ZIP, file_retrieval_service consumers migrated
- WebDAV COPY uses zero-copy dedup (copy_file)
- Removed dead code: get_file_content, get_file_mmap, read_blob, read_blob_bytes
from traits, impls, stubs, and mocks (18 files touched)
2026-02-23 00:51:46 +01:00
b7bd656a43
perf: fix issues #1,#2,#3,#13 from performance audit
...
- #1 image_transcode: dedicated rayon pool + moka cache (no tokio blocking)
- #2 ltree materialized paths: eliminate N+1 folder/file queries
- #3 compute_content_hash: read blob_hash column instead of loading file into RAM
- #13 event delegation + DocumentFragment: ~15 delegated listeners replace ~19k per-item listeners
Backend: 82/82 tests pass, cargo check clean.
Frontend: ui.js and app.js syntax-validated via Node.js.
2026-02-16 13:10:44 +01:00
0be7ef8c0b
style: fix clippy collapsible_if + cargo fmt
2026-02-15 18:04:32 +01:00
1ed20f425f
perf: Phase 4+5 optimizations — uploads 10x, downloads 2x, concurrent 2x. moka cache, 512KB buffers, remove sync_all, hash-on-write, preloaded queries, bench.sh v3, gitignore storage/. 500MB upload 12.6s->1.3s (392MB/s). RSS 69-113MB, 0 swap.
2026-02-15 17:56:47 +01:00
fac0b5e77b
fix: critical bugs from deep audit
...
- Fix copy_files() data loss: implement real copy_file across full stack
(FileWritePort, FileManagementUseCase, stubs, service, repository with
atomic CTE + dedup ref_count increment, batch_operations caller)
- Fix plaintext password in replace_default_admin: hash password via
PasswordHasherPort before User::new()
- Fix CalendarService hardcoded user_id: unify CalendarUseCase trait with
explicit user_id parameter on all methods, remove zombie _for_user
duplicates and hardcoded 'current_user_id', update 20 CalDAV handler
call sites
- Previous session: migrate DedupService to PostgreSQL (storage.blobs),
atomic CTEs with compensation for file/folder repository operations
2026-02-14 20:22:19 +01:00
3179e1dd91
quick fix
2026-02-14 19:30:49 +01:00
4c98c5a657
style: apply cargo fmt to entire codebase
...
Standardize code formatting across all 173 Rust source files
using rustfmt. No functional changes - purely cosmetic.
This establishes a consistent code style baseline for the
project going forward.
2026-02-14 01:29:34 +01:00
67137a3ef2
refactor: apply clippy auto-fixes (162 warnings resolved)
...
- Fix needless borrows and references
- Collapse nested if statements
- Replace manual strip_prefix with str::strip_prefix()
- Remove redundant closures in map/unwrap_or_else
- Use Iterator::next_back() instead of rev().next()
- Simplify map_or patterns
- Use std::io::Error::other() instead of new(ErrorKind::Other, ..)
- Use div_ceil() instead of manual ceiling division
- Consolidate format! string arguments
- Various other idiomatic Rust improvements
39 files changed, 220 insertions(+), 320 deletions(-)
2026-02-14 01:26:02 +01:00
a82faa5eaf
refactoring hexagonal and clean architecture
2026-02-08 13:40:23 +01:00
8f2b0a354c
big refactoring
2026-02-03 17:59:04 +01:00
Dionisio