perf: stream_files_in_subtree — replace Vec<File> with async Stream

Replace list_files_in_subtree (fetch_all → Vec) with stream_files_in_subtree
that returns a Pin<Box<dyn Stream<Item = Result<File/FileDto>>>> backed by a
PostgreSQL cursor via sqlx::fetch().

Changes:
- FileReadPort::stream_files_in_subtree() returns streaming cursor (no default)
- FileRetrievalUseCase::stream_files_in_subtree() maps File→FileDto on the fly
- FileBlobReadRepository: async_stream::try_stream! + sqlx::fetch() cursor
- batch_operations: consume stream into HashMap incrementally
- zip_service: consume stream into HashMap incrementally
- All stubs/mocks updated (return empty stream)

Eliminates:
- Double allocation: Vec<(9-tuple)> + Vec<File> materialized simultaneously
- Unbounded RAM proportional to subtree size (was ~500 bytes × N files)
- Latency: callers blocked until last row fetched from PG

RAM is now O(folders) for the HashMap, not O(files).
This commit is contained in:
Dionisio
2026-02-26 00:07:10 +01:00
parent 1ad7a32a61
commit 9f8a6f5177
9 changed files with 127 additions and 69 deletions
+14
View File
@@ -121,6 +121,13 @@ impl FileReadPort for StubFileReadPort {
) -> Result<usize, DomainError> {
Ok(0)
}
async fn stream_files_in_subtree(
&self,
_folder_id: &str,
) -> Result<Pin<Box<dyn Stream<Item = Result<File, DomainError>> + Send>>, DomainError> {
Ok(Box::pin(futures::stream::empty()))
}
}
// ---------------------------------------------------------------------------
@@ -526,6 +533,13 @@ impl FileRetrievalUseCase for StubFileRetrievalUseCase {
async fn get_file_by_path(&self, _path: &str) -> Result<FileDto, DomainError> {
Err(DomainError::not_found("File", "stub"))
}
async fn stream_files_in_subtree(
&self,
_folder_id: &str,
) -> Result<Pin<Box<dyn Stream<Item = Result<FileDto, DomainError>> + Send>>, DomainError> {
Ok(Box::pin(futures::stream::empty()))
}
}
// ---------------------------------------------------------------------------