adding several features

This commit is contained in:
DioCrafts
2025-03-19 00:44:27 +01:00
parent 7157762540
commit d9bbd575d2
66 changed files with 12137 additions and 1055 deletions
+24 -3
View File
@@ -2,20 +2,32 @@
## Build Commands
```bash
# Core development workflow
cargo build # Build the project
cargo run # Run the project locally (server at http://127.0.0.1:8085)
cargo check # Quick check for compilation errors without building
# Testing commands
cargo test # Run all tests
cargo test -- --nocapture # Run tests with output displayed
cargo test <test_name> # Run a specific test (e.g., cargo test file_service)
cargo test domain::entities::file::tests::test_create_file # Run a specific test function
RUST_LOG=debug cargo test # Run tests with debug-level logging
RUST_LOG=trace cargo test # Run tests with trace-level logging
# Code quality tools
cargo clippy # Run linter to catch common mistakes
cargo clippy --fix # Fix auto-fixable linting issues
cargo fmt --check # Check code formatting without changing files
cargo fmt # Format code according to Rust conventions
# Debugging
RUST_LOG=debug cargo run # Run with detailed logging for debugging
RUST_BACKTRACE=1 cargo run # Run with full backtrace for better error diagnostics
```
## Code Style Guidelines
- **Architecture**: Follow Clean Architecture layers (domain, application, infrastructure, interfaces)
- **Architecture**: Follow Clean Architecture with clear layer separation (domain → application → infrastructure → interfaces)
- **Naming**: Use `snake_case` for files, modules, functions, variables; `PascalCase` for types/structs/enums
- **Modules**: Use mod.rs files for explicit exports with visibility modifiers (pub, pub(crate))
- **Error Handling**: Use Result<T, E> with thiserror for custom error types; propagate errors with ? operator
@@ -24,9 +36,18 @@ RUST_LOG=debug cargo run # Run with detailed logging for debugging
- **Async**: Use async-trait for repository interfaces; handle futures with .await and tokio runtime
- **Testing**: Write unit tests in the same file as implementation (bottom of file, in a tests module)
- **Dependencies**: Use axum for web API, tower-http for middleware, serde for serialization
- **Logging**: Use tracing crate with appropriate levels (debug, info, warn, error)
- **Logging**: Use tracing with appropriate levels (debug, info, warn, error) and structured contexts
- **Repository Pattern**: Define interfaces in domain layer, implement in infrastructure layer
- **I18n**: Store translations in JSON files under static/locales/, use i18n service for text lookups
- **Type Safety**: Prefer strong typing with domain-specific types over primitive types
- **Error Messages**: Provide clear, actionable error messages that help diagnose the issue
## Project Structure
OxiCloud is a NextCloud-like file storage system built in Rust with a focus on performance and security. It provides a clean REST API and web interface for file management using a layered architecture approach. The roadmap in TODO-LIST.md outlines planned features including enhanced folder support, file previews, user authentication, sharing, and a sync client.
OxiCloud is a NextCloud-like file storage system built in Rust with a focus on performance and security. It provides a clean REST API and web interface for file management using a layered architecture approach:
- **Domain Layer**: Core business logic and entities (src/domain/)
- **Application Layer**: Use cases and application services (src/application/)
- **Infrastructure Layer**: External systems and implementations (src/infrastructure/)
- **Interfaces Layer**: API and web controllers (src/interfaces/)
The roadmap in TODO-LIST.md outlines planned features including enhanced folder support, file previews, user authentication, sharing, and a sync client.
Generated
+1275 -30
View File
File diff suppressed because it is too large Load Diff
+18 -2
View File
@@ -6,15 +6,31 @@ edition = "2021"
[dependencies]
axum = { version = "0.8.1", features = ["multipart"] }
tokio = { version = "1.44.1", features = ["full"] }
tokio-util = { version = "0.7.14", features = ["io"] }
tokio-util = { version = "0.7.14", features = ["io", "codec"] }
tokio-stream = { version = "0.1.15", features = ["fs"] }
bytes = "1.6.0"
tempfile = "3.10.1"
tower = "0.5.2"
tower-http = { version = "0.6.2", features = ["fs", "compression-gzip", "trace", "cors"] }
tower-http = { version = "0.6.2", features = ["fs", "compression-gzip", "trace", "cors", "add-extension"] }
flate2 = "1.0.28"
tracing = "0.1.41"
tracing-subscriber = { version = "0.3.19", features = ["env-filter"] }
chrono = { version = "0.4.37", features = ["serde"] }
http-body = "0.4.5"
serde = { version = "1.0.219", features = ["derive"] }
serde_json = "1.0.140"
futures = "0.3.31"
async-stream = "0.3.5"
mime_guess = "2.0.5"
uuid = { version = "1.16.0", features = ["v4", "serde"] }
async-trait = "0.1.88"
thiserror = "2.0.12"
reqwest = { version = "0.12.5", features = ["json", "multipart"] }
mockall = { version = "0.12.1", optional = true }
rand = "0.8.5"
pin-project-lite = "0.2.13"
[features]
default = []
test_utils = ["mockall"]
+131
View File
@@ -0,0 +1,131 @@
# 🚀 OxiCloud
![OxiCloud](static/oxicloud-logo.svg)
## The high-performance, Rust-powered file storage solution
OxiCloud is a NextCloud-like file storage system built with Rust, designed from the ground up with **performance**, **security**, and **scalability** as its core principles. Perfect for self-hosting your own cloud storage or deploying in enterprise environments.
## ✨ Key Features
- 🔥 **Blazing Fast Performance**: Built with Rust and optimized for speed
- 📁 **Advanced File Management**: Intuitive folder structure with powerful batch operations
- 🔄 **Concurrent Processing**: Parallel file operations for large files and batch processing
- 🔍 **Smart Caching**: Multi-layered caching system for metadata and file access
- 🌐 **Internationalization**: Full i18n support (currently English and Spanish)
- 📱 **Responsive Design**: Works seamlessly on desktop and mobile devices
- 🔌 **Extensible Architecture**: Clean, layered design following domain-driven principles
## 🚀 Performance Optimizations
OxiCloud incorporates multiple advanced performance optimizations:
### Concurrency and Parallelism
- **Parallel File Processing**: Automatically splits large files into chunks for parallel processing
- **Asynchronous I/O**: Built on Tokio for non-blocking operations
- **Worker Pools**: Smart thread management for optimal resource utilization
### Intelligent Caching
- **File Metadata Cache**: Drastically reduces filesystem calls
- **Smart Cache Invalidation**: Selectively invalidates cache entries
- **Preloading**: Strategic preloading for frequently accessed directories
### I/O Optimization
- **Buffer Pooling**: Reuses memory buffers to reduce GC pressure
- **Adaptive Streaming**: Adjusts chunk sizes based on file size
- **Size-Based Processing**: Different strategies for small, medium, and large files
### Batch Processing
- **ID Mapping Optimizer**: Groups mapping operations to reduce overhead
- **Operation Batching**: Processes multiple file operations concurrently
- **Debounced Saving**: Groups write operations for optimal I/O
## 📸 Screenshots
*Coming soon!*
## 🛠️ Getting Started
### Prerequisites
- Rust 1.70+ and Cargo
### Installation
```bash
# Clone the repository
git clone https://github.com/yourusername/oxicloud.git
cd oxicloud
# Build the project
cargo build --release
# Run the server
cargo run --release
```
The server will be available at `http://localhost:8085`
## 🧩 Project Structure
OxiCloud follows Clean Architecture principles with clear separation of concerns:
- **Domain Layer**: Core business logic and entities
- **Application Layer**: Use cases and application services
- **Infrastructure Layer**: External systems and implementations
- **Interfaces Layer**: API and web controllers
## 🚧 Development
```bash
# Core development workflow
cargo build # Build the project
cargo run # Run the project locally
cargo check # Quick check for compilation errors
# Testing
cargo test # Run all tests
cargo test <test_name> # Run a specific test
# Code quality
cargo clippy # Run linter
cargo fmt # Format code
# Debugging
RUST_LOG=debug cargo run # Run with detailed logging
```
## 🗺️ Roadmap
OxiCloud is under active development. Upcoming features include:
- User authentication and multi-user support
- File sharing and collaboration features
- WebDAV support and sync clients
- File versioning
- Encryption
- Mobile applications
See [TODO-LIST.md](TODO-LIST.md) for a detailed roadmap.
## 🤝 Contributing
Contributions are welcome! Whether it's bug reports, feature suggestions, or code contributions, please feel free to reach out.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
## 📜 License
OxiCloud is available under the MIT License. See the LICENSE file for more information.
## 🙏 Acknowledgements
- The Rust community for the amazing ecosystem
- All contributors who have helped shape this project
---
Designed with ❤️ by OxiCloud Team
+27 -8
View File
@@ -32,14 +32,33 @@ pub struct FileDto {
impl From<File> for FileDto {
fn from(file: File) -> Self {
Self {
id: file.id,
name: file.name,
path: file.path.to_string_lossy().to_string(),
size: file.size,
mime_type: file.mime_type,
folder_id: file.folder_id,
created_at: file.created_at,
modified_at: file.modified_at,
id: file.id().to_string(),
name: file.name().to_string(),
path: file.path_string().to_string(),
size: file.size(),
mime_type: file.mime_type().to_string(),
folder_id: file.folder_id().map(String::from),
created_at: file.created_at(),
modified_at: file.modified_at(),
}
}
}
// Para convertir de FileDto a File para los batch handlers
impl From<FileDto> for File {
fn from(dto: FileDto) -> Self {
// Usar constructor para crear una entidad desde DTO
// Nota: esto debe simplificarse si File tiene un constructor adecuado
// Si no, deberías hacer la conversión de la mejor manera posible
File::from_dto(
dto.id,
dto.name,
dto.path,
dto.size,
dto.mime_type,
dto.folder_id,
dto.created_at,
dto.modified_at
)
}
}
+24 -7
View File
@@ -52,15 +52,32 @@ pub struct FolderDto {
impl From<Folder> for FolderDto {
fn from(folder: Folder) -> Self {
let is_root = folder.parent_id.is_none();
let is_root = folder.parent_id().is_none();
Self {
id: folder.id,
name: folder.name,
path: folder.path.to_string_lossy().to_string(),
parent_id: folder.parent_id,
created_at: folder.created_at,
modified_at: folder.modified_at,
id: folder.id().to_string(),
name: folder.name().to_string(),
path: folder.path_string().to_string(),
parent_id: folder.parent_id().map(String::from),
created_at: folder.created_at(),
modified_at: folder.modified_at(),
is_root,
}
}
}
// Para convertir de FolderDto a Folder para los batch handlers
impl From<FolderDto> for Folder {
fn from(dto: FolderDto) -> Self {
// Usar constructor para crear una entidad desde DTO
// Nota: esto debe simplificarse si Folder tiene un constructor adecuado
Folder::from_dto(
dto.id,
dto.name,
dto.path,
dto.parent_id,
dto.created_at,
dto.modified_at
)
}
}
+1
View File
@@ -1,4 +1,5 @@
pub mod file_dto;
pub mod folder_dto;
pub mod i18n_dto;
pub mod pagination;
+117
View File
@@ -0,0 +1,117 @@
use serde::{Serialize, Deserialize};
/// Un DTO para representar información de paginación
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PaginationDto {
/// Página actual (comienza en 0)
pub page: usize,
/// Tamaño de página
pub page_size: usize,
/// Número total de elementos
pub total_items: usize,
/// Número total de páginas
pub total_pages: usize,
/// Indica si hay una página siguiente
pub has_next: bool,
/// Indica si hay una página anterior
pub has_prev: bool,
}
/// Un DTO para representar una solicitud de paginación
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PaginationRequestDto {
/// Página solicitada (comienza en 0)
#[serde(default)]
pub page: usize,
/// Tamaño de página solicitado
#[serde(default = "default_page_size")]
pub page_size: usize,
}
/// Un DTO para representar una respuesta paginada
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PaginatedResponseDto<T> {
/// Datos en la página actual
pub items: Vec<T>,
/// Información de paginación
pub pagination: PaginationDto,
}
impl Default for PaginationRequestDto {
fn default() -> Self {
Self {
page: 0,
page_size: default_page_size(),
}
}
}
/// Función para establecer el tamaño de página por defecto
fn default_page_size() -> usize {
100 // Por defecto, 100 items por página
}
impl PaginationRequestDto {
/// Calcula el offset para consultas paginadas
pub fn offset(&self) -> usize {
self.page * self.page_size
}
/// Calcula el límite para consultas paginadas
pub fn limit(&self) -> usize {
self.page_size
}
/// Valida y ajusta los parámetros de paginación
pub fn validate_and_adjust(&self) -> Self {
let mut page = self.page;
let mut page_size = self.page_size;
// Asegurar que la página sea al menos 0
if page < 1 {
page = 0;
}
// Asegurar que el tamaño de página esté entre 10 y 500
if page_size < 10 {
page_size = 10;
} else if page_size > 500 {
page_size = 500;
}
Self {
page,
page_size,
}
}
}
impl<T> PaginatedResponseDto<T> {
/// Crea una nueva respuesta paginada a partir de los datos y la información de paginación
pub fn new(
items: Vec<T>,
page: usize,
page_size: usize,
total_items: usize,
) -> Self {
let total_pages = if total_items == 0 {
0
} else {
(total_items + page_size - 1) / page_size
};
let pagination = PaginationDto {
page,
page_size,
total_items,
total_pages,
has_next: page < total_pages - 1,
has_prev: page > 0,
};
Self {
items,
pagination,
}
}
}
+2
View File
@@ -1,3 +1,5 @@
pub mod dtos;
pub mod ports;
pub mod services;
pub mod transactions;
+77
View File
@@ -0,0 +1,77 @@
use std::sync::Arc;
use async_trait::async_trait;
use bytes::Bytes;
use futures::Stream;
use crate::application::dtos::file_dto::FileDto;
use crate::application::dtos::folder_dto::{CreateFolderDto, FolderDto, MoveFolderDto, RenameFolderDto};
use crate::common::errors::DomainError;
/// Puerto primario para operaciones de archivos
#[async_trait]
pub trait FileUseCase: Send + Sync + 'static {
/// Sube un nuevo archivo desde bytes
async fn upload_file(
&self,
name: String,
folder_id: Option<String>,
content_type: String,
content: Vec<u8>,
) -> Result<FileDto, DomainError>;
/// Obtiene un archivo por su ID
async fn get_file(&self, id: &str) -> Result<FileDto, DomainError>;
/// Lista archivos en una carpeta
async fn list_files(&self, folder_id: Option<&str>) -> Result<Vec<FileDto>, DomainError>;
/// Elimina un archivo
async fn delete_file(&self, id: &str) -> Result<(), DomainError>;
/// Obtiene contenido de archivo como bytes (para archivos pequeños)
async fn get_file_content(&self, id: &str) -> Result<Vec<u8>, DomainError>;
/// Obtiene contenido de archivo como stream (para archivos grandes)
async fn get_file_stream(&self, id: &str) -> Result<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>, DomainError>;
/// Mueve un archivo a otra carpeta
async fn move_file(&self, file_id: &str, folder_id: Option<String>) -> Result<FileDto, DomainError>;
}
/// Puerto primario para operaciones de carpetas
#[async_trait]
pub trait FolderUseCase: Send + Sync + 'static {
/// Crea una nueva carpeta
async fn create_folder(&self, dto: CreateFolderDto) -> Result<FolderDto, DomainError>;
/// Obtiene una carpeta por su ID
async fn get_folder(&self, id: &str) -> Result<FolderDto, DomainError>;
/// Obtiene una carpeta por su ruta
async fn get_folder_by_path(&self, path: &str) -> Result<FolderDto, DomainError>;
/// Lista carpetas dentro de una carpeta padre
async fn list_folders(&self, parent_id: Option<&str>) -> Result<Vec<FolderDto>, DomainError>;
/// Lista carpetas con paginación
async fn list_folders_paginated(
&self,
parent_id: Option<&str>,
pagination: &crate::application::dtos::pagination::PaginationRequestDto
) -> Result<crate::application::dtos::pagination::PaginatedResponseDto<FolderDto>, DomainError>;
/// Renombra una carpeta
async fn rename_folder(&self, id: &str, dto: RenameFolderDto) -> Result<FolderDto, DomainError>;
/// Mueve una carpeta a otro padre
async fn move_folder(&self, id: &str, dto: MoveFolderDto) -> Result<FolderDto, DomainError>;
/// Elimina una carpeta
async fn delete_folder(&self, id: &str) -> Result<(), DomainError>;
}
/// Factory para crear implementaciones de casos de uso
pub trait UseCaseFactory {
fn create_file_use_case(&self) -> Arc<dyn FileUseCase>;
fn create_folder_use_case(&self) -> Arc<dyn FolderUseCase>;
}
+2
View File
@@ -0,0 +1,2 @@
pub mod inbound;
pub mod outbound;
+118
View File
@@ -0,0 +1,118 @@
use std::path::PathBuf;
use async_trait::async_trait;
use bytes::Bytes;
use futures::Stream;
use crate::domain::entities::file::File;
use crate::domain::entities::folder::Folder;
use crate::domain::services::path_service::StoragePath;
use crate::common::errors::DomainError;
/// Puerto secundario para operaciones de almacenamiento
#[async_trait]
pub trait StoragePort: Send + Sync + 'static {
/// Resuelve una ruta de dominio a una ruta física
fn resolve_path(&self, storage_path: &StoragePath) -> PathBuf;
/// Crea directorios si no existen
async fn ensure_directory(&self, storage_path: &StoragePath) -> Result<(), DomainError>;
/// Verifica si existe un archivo en la ruta dada
async fn file_exists(&self, storage_path: &StoragePath) -> Result<bool, DomainError>;
/// Verifica si existe un directorio en la ruta dada
async fn directory_exists(&self, storage_path: &StoragePath) -> Result<bool, DomainError>;
}
/// Puerto secundario para persistencia de archivos
#[async_trait]
pub trait FileStoragePort: Send + Sync + 'static {
/// Guarda un nuevo archivo desde bytes
async fn save_file(
&self,
name: String,
folder_id: Option<String>,
content_type: String,
content: Vec<u8>,
) -> Result<File, DomainError>;
/// Obtiene un archivo por su ID
async fn get_file(&self, id: &str) -> Result<File, DomainError>;
/// Lista archivos en una carpeta
async fn list_files(&self, folder_id: Option<&str>) -> Result<Vec<File>, DomainError>;
/// Elimina un archivo
async fn delete_file(&self, id: &str) -> Result<(), DomainError>;
/// Obtiene contenido de archivo como bytes
async fn get_file_content(&self, id: &str) -> Result<Vec<u8>, DomainError>;
/// Obtiene contenido de archivo como stream
async fn get_file_stream(&self, id: &str) -> Result<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>, DomainError>;
/// Mueve un archivo a otra carpeta
async fn move_file(&self, file_id: &str, target_folder_id: Option<String>) -> Result<File, DomainError>;
/// Obtiene la ruta de almacenamiento de un archivo
async fn get_file_path(&self, id: &str) -> Result<StoragePath, DomainError>;
}
/// Puerto secundario para persistencia de carpetas
#[async_trait]
pub trait FolderStoragePort: Send + Sync + 'static {
/// Crea una nueva carpeta
async fn create_folder(&self, name: String, parent_id: Option<String>) -> Result<Folder, DomainError>;
/// Obtiene una carpeta por su ID
async fn get_folder(&self, id: &str) -> Result<Folder, DomainError>;
/// Obtiene una carpeta por su ruta
async fn get_folder_by_path(&self, storage_path: &StoragePath) -> Result<Folder, DomainError>;
/// Lista carpetas dentro de una carpeta padre
async fn list_folders(&self, parent_id: Option<&str>) -> Result<Vec<Folder>, DomainError>;
/// Lista carpetas con paginación
async fn list_folders_paginated(
&self,
parent_id: Option<&str>,
offset: usize,
limit: usize,
include_total: bool
) -> Result<(Vec<Folder>, Option<usize>), DomainError>;
/// Renombra una carpeta
async fn rename_folder(&self, id: &str, new_name: String) -> Result<Folder, DomainError>;
/// Mueve una carpeta a otro padre
async fn move_folder(&self, id: &str, new_parent_id: Option<&str>) -> Result<Folder, DomainError>;
/// Elimina una carpeta
async fn delete_folder(&self, id: &str) -> Result<(), DomainError>;
/// Verifica si existe una carpeta en la ruta dada
async fn folder_exists(&self, storage_path: &StoragePath) -> Result<bool, DomainError>;
/// Obtiene la ruta de una carpeta
async fn get_folder_path(&self, id: &str) -> Result<StoragePath, DomainError>;
}
/// Puerto secundario para mapeo de IDs
#[async_trait]
pub trait IdMappingPort: Send + Sync + 'static {
/// Obtiene o crea un ID para una ruta
async fn get_or_create_id(&self, path: &StoragePath) -> Result<String, DomainError>;
/// Obtiene una ruta por su ID
async fn get_path_by_id(&self, id: &str) -> Result<StoragePath, DomainError>;
/// Actualiza la ruta para un ID existente
async fn update_path(&self, id: &str, new_path: &StoragePath) -> Result<(), DomainError>;
/// Elimina un ID del mapeo
async fn remove_id(&self, id: &str) -> Result<(), DomainError>;
/// Guarda cambios pendientes
async fn save_changes(&self) -> Result<(), DomainError>;
}
@@ -0,0 +1,832 @@
use std::sync::Arc;
use thiserror::Error;
use futures::{future::join_all, Future};
use tokio::sync::Semaphore;
use tracing::{info, error};
use crate::application::services::file_service::FileService;
use crate::application::services::folder_service::FolderService;
use crate::domain::services::path_service::StoragePath;
use crate::common::errors::DomainError;
use crate::common::config::AppConfig;
use crate::application::ports::inbound::FolderUseCase;
use crate::application::dtos::file_dto::FileDto;
use crate::application::dtos::folder_dto::FolderDto;
/// Errores específicos para operaciones por lotes
#[derive(Debug, Error)]
#[allow(dead_code)]
pub enum BatchOperationError {
#[error("Error de dominio: {0}")]
Domain(#[from] DomainError),
#[error("Operación cancelada: {0}")]
Cancelled(String),
#[error("Límite de concurrencia excedido: {0}")]
ConcurrencyLimit(String),
#[error("Error en operación del lote: {0} ({1} de {2} completadas)")]
PartialFailure(String, usize, usize),
#[error("Error interno: {0}")]
Internal(String),
}
/// Resultado de una operación por lotes con estadísticas
#[derive(Debug, Clone)]
pub struct BatchResult<T> {
/// Resultados exitosos
pub successful: Vec<T>,
/// Operaciones fallidas con sus errores
pub failed: Vec<(String, String)>,
/// Estadísticas de la operación
pub stats: BatchStats,
}
/// Estadísticas de una operación por lotes
#[derive(Debug, Clone, Default)]
pub struct BatchStats {
/// Número total de operaciones
pub total: usize,
/// Número de operaciones exitosas
pub successful: usize,
/// Número de operaciones fallidas
pub failed: usize,
/// Tiempo total de ejecución en milisegundos
pub execution_time_ms: u128,
/// Concurrencia máxima alcanzada
pub max_concurrency: usize,
}
/// Tipo de entidad para operaciones por lotes
#[derive(Debug, Clone, PartialEq, Eq)]
#[allow(dead_code)]
pub enum EntityType {
File,
Folder,
}
/// Tipo de operación por lotes
#[derive(Debug, Clone, PartialEq, Eq)]
#[allow(dead_code)]
pub enum BatchOperationType {
Create,
Read,
Update,
Delete,
Copy,
Move,
}
/// Identificador para una entidad (ID o ruta)
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub enum EntityIdentifier {
Id(String),
Path(StoragePath),
}
impl EntityIdentifier {
#[allow(dead_code)]
pub fn as_id(&self) -> Option<&str> {
match self {
EntityIdentifier::Id(id) => Some(id),
_ => None,
}
}
#[allow(dead_code)]
pub fn as_path(&self) -> Option<&StoragePath> {
match self {
EntityIdentifier::Path(path) => Some(path),
_ => None,
}
}
}
/// Servicio de operaciones por lotes
pub struct BatchOperationService {
file_service: Arc<FileService>,
folder_service: Arc<FolderService>,
config: AppConfig,
semaphore: Arc<Semaphore>,
}
impl BatchOperationService {
/// Crea una nueva instancia del servicio de operaciones por lotes
pub fn new(
file_service: Arc<FileService>,
folder_service: Arc<FolderService>,
config: AppConfig
) -> Self {
// Limitar la concurrencia basada en la configuración
let max_concurrency = config.concurrency.max_concurrent_files;
Self {
file_service,
folder_service,
config,
semaphore: Arc::new(Semaphore::new(max_concurrency)),
}
}
/// Crea una nueva instancia con la configuración por defecto
pub fn default(
file_service: Arc<FileService>,
folder_service: Arc<FolderService>
) -> Self {
Self::new(file_service, folder_service, AppConfig::default())
}
/// Copia múltiples archivos en paralelo
pub async fn copy_files(
&self,
file_ids: Vec<String>,
target_folder_id: Option<String>,
) -> Result<BatchResult<FileDto>, BatchOperationError> {
info!("Iniciando copia en lote de {} archivos", file_ids.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: file_ids.len(),
..Default::default()
},
};
// Definir la operación a realizar para cada archivo
let operations = file_ids.into_iter().map(|file_id| {
let file_service = self.file_service.clone();
let target_folder = target_folder_id.clone();
let semaphore = self.semaphore.clone();
async move {
// Adquirir permiso del semáforo
let permit = semaphore.acquire().await.unwrap();
let copy_result = file_service.move_file(&file_id, target_folder.clone()).await;
// Liberar el permiso explícitamente (también se libera al hacer drop)
drop(permit);
// Devolver el resultado junto con el ID para identificar éxitos/fallos
(file_id, copy_result)
}
});
// Ejecutar todas las operaciones en paralelo con control de concurrencia
let operation_results = join_all(operations).await;
// Procesar los resultados
for (file_id, operation_result) in operation_results {
match operation_result {
Ok(file) => {
result.successful.push(file);
result.stats.successful += 1;
}
Err(e) => {
result.failed.push((file_id, e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Copia en lote completada: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
/// Mueve múltiples archivos en paralelo
pub async fn move_files(
&self,
file_ids: Vec<String>,
target_folder_id: Option<String>,
) -> Result<BatchResult<FileDto>, BatchOperationError> {
info!("Iniciando movimiento en lote de {} archivos", file_ids.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: file_ids.len(),
..Default::default()
},
};
// Definir la operación a realizar para cada archivo
let operations = file_ids.into_iter().map(|file_id| {
let file_service = self.file_service.clone();
let target_folder = target_folder_id.clone();
let semaphore = self.semaphore.clone();
async move {
// Adquirir permiso del semáforo
let permit = semaphore.acquire().await.unwrap();
let move_result = file_service.move_file(&file_id, target_folder.clone()).await;
// Liberar el permiso explícitamente
drop(permit);
// Devolver el resultado junto con el ID para identificar éxitos/fallos
(file_id, move_result)
}
});
// Ejecutar todas las operaciones en paralelo con control de concurrencia
let operation_results = join_all(operations).await;
// Procesar los resultados
for (file_id, operation_result) in operation_results {
match operation_result {
Ok(file) => {
result.successful.push(file);
result.stats.successful += 1;
}
Err(e) => {
result.failed.push((file_id, e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Movimiento en lote completado: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
/// Elimina múltiples archivos en paralelo
pub async fn delete_files(
&self,
file_ids: Vec<String>,
) -> Result<BatchResult<String>, BatchOperationError> {
info!("Iniciando eliminación en lote de {} archivos", file_ids.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: file_ids.len(),
..Default::default()
},
};
// Definir la operación a realizar para cada archivo
let operations = file_ids.into_iter().map(|file_id| {
let file_service = self.file_service.clone();
let semaphore = self.semaphore.clone();
let id_clone = file_id.clone();
async move {
// Adquirir permiso del semáforo
let permit = semaphore.acquire().await.unwrap();
let delete_result = file_service.delete_file(&file_id).await;
// Liberar el permiso explícitamente
drop(permit);
// Devolver el resultado junto con el ID
(id_clone.clone(), delete_result.map(|_| id_clone))
}
});
// Ejecutar todas las operaciones en paralelo con control de concurrencia
let operation_results = join_all(operations).await;
// Procesar los resultados
for (file_id, operation_result) in operation_results {
match operation_result {
Ok(id) => {
result.successful.push(id);
result.stats.successful += 1;
}
Err(e) => {
result.failed.push((file_id, e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Eliminación en lote completada: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
/// Carga múltiples archivos en paralelo (datos en memoria)
pub async fn get_multiple_files(
&self,
file_ids: Vec<String>,
) -> Result<BatchResult<FileDto>, BatchOperationError> {
info!("Iniciando carga en lote de {} archivos", file_ids.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: file_ids.len(),
..Default::default()
},
};
// Definir la operación a realizar para cada archivo
let operations = file_ids.into_iter().map(|file_id| {
let file_service = self.file_service.clone();
let semaphore = self.semaphore.clone();
async move {
// Adquirir permiso del semáforo
let permit = semaphore.acquire().await.unwrap();
let get_result = file_service.get_file(&file_id).await;
// Liberar el permiso explícitamente
drop(permit);
// Devolver el resultado junto con el ID
(file_id, get_result)
}
});
// Ejecutar todas las operaciones en paralelo con control de concurrencia
let operation_results = join_all(operations).await;
// Procesar los resultados
for (file_id, operation_result) in operation_results {
match operation_result {
Ok(file) => {
result.successful.push(file);
result.stats.successful += 1;
}
Err(e) => {
result.failed.push((file_id, e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Carga en lote completada: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
/// Elimina múltiples carpetas en paralelo
pub async fn delete_folders(
&self,
folder_ids: Vec<String>,
_recursive: bool,
) -> Result<BatchResult<String>, BatchOperationError> {
info!("Iniciando eliminación en lote de {} carpetas", folder_ids.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: folder_ids.len(),
..Default::default()
},
};
// Definir la operación a realizar para cada carpeta
let operations = folder_ids.into_iter().map(|folder_id| {
let folder_service = self.folder_service.clone();
let semaphore = self.semaphore.clone();
let id_clone = folder_id.clone();
async move {
// Adquirir permiso del semáforo
let permit = semaphore.acquire().await.unwrap();
// For both recursive and non-recursive, use the standard delete_folder method
// since FolderUseCase only has a single delete_folder method
let delete_result = folder_service.delete_folder(&folder_id).await;
// Liberar el permiso explícitamente
drop(permit);
// Devolver el resultado junto con el ID
(id_clone.clone(), delete_result.map(|_| id_clone))
}
});
// Ejecutar todas las operaciones en paralelo con control de concurrencia
let operation_results = join_all(operations).await;
// Procesar los resultados
for (folder_id, operation_result) in operation_results {
match operation_result {
Ok(id) => {
result.successful.push(id);
result.stats.successful += 1;
}
Err(e) => {
result.failed.push((folder_id, e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Eliminación en lote de carpetas completada: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
/// Operación genérica de lote para cualquier tipo de función asíncrona
#[allow(dead_code)]
pub async fn generic_batch_operation<T, F, Fut>(
&self,
items: Vec<T>,
operation: F,
) -> Result<BatchResult<T>, BatchOperationError>
where
T: Clone + Send + 'static + std::fmt::Debug,
F: Fn(T, Arc<Semaphore>) -> Fut + Clone + Send + Sync + 'static,
Fut: Future<Output = Result<T, DomainError>> + Send + 'static,
{
info!("Iniciando operación genérica en lote con {} items", items.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: items.len(),
..Default::default()
},
};
// Convertir cada item a una tarea
let tasks = items.iter().map(|item| {
let item_clone = item.clone();
let op = operation.clone();
let semaphore = self.semaphore.clone();
async move {
// La función proporcionada debe manejar la adquisición del semáforo
let op_result = op(item_clone.clone(), semaphore).await;
// Devolver el resultado junto con el item original para identificación
(item_clone, op_result)
}
});
// Ejecutar todas las tareas en paralelo
let operation_results = join_all(tasks).await;
// Procesar resultados
for (item, operation_result) in operation_results {
match operation_result {
Ok(result_item) => {
result.successful.push(result_item);
result.stats.successful += 1;
}
Err(e) => {
// Convertir el item a string para el reporte de error
result.failed.push((format!("{:?}", item), e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Operación genérica en lote completada: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
/// Crear múltiples carpetas en paralelo
pub async fn create_folders(
&self,
folders: Vec<(String, Option<String>)>, // (nombre, padre_id)
) -> Result<BatchResult<FolderDto>, BatchOperationError> {
info!("Iniciando creación en lote de {} carpetas", folders.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: folders.len(),
..Default::default()
},
};
// Definir la operación para cada carpeta
let operations = folders.into_iter().map(|(name, parent_id)| {
let folder_service = self.folder_service.clone();
let semaphore = self.semaphore.clone();
async move {
// Adquirir permiso del semáforo
let permit = semaphore.acquire().await.unwrap();
let dto = crate::application::dtos::folder_dto::CreateFolderDto {
name: name.clone(),
parent_id: parent_id.clone()
};
let create_result = folder_service.create_folder(dto).await;
// Liberar el permiso explícitamente
drop(permit);
// Devolver el resultado con un identificador para los errores
let id = format!("{}:{}", name, parent_id.unwrap_or_default());
(id, create_result)
}
});
// Ejecutar todas las operaciones en paralelo
let operation_results = join_all(operations).await;
// Procesar los resultados
for (id, operation_result) in operation_results {
match operation_result {
Ok(folder) => {
result.successful.push(folder);
result.stats.successful += 1;
}
Err(e) => {
result.failed.push((id, e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Creación en lote de carpetas completada: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
/// Obtener metadatos de múltiples carpetas en paralelo
pub async fn get_multiple_folders(
&self,
folder_ids: Vec<String>,
) -> Result<BatchResult<FolderDto>, BatchOperationError> {
info!("Iniciando carga en lote de {} carpetas", folder_ids.len());
let start_time = std::time::Instant::now();
// Crear estructura para el resultado
let mut result = BatchResult {
successful: Vec::new(),
failed: Vec::new(),
stats: BatchStats {
total: folder_ids.len(),
..Default::default()
},
};
// Definir la operación para cada carpeta
let operations = folder_ids.into_iter().map(|folder_id| {
let folder_service = self.folder_service.clone();
let semaphore = self.semaphore.clone();
async move {
// Adquirir permiso del semáforo
let permit = semaphore.acquire().await.unwrap();
let get_result = folder_service.get_folder(&folder_id).await;
// Liberar el permiso explícitamente
drop(permit);
// Devolver el resultado con su ID
(folder_id, get_result)
}
});
// Ejecutar todas las operaciones en paralelo
let operation_results = join_all(operations).await;
// Procesar los resultados
for (folder_id, operation_result) in operation_results {
match operation_result {
Ok(folder) => {
result.successful.push(folder);
result.stats.successful += 1;
}
Err(e) => {
result.failed.push((folder_id, e.to_string()));
result.stats.failed += 1;
}
}
}
// Completar estadísticas
result.stats.execution_time_ms = start_time.elapsed().as_millis();
result.stats.max_concurrency = self.config.concurrency.max_concurrent_files
.min(result.stats.total);
info!(
"Carga en lote de carpetas completada: {}/{} exitosas en {}ms",
result.stats.successful,
result.stats.total,
result.stats.execution_time_ms
);
Ok(result)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use tokio::sync::Mutex;
use mockall::predicate::*;
use mockall::mock;
// Crear mocks para los servicios
mock! {
FileSvcMock {}
#[async_trait]
impl FileService for FileSvcMock {
async fn create_file(&self, name: String, folder_id: Option<String>, content_type: String, content: Vec<u8>) -> Result<File, DomainError>;
async fn get_file(&self, id: &str) -> Result<File, DomainError>;
async fn delete_file(&self, id: &str) -> Result<(), DomainError>;
async fn move_file(&self, id: &str, target_folder_id: Option<String>) -> Result<File, DomainError>;
async fn copy_file(&self, id: &str, target_folder_id: Option<String>) -> Result<File, DomainError>;
async fn list_files(&self, folder_id: Option<&str>) -> Result<Vec<File>, DomainError>;
async fn get_file_content(&self, id: &str) -> Result<Vec<u8>, DomainError>;
}
}
mock! {
FolderSvcMock {}
#[async_trait]
impl FolderService for FolderSvcMock {
async fn create_folder(&self, name: String, parent_id: Option<String>) -> Result<Folder, DomainError>;
async fn get_folder(&self, id: &str) -> Result<Folder, DomainError>;
async fn delete_folder(&self, id: &str) -> Result<(), DomainError>;
async fn delete_folder_recursive(&self, id: &str) -> Result<(), DomainError>;
async fn list_folders(&self, parent_id: Option<&str>) -> Result<Vec<Folder>, DomainError>;
async fn move_folder(&self, id: &str, target_parent_id: Option<String>) -> Result<Folder, DomainError>;
}
}
#[tokio::test]
async fn test_batch_delete_files() {
// Crear mocks
let mut file_service = MockFileSvcMock::new();
// Configurar comportamiento esperado
file_service.expect_delete_file()
.times(3)
.returning(|id| {
if id == "error-id" {
Err(DomainError::not_found("FileService", "File not found"))
} else {
Ok(())
}
});
// Crear el servicio de batch con los mocks
let batch_service = BatchOperationService::new(
Arc::new(file_service),
Arc::new(MockFolderSvcMock::new()),
AppConfig::default()
);
// Ejecutar la operación de batch
let file_ids = vec![
"id1".to_string(),
"id2".to_string(),
"error-id".to_string()
];
let result = batch_service.delete_files(file_ids).await.unwrap();
// Verificar los resultados
assert_eq!(result.stats.total, 3);
assert_eq!(result.stats.successful, 2);
assert_eq!(result.stats.failed, 1);
assert_eq!(result.successful.len(), 2);
assert_eq!(result.failed.len(), 1);
assert_eq!(result.failed[0].0, "error-id");
}
#[tokio::test]
async fn test_generic_batch_operation() {
// Crear el servicio de batch
let batch_service = BatchOperationService::new(
Arc::new(MockFileSvcMock::new()),
Arc::new(MockFolderSvcMock::new()),
AppConfig::default()
);
// Definir una operación genérica de prueba
let operation = |item: i32, semaphore: Arc<Semaphore>| async move {
// Adquirir y liberar el semáforo
let _permit = semaphore.acquire().await.unwrap();
if item % 2 == 0 {
// Simular éxito para números pares
Ok(item * 2)
} else {
// Simular error para números impares
Err(DomainError::invalid_input("Test", "Odd number not allowed"))
}
};
// Ejecutar la operación de batch
let items = vec![1, 2, 3, 4, 5];
let result = batch_service.generic_batch_operation(items, operation).await.unwrap();
// Verificar los resultados
assert_eq!(result.stats.total, 5);
assert_eq!(result.stats.successful, 2);
assert_eq!(result.stats.failed, 3);
// Los números pares deberían estar en los éxitos, duplicados
assert!(result.successful.contains(&4)); // 2*2
assert!(result.successful.contains(&8)); // 4*2
// Los impares deberían estar en los fallos
assert_eq!(result.failed.len(), 3);
}
}
+142 -79
View File
@@ -1,16 +1,81 @@
use std::sync::Arc;
use thiserror::Error;
use async_trait::async_trait;
use crate::domain::repositories::file_repository::{FileRepository, FileRepositoryResult};
use crate::domain::repositories::file_repository::FileRepositoryError;
use crate::application::dtos::file_dto::FileDto;
use crate::application::ports::inbound::FileUseCase;
use crate::application::ports::outbound::FileStoragePort;
use crate::common::errors::DomainError;
use futures::Stream;
use bytes::Bytes;
/// Errores específicos del servicio de archivos
#[derive(Debug, Error)]
pub enum FileServiceError {
#[error("Archivo no encontrado: {0}")]
NotFound(String),
#[error("Archivo ya existe: {0}")]
Conflict(String),
#[error("Error de acceso al archivo: {0}")]
AccessError(String),
#[error("Ruta de archivo inválida: {0}")]
InvalidPath(String),
#[error("Error interno: {0}")]
InternalError(String),
}
impl From<FileRepositoryError> for FileServiceError {
fn from(err: FileRepositoryError) -> Self {
match err {
FileRepositoryError::NotFound(id) => FileServiceError::NotFound(id),
FileRepositoryError::AlreadyExists(path) => FileServiceError::Conflict(path),
FileRepositoryError::InvalidPath(path) => FileServiceError::InvalidPath(path),
FileRepositoryError::IoError(e) => FileServiceError::AccessError(e.to_string()),
FileRepositoryError::Timeout(msg) => FileServiceError::AccessError(format!("Operación expiró: {}", msg)),
_ => FileServiceError::InternalError(err.to_string()),
}
}
}
impl From<DomainError> for FileServiceError {
fn from(err: DomainError) -> Self {
match err.kind {
crate::common::errors::ErrorKind::NotFound => FileServiceError::NotFound(err.to_string()),
crate::common::errors::ErrorKind::AlreadyExists => FileServiceError::Conflict(err.to_string()),
crate::common::errors::ErrorKind::InvalidInput => FileServiceError::InvalidPath(err.to_string()),
crate::common::errors::ErrorKind::AccessDenied => FileServiceError::AccessError(err.to_string()),
_ => FileServiceError::InternalError(err.to_string()),
}
}
}
impl From<FileServiceError> for DomainError {
fn from(err: FileServiceError) -> Self {
match err {
FileServiceError::NotFound(id) => DomainError::not_found("File", id),
FileServiceError::Conflict(path) => DomainError::already_exists("File", path),
FileServiceError::InvalidPath(path) => DomainError::validation_error("File", format!("Invalid path: {}", path)),
FileServiceError::AccessError(msg) => DomainError::access_denied("File", msg),
FileServiceError::InternalError(msg) => DomainError::internal_error("File", msg),
}
}
}
pub type FileServiceResult<T> = Result<T, FileServiceError>;
/// Service for file operations
pub struct FileService {
file_repository: Arc<dyn FileRepository>,
file_repository: Arc<dyn FileStoragePort>,
}
impl FileService {
/// Creates a new file service
pub fn new(file_repository: Arc<dyn FileRepository>) -> Self {
pub fn new(file_repository: Arc<dyn FileStoragePort>) -> Self {
Self { file_repository }
}
@@ -21,105 +86,103 @@ impl FileService {
folder_id: Option<String>,
content_type: String,
content: Vec<u8>,
) -> FileRepositoryResult<FileDto>
) -> FileServiceResult<FileDto>
{
let file = self.file_repository.save_file_from_bytes(name, folder_id, content_type, content).await?;
let file = self.file_repository.save_file(name, folder_id, content_type, content).await
.map_err(FileServiceError::from)?;
Ok(FileDto::from(file))
}
/// Gets a file by ID
pub async fn get_file(&self, id: &str) -> FileRepositoryResult<FileDto> {
let file = self.file_repository.get_file_by_id(id).await?;
pub async fn get_file(&self, id: &str) -> FileServiceResult<FileDto> {
let file = self.file_repository.get_file(id).await
.map_err(FileServiceError::from)?;
Ok(FileDto::from(file))
}
/// Lists files in a folder
pub async fn list_files(&self, folder_id: Option<&str>) -> FileRepositoryResult<Vec<FileDto>> {
let files = self.file_repository.list_files(folder_id).await?;
pub async fn list_files(&self, folder_id: Option<&str>) -> FileServiceResult<Vec<FileDto>> {
let files = self.file_repository.list_files(folder_id).await
.map_err(FileServiceError::from)?;
Ok(files.into_iter().map(FileDto::from).collect())
}
/// Deletes a file
pub async fn delete_file(&self, id: &str) -> FileRepositoryResult<()> {
pub async fn delete_file(&self, id: &str) -> FileServiceResult<()> {
self.file_repository.delete_file(id).await
.map_err(FileServiceError::from)
}
/// Gets file content
pub async fn get_file_content(&self, id: &str) -> FileRepositoryResult<Vec<u8>> {
/// Gets file content as bytes - use for small files only
pub async fn get_file_content(&self, id: &str) -> FileServiceResult<Vec<u8>> {
self.file_repository.get_file_content(id).await
.map_err(FileServiceError::from)
}
/// Moves a file to a new folder implementing direct save with new location without deleting first
pub async fn move_file(&self, file_id: &str, folder_id: Option<String>) -> FileRepositoryResult<FileDto> {
// Get the current file complete info
let source_file = match self.file_repository.get_file_by_id(file_id).await {
Ok(f) => f,
Err(e) => {
tracing::error!("Error al obtener archivo (ID: {}): {}", file_id, e);
return Err(e);
}
};
/// Gets file content as stream - better for large files
pub async fn get_file_stream(&self, id: &str) -> FileServiceResult<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>> {
self.file_repository.get_file_stream(id).await
.map_err(FileServiceError::from)
}
tracing::info!("Moviendo archivo: {} (ID: {}) de carpeta: {:?} a carpeta: {:?}",
source_file.name, file_id, source_file.folder_id, folder_id);
/// Moves a file to a new folder using filesystem operations directly
pub async fn move_file(&self, file_id: &str, folder_id: Option<String>) -> FileServiceResult<FileDto> {
tracing::info!("Moviendo archivo con ID: {} a carpeta: {:?}", file_id, folder_id);
// Special handling for PDF files
let is_pdf = source_file.name.to_lowercase().ends_with(".pdf");
if is_pdf {
tracing::info!("Moviendo un archivo PDF: {}", source_file.name);
}
// No hacer nada si ya estamos en la carpeta de destino
if source_file.folder_id == folder_id {
tracing::info!("El archivo ya está en la carpeta de destino, no es necesario moverlo");
return Ok(FileDto::from(source_file));
}
// Step 1: Get file content
tracing::info!("Leyendo contenido del archivo: {}", source_file.name);
let content = match self.file_repository.get_file_content(file_id).await {
Ok(content) => {
tracing::info!("Contenido del archivo leído correctamente: {} bytes", content.len());
content
},
Err(e) => {
tracing::error!("Error al leer el contenido del archivo {}: {}", file_id, e);
return Err(e);
}
};
// Step 2: Save the file to the new location with a new ID
tracing::info!("Guardando archivo en nueva ubicación: {} en carpeta: {:?}", source_file.name, folder_id);
let new_file = match self.file_repository.save_file_from_bytes(
source_file.name.clone(),
folder_id.clone(),
source_file.mime_type.clone(),
content
).await {
Ok(file) => {
tracing::info!("Archivo guardado en nueva ubicación con ID: {}", file.id);
file
},
Err(e) => {
tracing::error!("Error al guardar archivo en nueva ubicación: {}", e);
return Err(e);
}
};
// Step 3: Only after ensuring new file is saved, try to delete the old file
// If this fails, it's not critical - we already have the file in the new location
tracing::info!("Eliminando archivo original con ID: {}", file_id);
match self.file_repository.delete_file(file_id).await {
Ok(_) => tracing::info!("Archivo original eliminado correctamente"),
Err(e) => {
tracing::warn!("Error al eliminar archivo original (ID: {}): {} - archivo duplicado posible", file_id, e);
// Continue even if delete fails - at worst we'll have duplicate files
}
}
// Usar la implementación eficiente del repositorio que utiliza rename
let moved_file = self.file_repository.move_file(file_id, folder_id).await
.map_err(|e| {
tracing::error!("Error al mover archivo (ID: {}): {}", file_id, e);
FileServiceError::from(e)
})?;
tracing::info!("Archivo movido exitosamente: {} (ID: {}) a carpeta: {:?}",
new_file.name, new_file.id, folder_id);
moved_file.name(), moved_file.id(), moved_file.folder_id());
Ok(FileDto::from(new_file))
Ok(FileDto::from(moved_file))
}
}
#[async_trait]
impl FileUseCase for FileService {
async fn upload_file(
&self,
name: String,
folder_id: Option<String>,
content_type: String,
content: Vec<u8>,
) -> Result<FileDto, DomainError> {
FileService::upload_file_from_bytes(self, name, folder_id, content_type, content).await
.map_err(DomainError::from)
}
async fn get_file(&self, id: &str) -> Result<FileDto, DomainError> {
FileService::get_file(self, id).await
.map_err(DomainError::from)
}
async fn list_files(&self, folder_id: Option<&str>) -> Result<Vec<FileDto>, DomainError> {
FileService::list_files(self, folder_id).await
.map_err(DomainError::from)
}
async fn delete_file(&self, id: &str) -> Result<(), DomainError> {
FileService::delete_file(self, id).await
.map_err(DomainError::from)
}
async fn get_file_content(&self, id: &str) -> Result<Vec<u8>, DomainError> {
FileService::get_file_content(self, id).await
.map_err(DomainError::from)
}
async fn get_file_stream(&self, id: &str) -> Result<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>, DomainError> {
FileService::get_file_stream(self, id).await
.map_err(DomainError::from)
}
async fn move_file(&self, file_id: &str, folder_id: Option<String>) -> Result<FileDto, DomainError> {
FileService::move_file(self, file_id, folder_id).await
.map_err(DomainError::from)
}
}
+236 -38
View File
@@ -1,68 +1,266 @@
use std::path::PathBuf;
use std::sync::Arc;
use crate::domain::repositories::folder_repository::{FolderRepository, FolderRepositoryResult};
use async_trait::async_trait;
use crate::domain::services::path_service::StoragePath;
use crate::application::dtos::folder_dto::{CreateFolderDto, RenameFolderDto, MoveFolderDto, FolderDto};
use crate::application::ports::inbound::FolderUseCase;
use crate::application::ports::outbound::FolderStoragePort;
use crate::application::transactions::storage_transaction::StorageTransaction;
use crate::common::errors::{DomainError, ErrorKind, ErrorContext};
/// Service for folder operations
/// Implementación del caso de uso para operaciones de carpetas
pub struct FolderService {
folder_repository: Arc<dyn FolderRepository>,
folder_storage: Arc<dyn FolderStoragePort>,
}
impl FolderService {
/// Creates a new folder service
pub fn new(folder_repository: Arc<dyn FolderRepository>) -> Self {
Self { folder_repository }
/// Crea un nuevo servicio de carpetas
pub fn new(folder_storage: Arc<dyn FolderStoragePort>) -> Self {
Self { folder_storage }
}
}
/// Creates a new folder
pub async fn create_folder(&self, dto: CreateFolderDto) -> FolderRepositoryResult<FolderDto> {
let parent_path = match &dto.parent_id {
Some(parent_id) => {
let parent = self.folder_repository.get_folder_by_id(parent_id).await?;
Some(parent.path)
},
None => None
};
#[async_trait]
impl FolderUseCase for FolderService {
/// Crea una nueva carpeta
async fn create_folder(&self, dto: CreateFolderDto) -> Result<FolderDto, DomainError> {
// Validación de entrada
if dto.name.is_empty() {
return Err(DomainError::new(
ErrorKind::InvalidInput,
"Folder",
"Folder name cannot be empty"
));
}
let folder = self.folder_repository.create_folder(dto.name, parent_path).await?;
// Si se proporciona un parent_id, verificar que existe
if let Some(parent_id) = &dto.parent_id {
let parent_exists = self.folder_storage.get_folder(parent_id).await.is_ok();
if !parent_exists {
return Err(DomainError::not_found("Folder", parent_id));
}
}
// Crear la carpeta
let folder = self.folder_storage.create_folder(dto.name, dto.parent_id)
.await
.with_context(|| "Failed to create folder")?;
// Convertir a DTO
Ok(FolderDto::from(folder))
}
/// Gets a folder by ID
pub async fn get_folder(&self, id: &str) -> FolderRepositoryResult<FolderDto> {
let folder = self.folder_repository.get_folder_by_id(id).await?;
/// Obtiene una carpeta por su ID
async fn get_folder(&self, id: &str) -> Result<FolderDto, DomainError> {
let folder = self.folder_storage.get_folder(id)
.await
.with_context(|| format!("Failed to get folder with ID: {}", id))?;
Ok(FolderDto::from(folder))
}
/// Gets a folder by path
#[allow(dead_code)]
pub async fn get_folder_by_path(&self, path: &str) -> FolderRepositoryResult<FolderDto> {
let path_buf = PathBuf::from(path);
let folder = self.folder_repository.get_folder_by_path(&path_buf).await?;
/// Obtiene una carpeta por su ruta
async fn get_folder_by_path(&self, path: &str) -> Result<FolderDto, DomainError> {
// Convertir la ruta de string a StoragePath
let storage_path = StoragePath::from_string(path);
let folder = self.folder_storage.get_folder_by_path(&storage_path)
.await
.with_context(|| format!("Failed to get folder at path: {}", path))?;
Ok(FolderDto::from(folder))
}
/// Lists folders in a parent folder
pub async fn list_folders(&self, parent_id: Option<&str>) -> FolderRepositoryResult<Vec<FolderDto>> {
let folders = self.folder_repository.list_folders(parent_id).await?;
/// Lista carpetas dentro de una carpeta padre
async fn list_folders(&self, parent_id: Option<&str>) -> Result<Vec<FolderDto>, DomainError> {
let folders = self.folder_storage.list_folders(parent_id)
.await
.with_context(|| format!("Failed to list folders in parent: {:?}", parent_id))?;
// Convertir a DTOs
Ok(folders.into_iter().map(FolderDto::from).collect())
}
/// Renames a folder
pub async fn rename_folder(&self, id: &str, dto: RenameFolderDto) -> FolderRepositoryResult<FolderDto> {
let folder = self.folder_repository.rename_folder(id, dto.name).await?;
/// Lista carpetas con paginación
async fn list_folders_paginated(
&self,
parent_id: Option<&str>,
pagination: &crate::application::dtos::pagination::PaginationRequestDto
) -> Result<crate::application::dtos::pagination::PaginatedResponseDto<FolderDto>, DomainError> {
// Validar y ajustar la paginación
let pagination = pagination.validate_and_adjust();
// Obtener carpetas paginadas y conteo total
let (folders, total_items) = self.folder_storage.list_folders_paginated(
parent_id,
pagination.offset(),
pagination.limit(),
true // Siempre incluir total para mejor UX
)
.await
.with_context(|| format!("Failed to list folders with pagination in parent: {:?}", parent_id))?;
// El total es necesario para calcular la paginación
let total = total_items.unwrap_or(folders.len());
// Convertir a PaginatedResponseDto
let response = crate::application::dtos::pagination::PaginatedResponseDto::new(
folders.into_iter().map(FolderDto::from).collect(),
pagination.page,
pagination.page_size,
total
);
Ok(response)
}
/// Renombra una carpeta
async fn rename_folder(&self, id: &str, dto: RenameFolderDto) -> Result<FolderDto, DomainError> {
// Validación de entrada
if dto.name.is_empty() {
return Err(DomainError::new(
ErrorKind::InvalidInput,
"Folder",
"New folder name cannot be empty"
));
}
// Verificar que la carpeta existe
let existing_folder = self.folder_storage.get_folder(id)
.await
.with_context(|| format!("Failed to get folder with ID: {} for renaming", id))?;
// Crear transacción para renombrar
let mut transaction = StorageTransaction::new("rename_folder");
// Operación principal: renombrar carpeta
// Clone all values to avoid lifetime issues
let folder_storage = self.folder_storage.clone();
let id_owned = id.to_string();
let name_owned = dto.name.clone();
// Create future with owned values
let rename_op = async move {
folder_storage.rename_folder(&id_owned, name_owned).await?;
Ok(())
};
let rollback_op = {
let original_name = existing_folder.name().to_string();
let storage = self.folder_storage.clone();
let id_clone = id.to_string();
async move {
// En caso de fallo, restaurar el nombre original
storage.rename_folder(&id_clone, original_name).await
.map(|_| ())
.map_err(|e| DomainError::new(
ErrorKind::InternalError,
"Folder",
format!("Failed to rollback folder rename: {}", e)
))
}
};
// Añadir a la transacción
transaction.add_operation(rename_op, rollback_op);
// Ejecutar transacción
transaction.commit().await?;
// Obtener la carpeta renombrada
let folder = self.folder_storage.get_folder(id)
.await
.with_context(|| format!("Failed to get renamed folder with ID: {}", id))?;
Ok(FolderDto::from(folder))
}
/// Moves a folder to a new parent
pub async fn move_folder(&self, id: &str, dto: MoveFolderDto) -> FolderRepositoryResult<FolderDto> {
let folder = self.folder_repository.move_folder(id, dto.parent_id.as_deref()).await?;
/// Mueve una carpeta a un nuevo padre
async fn move_folder(&self, id: &str, dto: MoveFolderDto) -> Result<FolderDto, DomainError> {
// Verificar que la carpeta origen existe
let source_folder = self.folder_storage.get_folder(id)
.await
.with_context(|| format!("Failed to get folder with ID: {} for moving", id))?;
// Si se especifica un parent_id, verificar que existe
if let Some(parent_id) = &dto.parent_id {
// Verificar que no estamos intentando mover la carpeta a sí misma o a uno de sus descendientes
if parent_id == id {
return Err(DomainError::new(
ErrorKind::InvalidInput,
"Folder",
"Cannot move a folder into itself"
));
}
// Verificar que el destino existe
let parent_exists = self.folder_storage.get_folder(parent_id).await.is_ok();
if !parent_exists {
return Err(DomainError::not_found("Folder", parent_id));
}
// TODO: Idealmente deberíamos verificar toda la jerarquía para evitar ciclos
}
// Crear transacción para mover
let mut transaction = StorageTransaction::new("move_folder");
// Operación principal: mover carpeta
// Clone all values to avoid lifetime issues
let folder_storage = self.folder_storage.clone();
let id_owned = id.to_string();
// Get parent ID as owned string or None
let parent_id_owned = dto.parent_id.as_ref().map(|p| p.to_string());
// Create future with owned values
let move_op = async move {
// Convert Option<String> to Option<&str>
let parent_ref = parent_id_owned.as_deref();
folder_storage.move_folder(&id_owned, parent_ref).await?;
Ok(())
};
let rollback_op = {
let original_parent_id = source_folder.parent_id().map(String::from);
let storage = self.folder_storage.clone();
let id_clone = id.to_string();
async move {
// En caso de fallo, restaurar la ubicación original
storage.move_folder(&id_clone, original_parent_id.as_deref()).await
.map(|_| ())
.map_err(|e| DomainError::new(
ErrorKind::InternalError,
"Folder",
format!("Failed to rollback folder move: {}", e)
))
}
};
// Añadir a la transacción
transaction.add_operation(move_op, rollback_op);
// Ejecutar transacción
transaction.commit().await?;
// Obtener la carpeta movida
let folder = self.folder_storage.get_folder(id)
.await
.with_context(|| format!("Failed to get moved folder with ID: {}", id))?;
Ok(FolderDto::from(folder))
}
/// Deletes a folder
pub async fn delete_folder(&self, id: &str) -> FolderRepositoryResult<()> {
self.folder_repository.delete_folder(id).await
/// Elimina una carpeta
async fn delete_folder(&self, id: &str) -> Result<(), DomainError> {
// Verificar que la carpeta existe
let _folder = self.folder_storage.get_folder(id)
.await
.with_context(|| format!("Failed to get folder with ID: {} for deletion", id))?;
// En una implementación real, podríamos verificar permisos, dependencias, etc.
// Eliminar la carpeta
self.folder_storage.delete_folder(id)
.await
.with_context(|| format!("Failed to delete folder with ID: {}", id))
}
}
+2
View File
@@ -1,4 +1,6 @@
pub mod file_service;
pub mod folder_service;
pub mod i18n_application_service;
pub mod storage_mediator;
pub mod batch_operations;
@@ -0,0 +1,291 @@
use std::path::{Path, PathBuf};
use std::sync::Arc;
use async_trait::async_trait;
use thiserror::Error;
use crate::domain::entities::folder::Folder;
use crate::domain::repositories::folder_repository::{FolderRepository, FolderRepositoryError};
use crate::domain::repositories::file_repository::FileRepositoryError;
use crate::domain::services::path_service::{PathService, StoragePath};
use crate::application::ports::outbound::IdMappingPort;
/// Errores específicos del mediador de almacenamiento
#[derive(Debug, Error)]
pub enum StorageMediatorError {
#[error("Entidad no encontrada: {0}")]
NotFound(String),
#[error("Entidad ya existe: {0}")]
AlreadyExists(String),
#[error("Ruta inválida: {0}")]
InvalidPath(String),
#[error("Error de acceso: {0}")]
AccessError(String),
#[error("Error interno: {0}")]
InternalError(String),
#[error("Error de dominio: {0}")]
DomainError(#[from] crate::common::errors::DomainError),
}
impl From<FolderRepositoryError> for StorageMediatorError {
fn from(err: FolderRepositoryError) -> Self {
match err {
FolderRepositoryError::NotFound(id) => StorageMediatorError::NotFound(id),
FolderRepositoryError::AlreadyExists(path) => StorageMediatorError::AlreadyExists(path),
FolderRepositoryError::InvalidPath(path) => StorageMediatorError::InvalidPath(path),
FolderRepositoryError::IoError(e) => StorageMediatorError::AccessError(e.to_string()),
_ => StorageMediatorError::InternalError(err.to_string()),
}
}
}
impl From<FileRepositoryError> for StorageMediatorError {
fn from(err: FileRepositoryError) -> Self {
match err {
FileRepositoryError::NotFound(id) => StorageMediatorError::NotFound(id),
FileRepositoryError::AlreadyExists(path) => StorageMediatorError::AlreadyExists(path),
FileRepositoryError::InvalidPath(path) => StorageMediatorError::InvalidPath(path),
FileRepositoryError::IoError(e) => StorageMediatorError::AccessError(e.to_string()),
_ => StorageMediatorError::InternalError(err.to_string()),
}
}
}
/// Tipo de resultado para las operaciones del mediador
pub type StorageMediatorResult<T> = Result<T, StorageMediatorError>;
/// Interfaz del servicio mediador entre repositorios de archivos y carpetas
#[async_trait]
pub trait StorageMediator: Send + Sync + 'static {
/// Obtiene la ruta de una carpeta por su ID
async fn get_folder_path(&self, folder_id: &str) -> StorageMediatorResult<PathBuf>;
/// Obtiene la ruta de dominio de una carpeta por su ID
async fn get_folder_storage_path(&self, folder_id: &str) -> StorageMediatorResult<StoragePath>;
/// Obtiene todos los detalles de una carpeta por su ID
async fn get_folder(&self, folder_id: &str) -> StorageMediatorResult<Folder>;
/// Verifica si existe un archivo en una ruta específica
async fn file_exists_at_path(&self, path: &Path) -> StorageMediatorResult<bool>;
/// Verifica si existe un archivo en una ruta de dominio específica
async fn file_exists_at_storage_path(&self, storage_path: &StoragePath) -> StorageMediatorResult<bool>;
/// Verifica si existe una carpeta en una ruta específica
async fn folder_exists_at_path(&self, path: &Path) -> StorageMediatorResult<bool>;
/// Verifica si existe una carpeta en una ruta de dominio específica
async fn folder_exists_at_storage_path(&self, storage_path: &StoragePath) -> StorageMediatorResult<bool>;
/// Resuelve una ruta relativa a absoluta (legacy)
fn resolve_path(&self, relative_path: &Path) -> PathBuf;
/// Resuelve una ruta de dominio a una ruta física absoluta
fn resolve_storage_path(&self, storage_path: &StoragePath) -> PathBuf;
/// Crea un directorio si no existe (legacy)
async fn ensure_directory(&self, path: &Path) -> StorageMediatorResult<()>;
/// Crea un directorio si no existe
async fn ensure_storage_directory(&self, storage_path: &StoragePath) -> StorageMediatorResult<()>;
}
/// Implementación concreta del mediador de almacenamiento
pub struct FileSystemStorageMediator {
folder_repository: Arc<dyn FolderRepository>,
path_service: Arc<PathService>,
id_mapping: Arc<dyn IdMappingPort>,
}
impl FileSystemStorageMediator {
pub fn new(folder_repository: Arc<dyn FolderRepository>, path_service: Arc<PathService>, id_mapping: Arc<dyn IdMappingPort>) -> Self {
Self { folder_repository, path_service, id_mapping }
}
/// Creates a stub implementation for initialization bootstrapping
pub fn new_stub() -> StubStorageMediator {
StubStorageMediator::new()
}
}
/// Stub implementation for initialization dependency issues
pub struct StubStorageMediator {
#[allow(dead_code)]
_path_service: Arc<PathService>,
}
impl StubStorageMediator {
pub fn new() -> Self {
let root_path = PathBuf::from("/tmp");
let path_service = Arc::new(PathService::new(root_path));
Self { _path_service: path_service }
}
}
#[async_trait]
impl StorageMediator for StubStorageMediator {
async fn get_folder_path(&self, _folder_id: &str) -> StorageMediatorResult<PathBuf> {
// Return a stub path
Ok(PathBuf::from("/tmp"))
}
async fn get_folder_storage_path(&self, _folder_id: &str) -> StorageMediatorResult<StoragePath> {
// Return a stub storage path
Ok(StoragePath::root())
}
async fn get_folder(&self, _folder_id: &str) -> StorageMediatorResult<Folder> {
// This is a stub that should never be called during initialization
Err(StorageMediatorError::NotFound("Stub not implemented".to_string()))
}
async fn file_exists_at_path(&self, _path: &Path) -> StorageMediatorResult<bool> {
Ok(false)
}
async fn file_exists_at_storage_path(&self, _storage_path: &StoragePath) -> StorageMediatorResult<bool> {
Ok(false)
}
async fn folder_exists_at_path(&self, _path: &Path) -> StorageMediatorResult<bool> {
Ok(false)
}
async fn folder_exists_at_storage_path(&self, _storage_path: &StoragePath) -> StorageMediatorResult<bool> {
Ok(false)
}
fn resolve_path(&self, _relative_path: &Path) -> PathBuf {
PathBuf::from("/tmp")
}
fn resolve_storage_path(&self, _storage_path: &StoragePath) -> PathBuf {
PathBuf::from("/tmp")
}
async fn ensure_directory(&self, _path: &Path) -> StorageMediatorResult<()> {
Ok(())
}
async fn ensure_storage_directory(&self, _storage_path: &StoragePath) -> StorageMediatorResult<()> {
Ok(())
}
}
#[async_trait]
impl StorageMediator for FileSystemStorageMediator {
async fn get_folder_path(&self, folder_id: &str) -> StorageMediatorResult<PathBuf> {
let folder = self.folder_repository.get_folder_by_id(folder_id).await
.map_err(StorageMediatorError::from)?;
// Need to get the path from folder ID
let storage_path = self.id_mapping.get_path_by_id(folder.id()).await
.map_err(StorageMediatorError::from)?;
// Convert StoragePath to PathBuf
let path_buf = self.path_service.resolve_path(&storage_path);
Ok(path_buf)
}
async fn get_folder_storage_path(&self, folder_id: &str) -> StorageMediatorResult<StoragePath> {
let folder = self.folder_repository.get_folder_by_id(folder_id).await
.map_err(StorageMediatorError::from)?;
// Get path by folder ID - will already be a StoragePath
let storage_path = self.id_mapping.get_path_by_id(folder.id()).await
.map_err(StorageMediatorError::from)?;
Ok(storage_path)
}
async fn get_folder(&self, folder_id: &str) -> StorageMediatorResult<Folder> {
let folder = self.folder_repository.get_folder_by_id(folder_id).await
.map_err(StorageMediatorError::from)?;
Ok(folder)
}
async fn file_exists_at_path(&self, path: &Path) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_path(path);
// Verificar si existe como archivo (no como directorio)
let exists = abs_path.exists() && abs_path.is_file();
Ok(exists)
}
async fn file_exists_at_storage_path(&self, storage_path: &StoragePath) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_storage_path(storage_path);
// Verificar si existe como archivo (no como directorio)
let exists = abs_path.exists() && abs_path.is_file();
Ok(exists)
}
async fn folder_exists_at_path(&self, path: &Path) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_path(path);
// Verificar si existe como directorio
let exists = abs_path.exists() && abs_path.is_dir();
Ok(exists)
}
async fn folder_exists_at_storage_path(&self, storage_path: &StoragePath) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_storage_path(storage_path);
// Verificar si existe como directorio
let exists = abs_path.exists() && abs_path.is_dir();
Ok(exists)
}
fn resolve_path(&self, relative_path: &Path) -> PathBuf {
// Legacy method using PathBuf
let path_str = relative_path.to_string_lossy().to_string();
let storage_path = StoragePath::from_string(&path_str);
self.path_service.resolve_path(&storage_path)
}
fn resolve_storage_path(&self, storage_path: &StoragePath) -> PathBuf {
self.path_service.resolve_path(storage_path)
}
async fn ensure_directory(&self, path: &Path) -> StorageMediatorResult<()> {
let abs_path = self.resolve_path(path);
// Crear directorios si no existen
if !abs_path.exists() {
tokio::fs::create_dir_all(&abs_path).await
.map_err(|e| StorageMediatorError::AccessError(format!("No se pudo crear el directorio: {}", e)))?;
} else if !abs_path.is_dir() {
return Err(StorageMediatorError::InvalidPath(
format!("La ruta existe pero no es un directorio: {}", abs_path.display())
));
}
Ok(())
}
async fn ensure_storage_directory(&self, storage_path: &StoragePath) -> StorageMediatorResult<()> {
let abs_path = self.resolve_storage_path(storage_path);
// Crear directorios si no existen
if !abs_path.exists() {
tokio::fs::create_dir_all(&abs_path).await
.map_err(|e| StorageMediatorError::AccessError(format!("No se pudo crear el directorio: {}", e)))?;
} else if !abs_path.is_dir() {
return Err(StorageMediatorError::InvalidPath(
format!("La ruta existe pero no es un directorio: {}", abs_path.display())
));
}
Ok(())
}
}
+1
View File
@@ -0,0 +1 @@
pub mod storage_transaction;
@@ -0,0 +1,131 @@
use std::future::Future;
use std::pin::Pin;
use crate::common::errors::{DomainError, ErrorKind};
/// Tipo para operaciones y rollbacks asíncronos
type TransactionOp = Pin<Box<dyn Future<Output = Result<(), DomainError>> + Send>>;
/// Transacción para operaciones de almacenamiento
/// Permite definir un conjunto de operaciones y sus rollbacks correspondientes
pub struct StorageTransaction {
/// Operaciones a ejecutar
operations: Vec<Box<dyn FnOnce() -> TransactionOp + Send>>,
/// Operaciones de rollback para revertir cambios en caso de error
rollbacks: Vec<Box<dyn FnOnce() -> TransactionOp + Send>>,
/// Nombre de la transacción para logging
name: String,
}
impl StorageTransaction {
/// Crea una nueva transacción
pub fn new(name: &str) -> Self {
Self {
operations: Vec::new(),
rollbacks: Vec::new(),
name: name.to_string(),
}
}
/// Añade una operación a la transacción con su correspondiente rollback
pub fn add_operation<F, R>(&mut self, operation: F, rollback: R)
where
F: Future<Output = Result<(), DomainError>> + Send + 'static,
R: Future<Output = Result<(), DomainError>> + Send + 'static,
{
self.operations.push(Box::new(move || Box::pin(operation)));
self.rollbacks.push(Box::new(move || Box::pin(rollback)));
}
/// Añade una operación sin rollback (para limpieza o logging)
#[allow(dead_code)]
pub fn add_finalizer<F>(&mut self, finalizer: F)
where
F: Future<Output = Result<(), DomainError>> + Send + 'static,
{
// El rollback es una operación nula
let noop = async { Ok(()) };
self.operations.push(Box::new(move || Box::pin(finalizer)));
self.rollbacks.push(Box::new(move || Box::pin(noop)));
}
/// Ejecuta la transacción aplicando todas las operaciones en orden
/// Si alguna falla, ejecuta los rollbacks en orden inverso
pub async fn commit(mut self) -> Result<(), DomainError> {
tracing::debug!("Iniciando transacción: {}", self.name);
let mut completed_ops = Vec::new();
// Extraer operaciones para evitar problemas de propiedad
let operations = std::mem::take(&mut self.operations);
let transaction_name = self.name.clone();
// Ejecutar operaciones
for (i, op) in operations.into_iter().enumerate() {
match op().await {
Ok(()) => {
completed_ops.push(i);
tracing::trace!("Operación {} completada en transacción: {}", i, transaction_name);
}
Err(e) => {
tracing::error!("Error en operación {} de transacción {}: {}", i, transaction_name, e);
// Ejecutar rollbacks para las operaciones completadas en orden inverso
self.rollback(completed_ops).await?;
return Err(DomainError::new(
ErrorKind::InternalError,
"Transaction",
format!("Falló la transacción '{}': {}", transaction_name, e)
).with_source(e));
}
}
}
tracing::debug!("Transacción completada exitosamente: {}", transaction_name);
Ok(())
}
/// Ejecuta rollbacks para las operaciones completadas
async fn rollback(mut self, completed_ops: Vec<usize>) -> Result<(), DomainError> {
tracing::warn!("Iniciando rollback para transacción: {}", self.name);
let mut rollback_errors = Vec::new();
// Extraer rollbacks para evitar problemas de propiedad
let mut rollbacks = Vec::new();
std::mem::swap(&mut rollbacks, &mut self.rollbacks);
// Ejecutar rollbacks en orden inverso
for i in completed_ops.into_iter().rev() {
if i < rollbacks.len() {
// Tomar propiedad del rollback (obtener una referencia mutable)
if let Some(rb) = rollbacks.get_mut(i) {
// Intercambiar con una función vacía
let rollback = std::mem::replace(rb, Box::new(|| Box::pin(async { Ok(()) })));
if let Err(e) = rollback().await {
tracing::error!("Error en rollback de operación {} en transacción {}: {}",
i, self.name, e);
rollback_errors.push(e);
}
}
}
}
// Si hubo errores en el rollback, reportarlos
if !rollback_errors.is_empty() {
tracing::error!("Errores durante rollback de transacción {}: {} errores",
self.name, rollback_errors.len());
return Err(DomainError::new(
ErrorKind::InternalError,
"Transaction",
format!("Errores durante rollback de transacción '{}': {} errores",
self.name, rollback_errors.len())
));
}
tracing::info!("Rollback de transacción completado: {}", self.name);
Ok(())
}
}
+208
View File
@@ -0,0 +1,208 @@
use std::collections::HashMap;
use std::hash::Hash;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::RwLock;
use crate::common::errors::{DomainError, ErrorKind};
/// Entrada de caché con tiempo de expiración
#[derive(Debug, Clone)]
#[allow(dead_code)]
struct CacheEntry<V> {
value: V,
expiry: Instant,
}
impl<V> CacheEntry<V> {
/// Crea una nueva entrada en la caché
#[allow(dead_code)]
fn new(value: V, ttl: Duration) -> Self {
Self {
value,
expiry: Instant::now() + ttl,
}
}
/// Verifica si la entrada ha expirado
#[allow(dead_code)]
fn is_expired(&self) -> bool {
Instant::now() > self.expiry
}
}
/// Servicio genérico de caché con TTL
#[allow(dead_code)]
pub struct CacheService<K, V> {
cache: Arc<RwLock<HashMap<K, CacheEntry<V>>>>,
ttl: Duration,
max_entries: usize,
}
impl<K, V> CacheService<K, V>
where
K: Hash + Eq + Clone + Send + Sync + 'static + std::fmt::Debug,
V: Clone + Send + Sync + 'static,
{
/// Crea un nuevo servicio de caché
#[allow(dead_code)]
pub fn new(ttl: Duration, max_entries: usize) -> Self {
Self {
cache: Arc::new(RwLock::new(HashMap::new())),
ttl,
max_entries,
}
}
/// Obtiene un valor de la caché o lo inserta si no existe
#[allow(dead_code)]
pub async fn get_or_insert<F, E>(&self, key: K, loader: F) -> Result<V, DomainError>
where
F: FnOnce() -> Result<V, E>,
E: std::error::Error + Send + Sync + 'static,
{
// Intentar leer de la caché primero
{
let cache = self.cache.read().await;
if let Some(entry) = cache.get(&key) {
if !entry.is_expired() {
tracing::debug!("Cache hit for key: {:?}", key);
return Ok(entry.value.clone());
}
}
}
// Cache miss o entrada expirada, obtener valor y actualizar
let value = loader().map_err(|e| {
DomainError::new(
ErrorKind::InternalError,
"Cache",
format!("Failed to load value for cache: {}", e),
)
.with_source(e)
})?;
// Insertar en la caché
{
let mut cache = self.cache.write().await;
// Si alcanzamos el límite, eliminar una entrada aleatoria
if cache.len() >= self.max_entries {
if let Some(expired_key) = cache
.iter()
.find(|(_, v)| v.is_expired())
.map(|(k, _)| k.clone())
{
cache.remove(&expired_key);
} else if let Some(random_key) = cache.keys().next().cloned() {
cache.remove(&random_key);
}
}
cache.insert(key.clone(), CacheEntry::new(value.clone(), self.ttl));
}
tracing::debug!("Cache miss for key: {:?}, value loaded and cached", key);
Ok(value)
}
/// Invalida una entrada específica de la caché
#[allow(dead_code)]
pub async fn invalidate(&self, key: &K) {
let mut cache = self.cache.write().await;
cache.remove(key);
tracing::debug!("Cache entry invalidated for key: {:?}", key);
}
/// Invalida todas las entradas de la caché
#[allow(dead_code)]
pub async fn invalidate_all(&self) {
let mut cache = self.cache.write().await;
cache.clear();
tracing::debug!("Cache fully invalidated");
}
/// Obtiene el número de entradas en la caché
#[allow(dead_code)]
pub async fn len(&self) -> usize {
self.cache.read().await.len()
}
/// Limpia las entradas expiradas de la caché
#[allow(dead_code)]
pub async fn cleanup_expired(&self) -> usize {
let mut cache = self.cache.write().await;
let initial_len = cache.len();
cache.retain(|_, v| !v.is_expired());
let removed = initial_len - cache.len();
if removed > 0 {
tracing::debug!("Removed {} expired cache entries", removed);
}
removed
}
}
/// Caché específica para metadatos de archivos
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct FileMetadata {
pub size: u64,
pub created_at: u64,
pub modified_at: u64,
pub is_dir: bool,
}
/// Gestor de caché para operaciones comunes de almacenamiento
#[allow(dead_code)]
pub struct CacheManager {
/// Caché para metadatos de archivos/carpetas
metadata_cache: CacheService<std::path::PathBuf, FileMetadata>,
/// Caché para verificación de existencia de archivos
existence_cache: CacheService<std::path::PathBuf, bool>,
}
impl CacheManager {
/// Crea un nuevo gestor de caché
#[allow(dead_code)]
pub fn new(metadata_ttl: Duration, existence_ttl: Duration) -> Self {
Self {
metadata_cache: CacheService::new(metadata_ttl, 10000), // Caché para 10,000 elementos
existence_cache: CacheService::new(existence_ttl, 20000), // Caché para 20,000 elementos
}
}
/// Obtiene o carga los metadatos de un archivo/carpeta
#[allow(dead_code)]
pub async fn get_metadata<F>(&self, path: std::path::PathBuf, loader: F) -> Result<FileMetadata, DomainError>
where
F: FnOnce() -> Result<FileMetadata, std::io::Error>,
{
self.metadata_cache.get_or_insert(path, loader).await
}
/// Verifica o determina si un archivo/carpeta existe
#[allow(dead_code)]
pub async fn check_exists<F>(&self, path: std::path::PathBuf, checker: F) -> Result<bool, DomainError>
where
F: FnOnce() -> Result<bool, std::io::Error>,
{
self.existence_cache.get_or_insert(path, checker).await
}
/// Invalida la caché para una ruta específica
#[allow(dead_code)]
pub async fn invalidate_path(&self, path: &std::path::Path) {
self.metadata_cache.invalidate(&path.to_path_buf()).await;
self.existence_cache.invalidate(&path.to_path_buf()).await;
}
/// Limpia todas las entradas expiradas
#[allow(dead_code)]
pub async fn cleanup(&self) -> (usize, usize) {
let metadata_cleaned = self.metadata_cache.cleanup_expired().await;
let existence_cleaned = self.existence_cache.cleanup_expired().await;
(metadata_cleaned, existence_cleaned)
}
}
+185
View File
@@ -0,0 +1,185 @@
use std::time::Duration;
/// Configuración de timeouts para diferentes operaciones
#[derive(Debug, Clone)]
pub struct TimeoutConfig {
/// Timeout para operaciones de archivo (ms)
pub file_operation_ms: u64,
/// Timeout para operaciones de directorio (ms)
pub dir_operation_ms: u64,
/// Timeout para adquisición de locks (ms)
pub lock_acquisition_ms: u64,
/// Timeout para operaciones de red (ms)
#[allow(dead_code)]
pub network_operation_ms: u64,
}
impl Default for TimeoutConfig {
fn default() -> Self {
Self {
file_operation_ms: 10000, // 10 segundos
dir_operation_ms: 30000, // 30 segundos
lock_acquisition_ms: 5000, // 5 segundos
network_operation_ms: 15000, // 15 segundos
}
}
}
impl TimeoutConfig {
/// Obtiene un Duration para operaciones de archivo
pub fn file_timeout(&self) -> Duration {
Duration::from_millis(self.file_operation_ms)
}
/// Obtiene un Duration para operaciones de directorio
pub fn dir_timeout(&self) -> Duration {
Duration::from_millis(self.dir_operation_ms)
}
/// Obtiene un Duration para adquisición de locks
pub fn lock_timeout(&self) -> Duration {
Duration::from_millis(self.lock_acquisition_ms)
}
/// Obtiene un Duration para operaciones de red
#[allow(dead_code)]
pub fn network_timeout(&self) -> Duration {
Duration::from_millis(self.network_operation_ms)
}
}
/// Configuración para manejo de recursos grandes
#[derive(Debug, Clone)]
pub struct ResourceConfig {
/// Umbral en MB para considerar un archivo como grande
pub large_file_threshold_mb: u64,
/// Umbral de entradas para considerar un directorio como grande
#[allow(dead_code)]
pub large_dir_threshold_entries: usize,
/// Tamaño de chunk para procesamiento de archivos grandes (bytes)
pub chunk_size_bytes: usize,
/// Límite de tamaño de archivo para cargar en memoria (MB)
pub max_in_memory_file_size_mb: u64,
}
impl Default for ResourceConfig {
fn default() -> Self {
Self {
large_file_threshold_mb: 100, // 100 MB
large_dir_threshold_entries: 1000, // 1000 entradas
chunk_size_bytes: 1024 * 1024, // 1 MB
max_in_memory_file_size_mb: 50, // 50 MB
}
}
}
impl ResourceConfig {
/// Convierte un tamaño en bytes a MB
pub fn bytes_to_mb(&self, bytes: u64) -> u64 {
bytes / (1024 * 1024)
}
/// Determina si un archivo es considerado grande
pub fn is_large_file(&self, size_bytes: u64) -> bool {
self.bytes_to_mb(size_bytes) >= self.large_file_threshold_mb
}
/// Determina si un archivo es suficientemente grande para procesamiento paralelo
pub fn needs_parallel_processing(&self, size_bytes: u64, config: &ConcurrencyConfig) -> bool {
self.bytes_to_mb(size_bytes) >= config.min_size_for_parallel_chunks_mb
}
/// Determina si un archivo puede cargarse completo en memoria
pub fn can_load_in_memory(&self, size_bytes: u64) -> bool {
self.bytes_to_mb(size_bytes) <= self.max_in_memory_file_size_mb
}
/// Determina si un directorio es considerado grande
#[allow(dead_code)]
pub fn is_large_directory(&self, entry_count: usize) -> bool {
entry_count >= self.large_dir_threshold_entries
}
/// Calcula el número de chunks para procesamiento paralelo
pub fn calculate_optimal_chunks(&self, size_bytes: u64, config: &ConcurrencyConfig) -> usize {
// Si el archivo no es suficientemente grande, retornar 1
if !self.needs_parallel_processing(size_bytes, config) {
return 1;
}
// Calcular el número de chunks basado en el tamaño
let chunk_count = (size_bytes as usize + config.parallel_chunk_size_bytes - 1)
/ config.parallel_chunk_size_bytes;
// Limitar al máximo de chunks en paralelo
chunk_count.min(config.max_parallel_chunks)
}
/// Calcula el tamaño óptimo de cada chunk para procesamiento paralelo
pub fn calculate_chunk_size(&self, file_size: u64, chunk_count: usize) -> usize {
if chunk_count <= 1 {
return file_size as usize;
}
// Distribuir equitativamente el tamaño entre los chunks
((file_size as usize) + chunk_count - 1) / chunk_count
}
}
/// Configuración para operaciones concurrentes
#[derive(Debug, Clone)]
pub struct ConcurrencyConfig {
/// Máximo de tareas de archivo concurrentes
pub max_concurrent_files: usize,
/// Máximo de tareas de directorio concurrentes
#[allow(dead_code)]
pub max_concurrent_dirs: usize,
/// Máximo de operaciones de IO concurrentes
pub max_concurrent_io: usize,
/// Máximo de chunks para procesar en paralelo por archivo
pub max_parallel_chunks: usize,
/// Tamaño mínimo de archivo (MB) para aplicar procesamiento paralelo de chunks
pub min_size_for_parallel_chunks_mb: u64,
/// Tamaño de chunk para procesamiento paralelo (bytes)
pub parallel_chunk_size_bytes: usize,
}
impl Default for ConcurrencyConfig {
fn default() -> Self {
Self {
max_concurrent_files: 10,
max_concurrent_dirs: 5,
max_concurrent_io: 20,
max_parallel_chunks: 8,
min_size_for_parallel_chunks_mb: 200, // 200 MB
parallel_chunk_size_bytes: 8 * 1024 * 1024, // 8 MB
}
}
}
/// Configuración global de la aplicación
#[derive(Debug, Clone)]
pub struct AppConfig {
/// Configuración de timeouts
pub timeouts: TimeoutConfig,
/// Configuración de recursos
pub resources: ResourceConfig,
/// Configuración de concurrencia
pub concurrency: ConcurrencyConfig,
}
impl Default for AppConfig {
fn default() -> Self {
Self {
timeouts: TimeoutConfig::default(),
resources: ResourceConfig::default(),
concurrency: ConcurrencyConfig::default(),
}
}
}
/// Obtenemos una configuración global por defecto
#[allow(dead_code)]
pub fn default_config() -> AppConfig {
AppConfig::default()
}
+190
View File
@@ -0,0 +1,190 @@
use std::path::PathBuf;
use std::sync::Arc;
use async_trait::async_trait;
use crate::domain::services::path_service::PathService;
use crate::infrastructure::repositories::folder_fs_repository::FolderFsRepository;
use crate::infrastructure::repositories::file_fs_repository::FileFsRepository;
use crate::infrastructure::services::file_system_i18n_service::FileSystemI18nService;
use crate::infrastructure::services::id_mapping_service::IdMappingService;
use crate::infrastructure::services::cache_manager::StorageCacheManager;
use crate::application::services::folder_service::FolderService;
use crate::application::services::file_service::FileService;
use crate::application::services::i18n_application_service::I18nApplicationService;
use crate::application::services::storage_mediator::{StorageMediator, FileSystemStorageMediator};
use crate::application::ports::inbound::{FileUseCase, FolderUseCase, UseCaseFactory};
use crate::application::ports::outbound::{FileStoragePort, FolderStoragePort};
use crate::common::errors::DomainError;
use crate::domain::services::i18n_service::I18nService;
/// Fábrica para los diferentes componentes de la aplicación
#[allow(dead_code)]
pub struct AppServiceFactory {
storage_path: PathBuf,
locales_path: PathBuf,
}
impl AppServiceFactory {
/// Crea una nueva fábrica de servicios
#[allow(dead_code)]
pub fn new(storage_path: PathBuf, locales_path: PathBuf) -> Self {
Self {
storage_path,
locales_path,
}
}
/// Inicializa los servicios base del sistema
#[allow(dead_code)]
pub async fn create_core_services(&self) -> Result<CoreServices, DomainError> {
// Path service
let path_service = Arc::new(PathService::new(self.storage_path.clone()));
// Cache manager
// TTL values in milliseconds and max entries for cache
let file_ttl_ms = 60_000; // 1 minute for files
let dir_ttl_ms = 120_000; // 2 minutes for directories
let max_entries = 10_000; // Maximum cache entries
let cache_manager = Arc::new(StorageCacheManager::new(file_ttl_ms, dir_ttl_ms, max_entries));
// Iniciar tarea de limpieza de caché en segundo plano
let cache_manager_clone = cache_manager.clone();
tokio::spawn(async move {
StorageCacheManager::start_cleanup_task(cache_manager_clone).await;
});
// ID mapping service
let id_mapping_path = self.storage_path.join("folder_ids.json");
let id_mapping_service = Arc::new(
IdMappingService::new(id_mapping_path).await?
);
Ok(CoreServices {
path_service,
cache_manager,
id_mapping_service,
})
}
/// Inicializa los servicios de repositorio
#[allow(dead_code)]
pub fn create_repository_services(&self, core: &CoreServices) -> RepositoryServices {
// Storage mediator - create first because it's needed by folder repository
// (temporarily using a placeholder for folder repository, will update later)
let placeholder_folder_repo = Arc::new(FolderFsRepository::new_stub());
let storage_mediator = Arc::new(FileSystemStorageMediator::new(
placeholder_folder_repo.clone(),
core.path_service.clone(),
core.id_mapping_service.clone()
));
// Folder repository
let folder_repository = Arc::new(FolderFsRepository::new(
self.storage_path.clone(),
storage_mediator.clone(),
core.id_mapping_service.clone(),
core.path_service.clone(),
));
// Create a file metadata cache with default configuration
let metadata_cache = Arc::new(
crate::infrastructure::services::file_metadata_cache::FileMetadataCache::default_with_config(
crate::common::config::AppConfig::default()
)
);
// File repository
let file_repository = Arc::new(FileFsRepository::new(
self.storage_path.clone(),
storage_mediator.clone(),
core.id_mapping_service.clone(),
core.path_service.clone(),
metadata_cache,
));
// I18n repository
let i18n_repository = Arc::new(FileSystemI18nService::new(
self.locales_path.clone()
));
RepositoryServices {
folder_repository,
file_repository,
i18n_repository,
storage_mediator,
}
}
/// Inicializa los servicios de aplicación
#[allow(dead_code)]
pub fn create_application_services(&self, repos: &RepositoryServices) -> ApplicationServices {
// Servicios principales
let folder_service = Arc::new(FolderService::new(
repos.folder_repository.clone()
));
let file_service = Arc::new(FileService::new(
repos.file_repository.clone()
));
let i18n_service = Arc::new(I18nApplicationService::new(
repos.i18n_repository.clone()
));
ApplicationServices {
folder_service,
file_service,
i18n_service,
}
}
}
/// Contenedor para servicios base
#[allow(dead_code)]
pub struct CoreServices {
pub path_service: Arc<PathService>,
pub cache_manager: Arc<StorageCacheManager>,
pub id_mapping_service: Arc<IdMappingService>,
}
/// Contenedor para servicios de repositorio
#[allow(dead_code)]
pub struct RepositoryServices {
pub folder_repository: Arc<dyn FolderStoragePort>,
pub file_repository: Arc<dyn FileStoragePort>,
pub i18n_repository: Arc<dyn I18nService>,
pub storage_mediator: Arc<dyn StorageMediator>,
}
/// Contenedor para servicios de aplicación
#[allow(dead_code)]
pub struct ApplicationServices {
pub folder_service: Arc<dyn FolderUseCase>,
pub file_service: Arc<dyn FileUseCase>,
pub i18n_service: Arc<I18nApplicationService>,
}
/// Fábrica de casos de uso para la inyección de dependencias
#[allow(dead_code)]
pub struct AppUseCaseFactory {
services: ApplicationServices,
}
impl AppUseCaseFactory {
#[allow(dead_code)]
pub fn new(services: ApplicationServices) -> Self {
Self { services }
}
}
#[async_trait]
impl UseCaseFactory for AppUseCaseFactory {
fn create_file_use_case(&self) -> Arc<dyn FileUseCase> {
self.services.file_service.clone()
}
fn create_folder_use_case(&self) -> Arc<dyn FolderUseCase> {
self.services.folder_service.clone()
}
}
+211
View File
@@ -0,0 +1,211 @@
use std::fmt::{Display, Formatter, Result as FmtResult};
use std::error::Error as StdError;
use thiserror::Error;
/// Tipos de errores comunes en toda la aplicación
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ErrorKind {
/// Entidad no encontrada
NotFound,
/// Entidad ya existe
AlreadyExists,
/// Entrada inválida o validación fallida
InvalidInput,
/// Error de acceso o permisos
AccessDenied,
/// Tiempo de espera agotado
Timeout,
/// Error interno del sistema
InternalError,
}
impl Display for ErrorKind {
fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult {
match self {
ErrorKind::NotFound => write!(f, "Not Found"),
ErrorKind::AlreadyExists => write!(f, "Already Exists"),
ErrorKind::InvalidInput => write!(f, "Invalid Input"),
ErrorKind::AccessDenied => write!(f, "Access Denied"),
ErrorKind::Timeout => write!(f, "Timeout"),
ErrorKind::InternalError => write!(f, "Internal Error"),
}
}
}
/// Error base de dominio que proporciona contexto detallado
#[derive(Error, Debug)]
#[error("{kind}: {message}")]
pub struct DomainError {
/// Tipo de error
pub kind: ErrorKind,
/// Tipo de entidad afectada (ej: "File", "Folder")
pub entity_type: &'static str,
/// Identificador de la entidad si está disponible
pub entity_id: Option<String>,
/// Mensaje descriptivo del error
pub message: String,
/// Error fuente (opcional)
#[source]
pub source: Option<Box<dyn StdError + Send + Sync>>,
}
impl DomainError {
/// Crea un nuevo error de dominio
pub fn new<S: Into<String>>(
kind: ErrorKind,
entity_type: &'static str,
message: S,
) -> Self {
Self {
kind,
entity_type,
entity_id: None,
message: message.into(),
source: None,
}
}
/// Crea un error de entidad no encontrada
pub fn not_found<S: Into<String>>(entity_type: &'static str, entity_id: S) -> Self {
let id = entity_id.into();
Self {
kind: ErrorKind::NotFound,
entity_type,
entity_id: Some(id.clone()),
message: format!("{} not found: {}", entity_type, id),
source: None,
}
}
/// Crea un error de entidad ya existente
pub fn already_exists<S: Into<String>>(entity_type: &'static str, entity_id: S) -> Self {
let id = entity_id.into();
Self {
kind: ErrorKind::AlreadyExists,
entity_type,
entity_id: Some(id.clone()),
message: format!("{} already exists: {}", entity_type, id),
source: None,
}
}
/// Crea un error de tiempo agotado
pub fn timeout<S: Into<String>>(entity_type: &'static str, message: S) -> Self {
Self {
kind: ErrorKind::Timeout,
entity_type,
entity_id: None,
message: message.into(),
source: None,
}
}
/// Crea un error interno
pub fn internal_error<S: Into<String>>(entity_type: &'static str, message: S) -> Self {
Self {
kind: ErrorKind::InternalError,
entity_type,
entity_id: None,
message: message.into(),
source: None,
}
}
/// Crea un error de acceso denegado
pub fn access_denied<S: Into<String>>(entity_type: &'static str, message: S) -> Self {
Self {
kind: ErrorKind::AccessDenied,
entity_type,
entity_id: None,
message: message.into(),
source: None,
}
}
/// Crea un error de validación
pub fn validation_error<S: Into<String>>(entity_type: &'static str, message: S) -> Self {
Self {
kind: ErrorKind::InvalidInput,
entity_type,
entity_id: None,
message: message.into(),
source: None,
}
}
/// Establece el ID de la entidad
#[allow(dead_code)]
pub fn with_id<S: Into<String>>(mut self, entity_id: S) -> Self {
self.entity_id = Some(entity_id.into());
self
}
/// Establece el error fuente
pub fn with_source<E: StdError + Send + Sync + 'static>(mut self, source: E) -> Self {
self.source = Some(Box::new(source));
self
}
}
/// Trait para añadir contexto a los errores
pub trait ErrorContext<T, E> {
fn with_context<C, F>(self, context: F) -> Result<T, DomainError>
where
C: Into<String>,
F: FnOnce() -> C;
#[allow(dead_code)]
fn with_error_kind(self, kind: ErrorKind, entity_type: &'static str) -> Result<T, DomainError>;
}
impl<T, E: StdError + Send + Sync + 'static> ErrorContext<T, E> for Result<T, E> {
fn with_context<C, F>(self, context: F) -> Result<T, DomainError>
where
C: Into<String>,
F: FnOnce() -> C,
{
self.map_err(|e| {
DomainError {
kind: ErrorKind::InternalError,
entity_type: "Unknown",
entity_id: None,
message: context().into(),
source: Some(Box::new(e)),
}
})
}
fn with_error_kind(self, kind: ErrorKind, entity_type: &'static str) -> Result<T, DomainError> {
self.map_err(|e| {
DomainError {
kind,
entity_type,
entity_id: None,
message: format!("{}", e),
source: Some(Box::new(e)),
}
})
}
}
/// Macro para convertir errores específicos a DomainError
#[macro_export]
macro_rules! impl_from_error {
($error_type:ty, $entity_type:expr) => {
impl From<$error_type> for DomainError {
fn from(err: $error_type) -> Self {
DomainError {
kind: ErrorKind::InternalError,
entity_type: $entity_type,
entity_id: None,
message: format!("{}", err),
source: Some(Box::new(err)),
}
}
}
};
}
// Implementación para errores estándar comunes
impl_from_error!(std::io::Error, "IO");
impl_from_error!(serde_json::Error, "Serialization");
+4
View File
@@ -0,0 +1,4 @@
pub mod errors;
pub mod config;
pub mod cache;
pub mod di;
+285 -20
View File
@@ -1,67 +1,332 @@
use std::path::PathBuf;
use serde::{Serialize, Deserialize};
use crate::domain::services::path_service::StoragePath;
/// Error en la creación o manipulación de entidades de archivo
#[derive(Debug, thiserror::Error)]
pub enum FileError {
#[error("Nombre de archivo inválido: {0}")]
InvalidFileName(String),
#[error("Error en la validación: {0}")]
#[allow(dead_code)]
ValidationError(String),
}
/// Tipo de resultado para operaciones con entidades de archivo
pub type FileResult<T> = Result<T, FileError>;
/// Represents a file entity in the domain
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct File {
/// Unique identifier for the file
pub id: String,
id: String,
/// Name of the file
pub name: String,
name: String,
/// Path to the file (relative to user's root)
pub path: PathBuf,
/// Path to the file in the domain model
#[serde(skip_serializing, skip_deserializing)]
storage_path: StoragePath,
/// String representation of the path (for serialization compatibility)
#[serde(rename = "path")]
path_string: String,
/// Size of the file in bytes
pub size: u64,
size: u64,
/// MIME type of the file
pub mime_type: String,
mime_type: String,
/// Parent folder ID
pub folder_id: Option<String>,
folder_id: Option<String>,
/// Creation timestamp
pub created_at: u64,
created_at: u64,
/// Last modification timestamp
pub modified_at: u64,
modified_at: u64,
}
// Ya no necesitamos este módulo, ahora usamos un String directamente
impl File {
/// Creates a new file
/// Crea un nuevo archivo con validación
pub fn new(
id: String,
name: String,
path: PathBuf,
storage_path: StoragePath,
size: u64,
mime_type: String,
folder_id: Option<String>,
) -> Self {
) -> FileResult<Self> {
// Validar nombre de archivo
if name.is_empty() || name.contains('/') || name.contains('\\') {
return Err(FileError::InvalidFileName(name));
}
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.unwrap_or_default()
.as_secs();
Self {
// Almacenamos el string de la ruta para compatibilidad con serialización
let path_string = storage_path.to_string();
Ok(Self {
id,
name,
path,
storage_path,
path_string,
size,
mime_type,
folder_id,
created_at: now,
modified_at: now,
})
}
/// Crea un archivo con timestamps específicos (para reconstrucción)
pub fn with_timestamps(
id: String,
name: String,
storage_path: StoragePath,
size: u64,
mime_type: String,
folder_id: Option<String>,
created_at: u64,
modified_at: u64,
) -> FileResult<Self> {
// Validar nombre de archivo
if name.is_empty() || name.contains('/') || name.contains('\\') {
return Err(FileError::InvalidFileName(name));
}
// Almacenamos el string de la ruta para compatibilidad con serialización
let path_string = storage_path.to_string();
Ok(Self {
id,
name,
storage_path,
path_string,
size,
mime_type,
folder_id,
created_at,
modified_at,
})
}
// Getters
pub fn id(&self) -> &str {
&self.id
}
pub fn name(&self) -> &str {
&self.name
}
pub fn storage_path(&self) -> &StoragePath {
&self.storage_path
}
pub fn path_string(&self) -> &str {
&self.path_string
}
pub fn size(&self) -> u64 {
self.size
}
pub fn mime_type(&self) -> &str {
&self.mime_type
}
pub fn folder_id(&self) -> Option<&str> {
self.folder_id.as_deref()
}
pub fn created_at(&self) -> u64 {
self.created_at
}
pub fn modified_at(&self) -> u64 {
self.modified_at
}
/// Crea una nueva instancia de File desde un DTO
/// Esta función es principalmente para conversiones en los batch handlers
pub fn from_dto(
id: String,
name: String,
path: String,
size: u64,
mime_type: String,
folder_id: Option<String>,
created_at: u64,
modified_at: u64,
) -> Self {
// Crear storage_path desde el string
let storage_path = StoragePath::from_string(&path);
// Crear directamente sin validación para evitar errores en conversiones DTO
Self {
id,
name,
storage_path,
path_string: path,
size,
mime_type,
folder_id,
created_at,
modified_at,
}
}
/// Updates file modification time
// Métodos para crear nuevas versiones del archivo (inmutable)
/// Crea una nueva versión del archivo con nombre actualizado
#[allow(dead_code)]
pub fn touch(&mut self) {
self.modified_at = std::time::SystemTime::now()
pub fn with_name(&self, new_name: String) -> FileResult<Self> {
// Validar nombre de archivo
if new_name.is_empty() || new_name.contains('/') || new_name.contains('\\') {
return Err(FileError::InvalidFileName(new_name));
}
// Actualizar ruta basada en el nombre
let parent_path = self.storage_path.parent();
let new_storage_path = match parent_path {
Some(parent) => parent.join(&new_name),
None => StoragePath::from_string(&new_name),
};
// Actualizar representación en string
let new_path_string = new_storage_path.to_string();
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.unwrap_or_default()
.as_secs();
Ok(Self {
id: self.id.clone(),
name: new_name,
storage_path: new_storage_path,
path_string: new_path_string,
size: self.size,
mime_type: self.mime_type.clone(),
folder_id: self.folder_id.clone(),
created_at: self.created_at,
modified_at: now,
})
}
/// Crea una nueva versión del archivo con carpeta actualizada
pub fn with_folder(&self, folder_id: Option<String>, folder_path: Option<StoragePath>) -> FileResult<Self> {
// Necesitamos una ruta de carpeta para actualizar la ruta del archivo
let new_storage_path = match folder_path {
Some(path) => path.join(&self.name),
None => StoragePath::from_string(&self.name), // Raíz
};
// Actualizar representación en string
let new_path_string = new_storage_path.to_string();
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs();
Ok(Self {
id: self.id.clone(),
name: self.name.clone(),
storage_path: new_storage_path,
path_string: new_path_string,
size: self.size,
mime_type: self.mime_type.clone(),
folder_id,
created_at: self.created_at,
modified_at: now,
})
}
/// Crea una nueva versión del archivo con tamaño actualizado
#[allow(dead_code)]
pub fn with_size(&self, new_size: u64) -> Self {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs();
Self {
id: self.id.clone(),
name: self.name.clone(),
storage_path: self.storage_path.clone(),
path_string: self.path_string.clone(),
size: new_size,
mime_type: self.mime_type.clone(),
folder_id: self.folder_id.clone(),
created_at: self.created_at,
modified_at: now,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_file_creation_with_valid_name() {
let storage_path = StoragePath::from_string("/test/file.txt");
let file = File::new(
"123".to_string(),
"file.txt".to_string(),
storage_path,
100,
"text/plain".to_string(),
None,
);
assert!(file.is_ok());
}
#[test]
fn test_file_creation_with_invalid_name() {
let storage_path = StoragePath::from_string("/test/invalid/file.txt");
let file = File::new(
"123".to_string(),
"file/with/slash.txt".to_string(), // Nombre inválido
storage_path,
100,
"text/plain".to_string(),
None,
);
assert!(file.is_err());
match file {
Err(FileError::InvalidFileName(_)) => (),
_ => panic!("Expected InvalidFileName error"),
}
}
#[test]
fn test_file_with_name() {
let storage_path = StoragePath::from_string("/test/file.txt");
let file = File::new(
"123".to_string(),
"file.txt".to_string(),
storage_path,
100,
"text/plain".to_string(),
None,
).unwrap();
let renamed = file.with_name("newname.txt".to_string());
assert!(renamed.is_ok());
let renamed = renamed.unwrap();
assert_eq!(renamed.name(), "newname.txt");
assert_eq!(renamed.id(), "123"); // El ID no cambia
}
}
+257 -21
View File
@@ -1,57 +1,293 @@
use std::path::PathBuf;
use serde::{Serialize, Deserialize};
use crate::domain::services::path_service::StoragePath;
/// Error en la creación o manipulación de entidades de carpeta
#[derive(Debug, thiserror::Error)]
pub enum FolderError {
#[error("Nombre de carpeta inválido: {0}")]
InvalidFolderName(String),
#[error("Error en la validación: {0}")]
#[allow(dead_code)]
ValidationError(String),
}
/// Tipo de resultado para operaciones con entidades de carpeta
pub type FolderResult<T> = Result<T, FolderError>;
/// Represents a folder entity in the domain
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct Folder {
/// Unique identifier for the folder
pub id: String,
id: String,
/// Name of the folder
pub name: String,
name: String,
/// Path to the folder (relative to user's root)
pub path: PathBuf,
/// Path to the folder in the domain model
#[serde(skip_serializing, skip_deserializing)]
storage_path: StoragePath,
/// String representation of the path (for serialization compatibility)
#[serde(rename = "path")]
path_string: String,
/// Parent folder ID (None if it's a root folder)
pub parent_id: Option<String>,
parent_id: Option<String>,
/// Creation timestamp
pub created_at: u64,
created_at: u64,
/// Last modification timestamp
pub modified_at: u64,
modified_at: u64,
}
// Ya no necesitamos este módulo, ahora usamos un String directamente
impl Folder {
/// Creates a new folder
pub fn new(id: String, name: String, path: PathBuf, parent_id: Option<String>) -> Self {
/// Creates a new folder with validation
pub fn new(
id: String,
name: String,
storage_path: StoragePath,
parent_id: Option<String>,
) -> FolderResult<Self> {
// Validar nombre de carpeta
if name.is_empty() || name.contains('/') || name.contains('\\') {
return Err(FolderError::InvalidFolderName(name));
}
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.unwrap_or_default()
.as_secs();
Self {
// Almacenamos el string de la ruta para compatibilidad con serialización
let path_string = storage_path.to_string();
Ok(Self {
id,
name,
path,
storage_path,
path_string,
parent_id,
created_at: now,
modified_at: now,
})
}
/// Creates a folder with specific timestamps (for reconstruction)
pub fn with_timestamps(
id: String,
name: String,
storage_path: StoragePath,
parent_id: Option<String>,
created_at: u64,
modified_at: u64,
) -> FolderResult<Self> {
// Validar nombre de carpeta
if name.is_empty() || name.contains('/') || name.contains('\\') {
return Err(FolderError::InvalidFolderName(name));
}
// Almacenamos el string de la ruta para compatibilidad con serialización
let path_string = storage_path.to_string();
Ok(Self {
id,
name,
storage_path,
path_string,
parent_id,
created_at,
modified_at,
})
}
// Getters
pub fn id(&self) -> &str {
&self.id
}
pub fn name(&self) -> &str {
&self.name
}
pub fn storage_path(&self) -> &StoragePath {
&self.storage_path
}
pub fn path_string(&self) -> &str {
&self.path_string
}
pub fn parent_id(&self) -> Option<&str> {
self.parent_id.as_deref()
}
pub fn created_at(&self) -> u64 {
self.created_at
}
pub fn modified_at(&self) -> u64 {
self.modified_at
}
/// Crea una nueva instancia de Folder desde un DTO
/// Esta función es principalmente para conversiones en los batch handlers
pub fn from_dto(
id: String,
name: String,
path: String,
parent_id: Option<String>,
created_at: u64,
modified_at: u64,
) -> Self {
// Crear storage_path desde el string
let storage_path = StoragePath::from_string(&path);
// Crear directamente sin validación para evitar errores en conversiones DTO
Self {
id,
name,
storage_path,
path_string: path,
parent_id,
created_at,
modified_at,
}
}
/// Returns the absolute path of the folder
#[allow(dead_code)]
pub fn get_absolute_path(&self, root_path: &PathBuf) -> PathBuf {
root_path.join(&self.path)
// Métodos para crear nuevas versiones de la carpeta (inmutable)
/// Creates a new version of the folder with updated name
pub fn with_name(&self, new_name: String) -> FolderResult<Self> {
// Validar nombre de carpeta
if new_name.is_empty() || new_name.contains('/') || new_name.contains('\\') {
return Err(FolderError::InvalidFolderName(new_name));
}
// Actualizar ruta basada en el nombre
let parent_path = self.storage_path.parent();
let new_storage_path = match parent_path {
Some(parent) => parent.join(&new_name),
None => StoragePath::from_string(&new_name),
};
// Actualizar representación en string
let new_path_string = new_storage_path.to_string();
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs();
Ok(Self {
id: self.id.clone(),
name: new_name,
storage_path: new_storage_path,
path_string: new_path_string,
parent_id: self.parent_id.clone(),
created_at: self.created_at,
modified_at: now,
})
}
/// Updates folder modification time
pub fn touch(&mut self) {
self.modified_at = std::time::SystemTime::now()
/// Creates a new version of the folder with updated parent
pub fn with_parent(&self, parent_id: Option<String>, parent_path: Option<StoragePath>) -> FolderResult<Self> {
// Necesitamos una ruta de carpeta para actualizar la ruta
let new_storage_path = match parent_path {
Some(path) => path.join(&self.name),
None => StoragePath::from_string(&self.name), // Raíz
};
// Actualizar representación en string
let new_path_string = new_storage_path.to_string();
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.unwrap_or_default()
.as_secs();
Ok(Self {
id: self.id.clone(),
name: self.name.clone(),
storage_path: new_storage_path,
path_string: new_path_string,
parent_id,
created_at: self.created_at,
modified_at: now,
})
}
/// Returns an absolute path for this folder
#[allow(dead_code)]
pub fn get_absolute_path<P: AsRef<std::path::Path>>(&self, root_path: P) -> std::path::PathBuf {
let mut result = std::path::PathBuf::from(root_path.as_ref());
// Skip leading '/' from path_string to avoid creating absolute path incorrectly
let relative_path = if self.path_string.starts_with('/') {
&self.path_string[1..]
} else {
&self.path_string
};
if !relative_path.is_empty() {
result.push(relative_path);
}
result
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_folder_creation_with_valid_name() {
let storage_path = StoragePath::from_string("/test/folder");
let folder = Folder::new(
"123".to_string(),
"my_folder".to_string(),
storage_path,
None,
);
assert!(folder.is_ok());
}
#[test]
fn test_folder_creation_with_invalid_name() {
let storage_path = StoragePath::from_string("/test/invalid/folder");
let folder = Folder::new(
"123".to_string(),
"folder/with/slash".to_string(), // Nombre inválido
storage_path,
None,
);
assert!(folder.is_err());
match folder {
Err(FolderError::InvalidFolderName(_)) => (),
_ => panic!("Expected InvalidFolderName error"),
}
}
#[test]
fn test_folder_with_name() {
let storage_path = StoragePath::from_string("/test/folder");
let folder = Folder::new(
"123".to_string(),
"old_name".to_string(),
storage_path,
None,
).unwrap();
let renamed = folder.with_name("new_name".to_string());
assert!(renamed.is_ok());
let renamed = renamed.unwrap();
assert_eq!(renamed.name(), "new_name");
assert_eq!(renamed.id(), "123"); // El ID no cambia
}
}
+22 -8
View File
@@ -1,6 +1,8 @@
use std::path::PathBuf;
use async_trait::async_trait;
use crate::domain::entities::file::File;
use crate::domain::services::path_service::StoragePath;
use futures::Stream;
use bytes::Bytes;
/// Error types for file repository operations
#[derive(Debug, thiserror::Error)]
@@ -18,6 +20,12 @@ pub enum FileRepositoryError {
#[error("IO Error: {0}")]
IoError(#[from] std::io::Error),
#[error("Mapping error: {0}")]
MappingError(String),
#[error("Timeout error: {0}")]
Timeout(String),
#[error("Other error: {0}")]
Other(String),
}
@@ -26,11 +34,10 @@ pub enum FileRepositoryError {
pub type FileRepositoryResult<T> = Result<T, FileRepositoryError>;
/// Repository interface for file operations (primary port)
/// Esta interfaz define las operaciones de negocio relacionadas con archivos
/// sin exponer detalles de implementación como rutas o sistemas de archivos
#[async_trait]
pub trait FileRepository: Send + Sync + 'static {
/// Gets a folder by its ID - helper method for file repository to work with folders
#[allow(dead_code)]
async fn get_folder_by_id(&self, id: &str) -> FileRepositoryResult<crate::domain::entities::folder::Folder>;
/// Saves a file from bytes
async fn save_file_from_bytes(
&self,
@@ -60,13 +67,20 @@ pub trait FileRepository: Send + Sync + 'static {
/// Deletes a file
async fn delete_file(&self, id: &str) -> FileRepositoryResult<()>;
/// Deletes a file and its entry from the map
/// Deletes a file and its entry from mapping systems
#[allow(dead_code)]
async fn delete_file_entry(&self, id: &str) -> FileRepositoryResult<()>;
/// Gets file content as bytes
/// Gets file content as bytes - use only for small files
async fn get_file_content(&self, id: &str) -> FileRepositoryResult<Vec<u8>>;
/// Checks if a file exists at the given path
async fn file_exists(&self, path: &PathBuf) -> FileRepositoryResult<bool>;
/// Gets file content as a stream - better for large files
#[allow(clippy::type_complexity)]
async fn get_file_stream(&self, id: &str) -> FileRepositoryResult<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>>;
/// Moves a file to a different folder
async fn move_file(&self, id: &str, target_folder_id: Option<String>) -> FileRepositoryResult<File>;
/// Gets the storage path for a file
async fn get_file_path(&self, id: &str) -> FileRepositoryResult<StoragePath>;
}
+38 -5
View File
@@ -1,6 +1,6 @@
use std::path::PathBuf;
use async_trait::async_trait;
use crate::domain::entities::folder::Folder;
use crate::domain::services::path_service::StoragePath;
/// Error types for folder repository operations
#[derive(Debug, thiserror::Error)]
@@ -18,6 +18,12 @@ pub enum FolderRepositoryError {
#[error("IO Error: {0}")]
IoError(#[from] std::io::Error),
#[error("Mapping error: {0}")]
MappingError(String),
#[error("Validation error: {0}")]
ValidationError(String),
#[error("Other error: {0}")]
Other(String),
}
@@ -29,17 +35,31 @@ pub type FolderRepositoryResult<T> = Result<T, FolderRepositoryError>;
#[async_trait]
pub trait FolderRepository: Send + Sync + 'static {
/// Creates a new folder
async fn create_folder(&self, name: String, parent_path: Option<PathBuf>) -> FolderRepositoryResult<Folder>;
async fn create_folder(&self, name: String, parent_id: Option<String>) -> FolderRepositoryResult<Folder>;
/// Gets a folder by its ID
async fn get_folder_by_id(&self, id: &str) -> FolderRepositoryResult<Folder>;
/// Gets a folder by its path
async fn get_folder_by_path(&self, path: &PathBuf) -> FolderRepositoryResult<Folder>;
async fn get_folder_by_storage_path(&self, storage_path: &StoragePath) -> FolderRepositoryResult<Folder>;
/// Lists folders in a parent folder
/// Lists all folders in a parent folder (use with caution for large directories)
async fn list_folders(&self, parent_id: Option<&str>) -> FolderRepositoryResult<Vec<Folder>>;
/// Lists folders in a parent folder with pagination support
///
/// * `parent_id` - Optional parent folder ID
/// * `offset` - Number of folders to skip
/// * `limit` - Maximum number of folders to return
/// * `include_total` - If true, returns the total count of folders as well
async fn list_folders_paginated(
&self,
parent_id: Option<&str>,
offset: usize,
limit: usize,
include_total: bool
) -> FolderRepositoryResult<(Vec<Folder>, Option<usize>)>;
/// Renames a folder
async fn rename_folder(&self, id: &str, new_name: String) -> FolderRepositoryResult<Folder>;
@@ -50,5 +70,18 @@ pub trait FolderRepository: Send + Sync + 'static {
async fn delete_folder(&self, id: &str) -> FolderRepositoryResult<()>;
/// Checks if a folder exists at the given path
async fn folder_exists(&self, path: &PathBuf) -> FolderRepositoryResult<bool>;
async fn folder_exists_at_storage_path(&self, storage_path: &StoragePath) -> FolderRepositoryResult<bool>;
/// Gets the storage path for a folder
async fn get_folder_storage_path(&self, id: &str) -> FolderRepositoryResult<StoragePath>;
/// Legacy method - checks if a folder exists at the given PathBuf path
#[deprecated(note = "Use folder_exists_at_storage_path instead")]
#[allow(dead_code)]
async fn folder_exists(&self, path: &std::path::PathBuf) -> FolderRepositoryResult<bool>;
/// Legacy method - gets a folder by its PathBuf path
#[deprecated(note = "Use get_folder_by_storage_path instead")]
#[allow(dead_code)]
async fn get_folder_by_path(&self, path: &std::path::PathBuf) -> FolderRepositoryResult<Folder>;
}
+1
View File
@@ -1 +1,2 @@
pub mod i18n_service;
pub mod path_service;
+381
View File
@@ -0,0 +1,381 @@
/// Abstracto servicio de dominio para rutas, sin dependencias de sistema de archivos
/// Representa una ruta de almacenamiento en el dominio
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct StoragePath {
segments: Vec<String>,
}
impl StoragePath {
/// Crea una nueva ruta de almacenamiento
#[allow(dead_code)]
pub fn new(segments: Vec<String>) -> Self {
Self { segments }
}
/// Crea una ruta vacía (raíz)
pub fn root() -> Self {
Self { segments: Vec::new() }
}
/// Crea una ruta a partir de una cadena con segmentos separados por /
pub fn from_string(path: &str) -> Self {
let segments = path
.split('/')
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
.collect();
Self { segments }
}
/// Crea una ruta a partir de un PathBuf
pub fn from(path_buf: PathBuf) -> Self {
let segments = path_buf
.components()
.filter_map(|c| match c {
std::path::Component::Normal(os_str) => Some(os_str.to_string_lossy().to_string()),
_ => None,
})
.collect();
Self { segments }
}
/// Añade un segmento a la ruta
pub fn join(&self, segment: &str) -> Self {
let mut new_segments = self.segments.clone();
new_segments.push(segment.to_string());
Self { segments: new_segments }
}
/// Obtiene el nombre del archivo (último segmento)
pub fn file_name(&self) -> Option<String> {
self.segments.last().cloned()
}
/// Obtiene la ruta del directorio padre
pub fn parent(&self) -> Option<Self> {
if self.segments.is_empty() {
None
} else {
let parent_segments = self.segments[..self.segments.len() - 1].to_vec();
Some(Self { segments: parent_segments })
}
}
/// Verifica si la ruta está vacía (es la raíz)
pub fn is_empty(&self) -> bool {
self.segments.is_empty()
}
/// Convierte la ruta a una cadena con formato "/segment1/segment2/..."
pub fn to_string(&self) -> String {
if self.segments.is_empty() {
"/".to_string()
} else {
format!("/{}", self.segments.join("/"))
}
}
/// Obtiene los segmentos de la ruta
pub fn segments(&self) -> &[String] {
&self.segments
}
}
use std::path::{Path, PathBuf};
use async_trait::async_trait;
use tokio::fs;
use crate::common::errors::{DomainError, ErrorKind};
use crate::application::ports::outbound::StoragePort;
use crate::application::services::storage_mediator::{StorageMediator, StorageMediatorResult, StorageMediatorError};
use crate::domain::entities::folder::Folder;
/// Servicio de dominio para manejar operaciones con rutas de almacenamiento
pub struct PathService {
root_path: PathBuf, // Necesario para la implementación
}
impl PathService {
/// Crea un nuevo servicio de rutas con una raíz específica
pub fn new(root_path: PathBuf) -> Self {
Self { root_path }
}
/// Convierte una ruta del dominio a una ruta física absoluta
pub fn resolve_path(&self, storage_path: &StoragePath) -> PathBuf {
let mut path = self.root_path.clone();
for segment in storage_path.segments() {
path.push(segment);
}
path
}
/// Convierte una ruta física a una ruta de dominio
#[allow(dead_code)]
pub fn to_storage_path(&self, physical_path: &Path) -> Option<StoragePath> {
physical_path.strip_prefix(&self.root_path).ok().map(|rel_path| {
let segments = rel_path
.components()
.filter_map(|c| match c {
std::path::Component::Normal(os_str) => Some(os_str.to_string_lossy().to_string()),
_ => None,
})
.collect();
StoragePath { segments }
})
}
/// Crea una ruta de archivo dentro de una carpeta
#[allow(dead_code)]
pub fn create_file_path(&self, folder_path: &StoragePath, file_name: &str) -> StoragePath {
folder_path.join(file_name)
}
/// Verifica si una ruta es directamente hija de otra
#[allow(dead_code)]
pub fn is_direct_child(&self, parent_path: &StoragePath, potential_child: &StoragePath) -> bool {
if let Some(child_parent) = potential_child.parent() {
&child_parent == parent_path
} else {
parent_path.is_empty()
}
}
/// Verifica si una ruta está en la raíz
#[allow(dead_code)]
pub fn is_in_root(&self, path: &StoragePath) -> bool {
path.parent().map_or(true, |p| p.is_empty())
}
/// Gets the root path used by this service
#[allow(dead_code)]
pub fn get_root_path(&self) -> &Path {
&self.root_path
}
/// Valida una ruta para asegurar que no contiene componentes peligrosos
pub fn validate_path(&self, path: &StoragePath) -> Result<(), DomainError> {
// Verificar que no haya segmentos vacíos
if path.segments().iter().any(|s| s.is_empty()) {
return Err(DomainError::new(
ErrorKind::InvalidInput,
"Path",
format!("Path contains empty segments: {}", path.to_string())
));
}
// Verificar que no haya caracteres peligrosos
let dangerous_chars = ['\\', ':', '*', '?', '"', '<', '>', '|'];
for segment in path.segments() {
if segment.contains(&dangerous_chars[..]) {
return Err(DomainError::new(
ErrorKind::InvalidInput,
"Path",
format!("Path contains dangerous characters: {}", segment)
));
}
// Verificar que no empiece con . (oculto en Unix)
if segment.starts_with('.') && segment != ".well-known" {
return Err(DomainError::new(
ErrorKind::InvalidInput,
"Path",
format!("Path segments cannot start with dot: {}", segment)
));
}
}
Ok(())
}
}
#[async_trait]
impl StoragePort for PathService {
fn resolve_path(&self, storage_path: &StoragePath) -> PathBuf {
let mut path = self.root_path.clone();
for segment in storage_path.segments() {
path.push(segment);
}
path
}
async fn ensure_directory(&self, storage_path: &StoragePath) -> Result<(), DomainError> {
// Primero validar la ruta
self.validate_path(storage_path)?;
// Resolver a ruta física
let physical_path = self.resolve_path(storage_path);
// Crear directorios si no existen
if !physical_path.exists() {
fs::create_dir_all(&physical_path).await
.map_err(|e| DomainError::new(
ErrorKind::AccessDenied,
"Storage",
format!("Failed to create directory: {}", physical_path.display())
).with_source(e))?;
tracing::debug!("Created directory: {}", physical_path.display());
} else if !physical_path.is_dir() {
return Err(DomainError::new(
ErrorKind::InvalidInput,
"Storage",
format!("Path exists but is not a directory: {}", physical_path.display())
));
}
Ok(())
}
async fn file_exists(&self, storage_path: &StoragePath) -> Result<bool, DomainError> {
let physical_path = self.resolve_path(storage_path);
let exists = physical_path.exists() && physical_path.is_file();
Ok(exists)
}
async fn directory_exists(&self, storage_path: &StoragePath) -> Result<bool, DomainError> {
let physical_path = self.resolve_path(storage_path);
let exists = physical_path.exists() && physical_path.is_dir();
Ok(exists)
}
}
#[async_trait]
impl StorageMediator for PathService {
async fn get_folder_path(&self, folder_id: &str) -> StorageMediatorResult<PathBuf> {
// This is a simplified implementation since PathService doesn't have direct
// access to folder repository. It's typically used through a proxy.
Err(StorageMediatorError::NotFound(format!("Folder with ID {} not found", folder_id)))
}
async fn get_folder_storage_path(&self, folder_id: &str) -> StorageMediatorResult<StoragePath> {
// Simplified implementation - should be overridden by actual implementations
Err(StorageMediatorError::NotFound(format!("Folder with ID {} not found", folder_id)))
}
async fn get_folder(&self, folder_id: &str) -> StorageMediatorResult<Folder> {
// Simplified implementation - should be overridden by actual implementations
Err(StorageMediatorError::NotFound(format!("Folder with ID {} not found", folder_id)))
}
async fn file_exists_at_path(&self, path: &Path) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_path(&StoragePath::from_string(&path.to_string_lossy()));
Ok(abs_path.exists() && abs_path.is_file())
}
async fn file_exists_at_storage_path(&self, storage_path: &StoragePath) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_path(storage_path);
Ok(abs_path.exists() && abs_path.is_file())
}
async fn folder_exists_at_path(&self, path: &Path) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_path(&StoragePath::from_string(&path.to_string_lossy()));
Ok(abs_path.exists() && abs_path.is_dir())
}
async fn folder_exists_at_storage_path(&self, storage_path: &StoragePath) -> StorageMediatorResult<bool> {
let abs_path = self.resolve_path(storage_path);
Ok(abs_path.exists() && abs_path.is_dir())
}
fn resolve_path(&self, relative_path: &Path) -> PathBuf {
// Convert path to storage path then resolve
let path_str = relative_path.to_string_lossy().to_string();
let storage_path = StoragePath::from_string(&path_str);
self.resolve_path(&storage_path)
}
fn resolve_storage_path(&self, storage_path: &StoragePath) -> PathBuf {
self.resolve_path(storage_path)
}
async fn ensure_directory(&self, path: &Path) -> StorageMediatorResult<()> {
let abs_path = self.resolve_path(&StoragePath::from_string(&path.to_string_lossy()));
if !abs_path.exists() {
fs::create_dir_all(&abs_path).await
.map_err(|e| StorageMediatorError::AccessError(format!("Failed to create directory: {}", e)))?;
} else if !abs_path.is_dir() {
return Err(StorageMediatorError::InvalidPath(
format!("Path exists but is not a directory: {}", abs_path.display())
));
}
Ok(())
}
async fn ensure_storage_directory(&self, storage_path: &StoragePath) -> StorageMediatorResult<()> {
let abs_path = self.resolve_path(storage_path);
if !abs_path.exists() {
fs::create_dir_all(&abs_path).await
.map_err(|e| StorageMediatorError::AccessError(format!("Failed to create directory: {}", e)))?;
} else if !abs_path.is_dir() {
return Err(StorageMediatorError::InvalidPath(
format!("Path exists but is not a directory: {}", abs_path.display())
));
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_resolve_path() {
let service = PathService::new(PathBuf::from("/storage"));
let storage_path = StoragePath::from_string("test/file.txt");
let absolute = service.resolve_path(&storage_path);
assert_eq!(absolute, PathBuf::from("/storage/test/file.txt"));
}
#[test]
fn test_to_storage_path() {
let service = PathService::new(PathBuf::from("/storage"));
let physical_path = PathBuf::from("/storage/folder/file.txt");
let storage_path = service.to_storage_path(&physical_path).unwrap();
assert_eq!(storage_path.to_string(), "/folder/file.txt");
}
#[test]
fn test_is_in_root() {
let service = PathService::new(PathBuf::from("/storage"));
let root_path = StoragePath::from_string("file.txt");
let nested_path = StoragePath::from_string("folder/file.txt");
assert!(service.is_in_root(&root_path));
assert!(!service.is_in_root(&nested_path));
}
#[test]
fn test_is_direct_child() {
let service = PathService::new(PathBuf::from("/storage"));
let parent = StoragePath::from_string("folder");
let child = StoragePath::from_string("folder/file.txt");
let not_child = StoragePath::from_string("folder2/file.txt");
assert!(service.is_direct_child(&parent, &child));
assert!(!service.is_direct_child(&parent, &not_child));
}
#[test]
fn test_create_file_path() {
let service = PathService::new(PathBuf::from("/storage"));
let folder_path = StoragePath::from_string("folder");
let file_path = service.create_file_path(&folder_path, "file.txt");
assert_eq!(file_path.to_string(), "/folder/file.txt");
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+1
View File
@@ -1,3 +1,4 @@
pub mod file_fs_repository;
pub mod folder_fs_repository;
pub mod parallel_file_processor;
@@ -0,0 +1,525 @@
use std::path::PathBuf;
use std::sync::Arc;
use std::io::{self, SeekFrom};
use tokio::fs::File;
use tokio::io::{AsyncReadExt, AsyncSeekExt, AsyncWriteExt};
use tokio::task;
use tokio::sync::{Semaphore, Mutex};
use futures::future::join_all;
use tracing::{info, debug, error};
use bytes::{Bytes, BytesMut};
use crate::common::config::AppConfig;
use crate::domain::repositories::file_repository::FileRepositoryError;
use crate::infrastructure::services::buffer_pool::BufferPool;
/// Estructura para el rango de bytes a procesar
#[derive(Debug, Clone, Copy)]
pub struct ChunkRange {
/// Índice del chunk
pub index: usize,
/// Posición de inicio en bytes
pub start: u64,
/// Tamaño del chunk en bytes
pub size: usize,
}
/// Buffer pooling específico para BytesMut
pub struct BytesBufferPool {
buffers: Mutex<Vec<BytesMut>>,
buffer_size: usize,
max_buffers: usize,
}
impl BytesBufferPool {
pub fn new(buffer_size: usize, max_buffers: usize) -> Self {
Self {
buffers: Mutex::new(Vec::with_capacity(max_buffers)),
buffer_size,
max_buffers,
}
}
/// Obtener un buffer del pool o crear uno nuevo
pub async fn get_buffer(&self) -> BytesMut {
let mut buffers = self.buffers.lock().await;
if let Some(mut buffer) = buffers.pop() {
// Reutilizar buffer existente
buffer.clear(); // Mantener capacidad, limpiar contenido
buffer
} else {
// Crear nuevo buffer si el pool está vacío
BytesMut::with_capacity(self.buffer_size)
}
}
/// Devolver un buffer al pool para reutilización
pub async fn return_buffer(&self, mut buffer: BytesMut) {
// Restablece el buffer para reutilización
buffer.clear();
let mut buffers = self.buffers.lock().await;
// Solo mantener hasta max_buffers
if buffers.len() < self.max_buffers {
buffers.push(buffer);
}
// Si ya tenemos suficientes buffers, este se descartará
}
}
/// Procesador paralelo de archivos para operaciones IO intensivas
pub struct ParallelFileProcessor {
/// Configuración de la aplicación
config: AppConfig,
/// Semáforo para limitar concurrencia global
concurrency_limiter: Arc<Semaphore>,
/// Pool de buffers para optimizar memoria
buffer_pool: Option<Arc<BufferPool>>,
/// Pool de buffers BytesMut para operaciones zero-copy
bytes_pool: Arc<BytesBufferPool>,
}
impl ParallelFileProcessor {
/// Crea una nueva instancia del procesador
pub fn new(config: AppConfig) -> Self {
let concurrency_limiter = Arc::new(Semaphore::new(config.concurrency.max_concurrent_io));
// Crear pool de BytesMut para operaciones eficientes
let chunk_size = config.resources.chunk_size_bytes;
let max_chunks = config.concurrency.max_parallel_chunks;
let bytes_pool = Arc::new(BytesBufferPool::new(chunk_size, max_chunks * 2));
Self {
config,
concurrency_limiter,
buffer_pool: None,
bytes_pool,
}
}
/// Crea una nueva instancia del procesador con un pool de buffers
pub fn new_with_buffer_pool(config: AppConfig, buffer_pool: Arc<BufferPool>) -> Self {
let concurrency_limiter = Arc::new(Semaphore::new(config.concurrency.max_concurrent_io));
// Crear pool de BytesMut para operaciones eficientes
let chunk_size = config.resources.chunk_size_bytes;
let max_chunks = config.concurrency.max_parallel_chunks;
let bytes_pool = Arc::new(BytesBufferPool::new(chunk_size, max_chunks * 2));
Self {
config,
concurrency_limiter,
buffer_pool: Some(buffer_pool),
bytes_pool,
}
}
/// Divide un archivo en chunks para procesamiento paralelo
pub fn calculate_chunks(&self, file_size: u64) -> Vec<ChunkRange> {
// Determinar si el archivo necesita procesamiento paralelo
let needs_parallel = self.config.resources.needs_parallel_processing(
file_size, &self.config.concurrency
);
if !needs_parallel {
// Para archivos pequeños, usar un solo chunk
return vec![ChunkRange {
index: 0,
start: 0,
size: file_size as usize
}];
}
// Calcular número óptimo de chunks
let chunk_count = self.config.resources.calculate_optimal_chunks(
file_size, &self.config.concurrency
);
// Calcular tamaño de cada chunk
let chunk_size = self.config.resources.calculate_chunk_size(file_size, chunk_count);
// Crear los rangos de chunks
let mut chunks = Vec::with_capacity(chunk_count);
let mut start = 0;
for i in 0..chunk_count {
let current_chunk_size = if i == chunk_count - 1 {
// Último chunk puede ser más pequeño
(file_size - start) as usize
} else {
chunk_size
};
chunks.push(ChunkRange {
index: i,
start,
size: current_chunk_size,
});
start += current_chunk_size as u64;
}
debug!("File size: {} bytes, divided into {} chunks of ~{} bytes each",
file_size, chunks.len(), chunk_size);
chunks
}
/// Lee un archivo en paralelo y devuelve el contenido completo
/// Implementación optimizada usando BytesMut para reducir copias de memoria
pub async fn read_file_parallel(&self, file_path: &PathBuf) -> Result<Vec<u8>, FileRepositoryError> {
// Obtener tamaño del archivo
let metadata = tokio::fs::metadata(file_path).await
.map_err(FileRepositoryError::IoError)?;
let file_size = metadata.len();
// Verificar si el archivo es demasiado grande para memoria
if !self.config.resources.can_load_in_memory(file_size) {
return Err(FileRepositoryError::Other(
format!("File too large to load in memory: {} MB (max: {} MB)",
file_size / (1024 * 1024),
self.config.resources.max_in_memory_file_size_mb)
));
}
// Calcular chunks
let chunks = self.calculate_chunks(file_size);
if chunks.len() == 1 {
// Para un solo chunk, usar lectura simple con buffer pool si está disponible
info!("Reading file with size {}MB as a single chunk", file_size / (1024 * 1024));
if let Some(pool) = &self.buffer_pool {
// Usar buffer del pool para lectura eficiente
debug!("Using buffer pool for single chunk read");
let mut buffer = pool.get_buffer().await;
// Si el buffer es demasiado pequeño, revertir a la implementación estándar
if buffer.capacity() < file_size as usize {
debug!("Buffer from pool too small ({}), using standard read", buffer.capacity());
let content = tokio::fs::read(file_path).await
.map_err(FileRepositoryError::IoError)?;
return Ok(content);
}
// Usar el buffer de memoria del pool
let mut file = File::open(file_path).await
.map_err(FileRepositoryError::IoError)?;
let read_size = file.read(buffer.as_mut_slice()).await
.map_err(FileRepositoryError::IoError)?;
buffer.set_used(read_size);
// Convertir en Vec<u8>
let content = buffer.into_vec();
return Ok(content);
} else {
// Implementación estándar sin pool
let content = tokio::fs::read(file_path).await
.map_err(FileRepositoryError::IoError)?;
return Ok(content);
}
}
// Para múltiples chunks, usar lectura paralela
info!("Reading file with size {}MB in {} parallel chunks using BytesMut",
file_size / (1024 * 1024), chunks.len());
// Crear buffer de resultado final (pre-allocated)
let mut result = BytesMut::with_capacity(file_size as usize);
result.resize(file_size as usize, 0);
let result_mutex = Arc::new(Mutex::new(result));
// Crear tareas para cada chunk
let mut tasks = Vec::with_capacity(chunks.len());
// Abrir archivo una sola vez y compartirlo
let file = Arc::new(File::open(file_path).await
.map_err(FileRepositoryError::IoError)?);
// Referencia al pool de BytesMut
let bytes_pool = self.bytes_pool.clone();
// Procesar chunks en paralelo
for chunk in chunks {
let file_clone = file.clone();
let result_clone = result_mutex.clone();
let semaphore_clone = self.concurrency_limiter.clone();
let bytes_pool_clone = bytes_pool.clone();
// Spawn task para este chunk - no hay necesidad de copiar los datos originales
let task = task::spawn(async move {
// Adquirir permiso del semáforo
let _permit = semaphore_clone.acquire().await.unwrap();
// Obtener un buffer reusable del pool de BytesMut
let mut chunk_buffer = bytes_pool_clone.get_buffer().await;
// Asegurar que tenga suficiente capacidad
if chunk_buffer.capacity() < chunk.size {
chunk_buffer = BytesMut::with_capacity(chunk.size);
}
// Resize al tamaño exacto necesario
chunk_buffer.resize(chunk.size, 0);
// Crear un descriptor de archivo duplicado para uso independiente
let mut file_handle = file_clone.try_clone().await?;
// Posicionar y leer directamente en el BytesMut
file_handle.seek(SeekFrom::Start(chunk.start)).await?;
let bytes_read = file_handle.read_exact(&mut chunk_buffer[..chunk.size]).await?;
if bytes_read != chunk.size {
return Err(io::Error::new(
io::ErrorKind::UnexpectedEof,
format!("Expected to read {} bytes but got {}", chunk.size, bytes_read)
));
}
// Escribir en resultado final
let mut result_lock = result_clone.lock().await;
let start_pos = chunk.start as usize;
let end_pos = start_pos + chunk.size;
// Usar copy_from_slice para copiar desde BytesMut al buffer de resultado
result_lock[start_pos..end_pos].copy_from_slice(&chunk_buffer[..chunk.size]);
// Devolver el buffer al pool para su reutilización
bytes_pool_clone.return_buffer(chunk_buffer).await;
// Registrar progreso
debug!("Chunk {} processed: {} bytes from offset {}",
chunk.index, chunk.size, chunk.start);
Ok::<_, io::Error>(())
});
tasks.push(task);
}
// Esperar a que todas las tareas terminen
let results = join_all(tasks).await;
// Verificar errores
for (i, task_result) in results.into_iter().enumerate() {
match task_result {
Ok(Ok(())) => {},
Ok(Err(e)) => {
error!("Error in chunk {}: {}", i, e);
return Err(FileRepositoryError::IoError(e));
},
Err(e) => {
error!("Task error in chunk {}: {}", i, e);
return Err(FileRepositoryError::Other(format!("Task error: {}", e)));
}
}
}
// Obtener el resultado final y convertir a Vec<u8>
let result_buffer = result_mutex.lock().await;
let result_vec = result_buffer.to_vec();
info!("Successfully read file of {}MB in parallel with optimized BytesMut", file_size / (1024 * 1024));
Ok(result_vec)
}
/// Escribe un archivo en paralelo desde un buffer
/// Implementación optimizada usando BytesMut/Bytes para reducir copias de memoria
pub async fn write_file_parallel(
&self,
file_path: &PathBuf,
content: &[u8]
) -> Result<(), FileRepositoryError> {
let file_size = content.len() as u64;
// Calcular chunks
let chunks = self.calculate_chunks(file_size);
if chunks.len() == 1 {
// Para un solo chunk, usar escritura simple
info!("Writing file with size {}MB as a single chunk", file_size / (1024 * 1024));
// Implementación estándar (el buffer pooling no ofrece ventajas para escritura simple)
tokio::fs::write(file_path, content).await
.map_err(FileRepositoryError::IoError)?;
return Ok(());
}
// Para múltiples chunks, usar escritura paralela
info!("Writing file with size {}MB in {} parallel chunks using Bytes",
file_size / (1024 * 1024), chunks.len());
// Crear archivo (no usamos Mutex para reducir contención)
let file = File::create(file_path).await
.map_err(FileRepositoryError::IoError)?;
// Convertir contenido a Bytes (un solo paso de copia)
let content_bytes = Bytes::copy_from_slice(content);
// Crear tareas para cada chunk
let mut tasks = Vec::with_capacity(chunks.len());
// Procesar chunks en paralelo
for chunk in chunks {
let file_clone = file.try_clone().await
.map_err(FileRepositoryError::IoError)?;
let semaphore_clone = self.concurrency_limiter.clone();
// Crear slice de Bytes (no copia datos, solo referencia)
let start_idx = chunk.start as usize;
let end_idx = start_idx + chunk.size;
let chunk_data = content_bytes.slice(start_idx..end_idx);
// Crear y lanzar tarea
let task = task::spawn(async move {
// Adquirir permiso del semáforo
let _permit = semaphore_clone.acquire().await.unwrap();
// Posicionar y escribir
let mut file_handle = file_clone;
file_handle.seek(SeekFrom::Start(chunk.start)).await?;
file_handle.write_all(&chunk_data).await?;
// Registrar progreso
debug!("Chunk {} written: {} bytes at offset {}",
chunk.index, chunk.size, chunk.start);
Ok::<_, io::Error>(())
});
tasks.push(task);
}
// Esperar a que todas las tareas terminen
let results = join_all(tasks).await;
// Verificar errores
for (i, task_result) in results.into_iter().enumerate() {
match task_result {
Ok(Ok(())) => {},
Ok(Err(e)) => {
error!("Error in chunk {}: {}", i, e);
return Err(FileRepositoryError::IoError(e));
},
Err(e) => {
error!("Task error in chunk {}: {}", i, e);
return Err(FileRepositoryError::Other(format!("Task error: {}", e)));
}
}
}
// Garantizar que todo se ha escrito correctamente
let mut file_handle = file;
file_handle.flush().await.map_err(FileRepositoryError::IoError)?;
info!("Successfully wrote file of {}MB in parallel with optimized Bytes", file_size / (1024 * 1024));
Ok(())
}
/// Escribe un chunk en un archivo en una posición específica
#[allow(dead_code)]
async fn write_chunk_optimized(
file: &mut File,
offset: u64,
data: Bytes
) -> Result<(), std::io::Error> {
// Preparar la escritura en la posición correcta
file.seek(SeekFrom::Start(offset)).await?;
// Escribir datos sin copias adicionales
file.write_all(&data).await?;
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
#[tokio::test]
async fn test_parallel_read_write() {
// Crear configuración con umbral bajo para testing
let mut config = AppConfig::default();
config.concurrency.min_size_for_parallel_chunks_mb = 1; // 1MB para testing
config.concurrency.max_parallel_chunks = 4;
let processor = ParallelFileProcessor::new(config);
// Crear directorio temporal
let temp_dir = tempdir().unwrap();
let file_path = temp_dir.path().join("test_file.bin");
// Crear datos de prueba (2MB)
let size = 2 * 1024 * 1024;
let mut test_data = Vec::with_capacity(size);
for i in 0..size {
test_data.push((i % 256) as u8);
}
// Escribir archivo en paralelo
processor.write_file_parallel(&file_path, &test_data).await.unwrap();
// Leer archivo en paralelo
let read_data = processor.read_file_parallel(&file_path).await.unwrap();
// Verificar que los datos son idénticos
assert_eq!(test_data.len(), read_data.len());
assert_eq!(test_data, read_data);
}
#[tokio::test]
async fn test_bytesmut_pool() {
// Crear pool
let pool = BytesBufferPool::new(1024, 5);
// Obtener buffer
let mut buffer1 = pool.get_buffer().await;
buffer1.put_slice(b"test data");
assert_eq!(&buffer1[..9], b"test data");
// Devolver buffer al pool
pool.return_buffer(buffer1).await;
// Obtener otro buffer (debería ser el mismo)
let buffer2 = pool.get_buffer().await;
assert_eq!(buffer2.capacity(), 1024);
// El buffer debería estar vacío (clear)
assert_eq!(buffer2.len(), 0);
}
#[test]
fn test_chunk_calculation() {
// Crear configuración de prueba
let mut config = AppConfig::default();
config.concurrency.min_size_for_parallel_chunks_mb = 100; // 100MB
config.concurrency.max_parallel_chunks = 4;
config.concurrency.parallel_chunk_size_bytes = 50 * 1024 * 1024; // 50MB
let processor = ParallelFileProcessor::new(config);
// Archivo pequeño (10MB)
let small_file_size = 10 * 1024 * 1024;
let chunks = processor.calculate_chunks(small_file_size);
assert_eq!(chunks.len(), 1);
assert_eq!(chunks[0].size as u64, small_file_size);
// Archivo grande (300MB)
let large_file_size = 300 * 1024 * 1024;
let chunks = processor.calculate_chunks(large_file_size);
assert_eq!(chunks.len(), 4); // Limitado a max_parallel_chunks
// Verificar que todos los chunks suman el tamaño total
let total_size: u64 = chunks.iter().map(|c| c.size as u64).sum();
assert_eq!(total_size, large_file_size);
}
}
+487
View File
@@ -0,0 +1,487 @@
use std::cmp::min;
use std::collections::VecDeque;
use std::sync::Arc;
use tokio::sync::{Mutex, Semaphore};
use std::time::{Duration, Instant};
use tracing::debug;
/// Tamaño por defecto de los buffers en el pool
pub const DEFAULT_BUFFER_SIZE: usize = 64 * 1024; // 64KB
/// Número máximo por defecto de buffers en el pool
#[allow(dead_code)]
pub const DEFAULT_MAX_BUFFERS: usize = 100;
/// Tiempo de vida por defecto de un buffer inactivo (en segundos)
#[allow(dead_code)]
pub const DEFAULT_BUFFER_TTL: u64 = 60;
/// Buffer pooling para optimizar operaciones de lectura/escritura
pub struct BufferPool {
/// Pool de buffers disponibles
pool: Mutex<VecDeque<PooledBuffer>>,
/// Semáforo para limitar el número máximo de buffers
limit: Semaphore,
/// Tamaño de los buffers en el pool
buffer_size: usize,
/// Estadísticas del pool
stats: Mutex<BufferPoolStats>,
/// Tiempo de vida de un buffer inactivo
buffer_ttl: Duration,
}
/// Estructura para tracking de estadísticas del pool
#[derive(Debug, Clone, Default)]
pub struct BufferPoolStats {
/// Número total de operaciones de get
pub gets: usize,
/// Número de hits del pool (reutilización exitosa)
pub hits: usize,
/// Número de misses (creación de nuevo buffer)
pub misses: usize,
/// Número de retornos al pool
pub returns: usize,
/// Número de eviction por TTL
pub evictions: usize,
/// Número máximo de buffers alcanzado
pub max_buffers_reached: usize,
/// Esperas por semáforo
pub waits: usize,
}
/// Buffer del pool con metadatos para gestión
struct PooledBuffer {
/// Buffer real de bytes
buffer: Vec<u8>,
/// Timestamp de cuándo se añadió/retornó al pool
last_used: Instant,
}
/// Buffer prestado del pool con cleanup automático
#[derive(Clone)]
pub struct BorrowedBuffer {
/// Buffer actual
buffer: Vec<u8>,
/// Tamaño real utilizado del buffer
used_size: usize,
/// Referencia al pool para retornar
pool: Arc<BufferPool>,
/// Si el buffer debe o no retornarse al pool
return_to_pool: bool,
}
impl BufferPool {
/// Crea un nuevo pool de buffers
pub fn new(buffer_size: usize, max_buffers: usize, buffer_ttl_secs: u64) -> Arc<Self> {
Arc::new(Self {
pool: Mutex::new(VecDeque::with_capacity(max_buffers)),
limit: Semaphore::new(max_buffers),
buffer_size,
stats: Mutex::new(BufferPoolStats::default()),
buffer_ttl: Duration::from_secs(buffer_ttl_secs),
})
}
/// Crea un pool con configuración por defecto
#[allow(dead_code)]
pub fn default() -> Arc<Self> {
Self::new(
DEFAULT_BUFFER_SIZE,
DEFAULT_MAX_BUFFERS,
DEFAULT_BUFFER_TTL
)
}
/// Obtiene un buffer del pool o crea uno nuevo si es necesario
#[allow(unused_variables)]
pub async fn get_buffer(&self) -> BorrowedBuffer {
// Incrementar contador de gets
{
let mut stats = self.stats.lock().await;
stats.gets += 1;
}
// Control de concurrencia
// Usando el mecanismo RAII de Rust para gestión automática
// de recursos al finalizar la función
let _ = match self.limit.try_acquire() {
Ok(_permit) => _permit, // _ prefix para indicar que es intencional
Err(_) => {
// No hay permisos disponibles, esperamos
let mut stats = self.stats.lock().await;
stats.waits += 1;
stats.max_buffers_reached += 1;
drop(stats);
debug!("Buffer pool: waiting for available buffer");
let _permit = self.limit.acquire().await.expect("Semaphore should not be closed");
debug!("Buffer pool: acquired buffer after waiting");
_permit
}
};
// Intentar obtener un buffer existente del pool
let mut pool_locked = self.pool.lock().await;
if let Some(mut pooled_buffer) = pool_locked.pop_front() {
// Verificar si el buffer ha expirado
if pooled_buffer.last_used.elapsed() > self.buffer_ttl {
// Buffer expirado, descartamos y creamos uno nuevo
let mut stats = self.stats.lock().await;
stats.evictions += 1;
stats.misses += 1;
drop(stats);
debug!("Buffer pool: evicted expired buffer");
// Crear nuevo buffer (reutilizando el permiso)
drop(pool_locked); // Liberar el lock antes de retornar
BorrowedBuffer {
buffer: vec![0; self.buffer_size],
used_size: 0,
pool: Arc::new(self.clone()),
return_to_pool: true,
}
} else {
// Buffer válido, lo reutilizamos
let mut stats = self.stats.lock().await;
stats.hits += 1;
drop(stats);
// Liberar el lock antes de retornar
drop(pool_locked);
// Limpiar buffer por seguridad
pooled_buffer.buffer.fill(0);
BorrowedBuffer {
buffer: pooled_buffer.buffer,
used_size: 0,
pool: Arc::new(self.clone()),
return_to_pool: true,
}
}
} else {
// No hay buffers disponibles, creamos uno nuevo
let mut stats = self.stats.lock().await;
stats.misses += 1;
drop(stats);
// Liberar el lock antes de retornar
drop(pool_locked);
debug!("Buffer pool: creating new buffer");
BorrowedBuffer {
buffer: vec![0; self.buffer_size],
used_size: 0,
pool: Arc::new(self.clone()),
return_to_pool: true,
}
}
}
/// Retorna un buffer al pool
async fn return_buffer(&self, mut buffer: Vec<u8>) {
// Si el buffer es del tamaño incorrecto, lo descartamos
if buffer.capacity() != self.buffer_size {
debug!("Buffer pool: discarding buffer of wrong size: {} (expected {})",
buffer.capacity(), self.buffer_size);
return;
}
// Resize para asegurar capacidad correcta
buffer.resize(self.buffer_size, 0);
// Añadir al pool
let mut pool_locked = self.pool.lock().await;
pool_locked.push_back(PooledBuffer {
buffer,
last_used: Instant::now(),
});
// Actualizar estadísticas
let mut stats = self.stats.lock().await;
stats.returns += 1;
}
/// Limpia buffers expirados del pool
pub async fn clean_expired_buffers(&self) {
let _now = Instant::now();
let mut pool_locked = self.pool.lock().await;
// Contar expirados
let count_before = pool_locked.len();
// Filtrar manteniendo solo los no expirados
pool_locked.retain(|buffer| {
buffer.last_used.elapsed() <= self.buffer_ttl
});
// Contar cuántos se eliminaron
let removed = count_before - pool_locked.len();
if removed > 0 {
// Actualizar estadísticas
let mut stats = self.stats.lock().await;
stats.evictions += removed;
debug!("Buffer pool: cleaned {} expired buffers", removed);
}
}
/// Obtiene estadísticas actuales del pool
pub async fn get_stats(&self) -> BufferPoolStats {
self.stats.lock().await.clone()
}
/// Inicia la tarea periódica de limpieza
pub fn start_cleaner(pool: Arc<Self>) {
tokio::spawn(async move {
let interval = Duration::from_secs(30); // Limpiar cada 30 segundos
loop {
tokio::time::sleep(interval).await;
pool.clean_expired_buffers().await;
// Loguear estadísticas periódicamente
let stats = pool.get_stats().await;
debug!("Buffer pool stats: gets={}, hits={}, misses={}, hit_ratio={:.2}%, returns={}, \
evictions={}, max_reached={}, waits={}",
stats.gets,
stats.hits,
stats.misses,
if stats.gets > 0 { (stats.hits as f64 * 100.0) / stats.gets as f64 } else { 0.0 },
stats.returns,
stats.evictions,
stats.max_buffers_reached,
stats.waits);
}
});
}
}
impl Clone for BufferPool {
fn clone(&self) -> Self {
Self {
pool: Mutex::new(VecDeque::new()),
limit: Semaphore::new(self.limit.available_permits()),
buffer_size: self.buffer_size,
stats: Mutex::new(BufferPoolStats::default()),
buffer_ttl: self.buffer_ttl,
}
}
}
impl BorrowedBuffer {
/// Accede al buffer interno
pub fn as_mut_slice(&mut self) -> &mut [u8] {
&mut self.buffer
}
/// Obtiene una referencia a los datos utilizados
#[allow(dead_code)]
pub fn as_slice(&self) -> &[u8] {
&self.buffer[..self.used_size]
}
/// Establece cuántos bytes se utilizaron realmente
pub fn set_used(&mut self, size: usize) {
self.used_size = min(size, self.buffer.len());
}
/// Convierte en un Vec<u8> que incluye solo los datos utilizados
pub fn into_vec(mut self) -> Vec<u8> {
// Marcar para no devolver al pool
self.return_to_pool = false;
// Crear un nuevo vector solo con los datos utilizados
self.buffer[..self.used_size].to_vec()
}
/// Copia datos a este buffer y actualiza el tamaño usado
#[allow(dead_code)]
pub fn copy_from_slice(&mut self, data: &[u8]) -> usize {
let copy_size = min(data.len(), self.buffer.len());
self.buffer[..copy_size].copy_from_slice(&data[..copy_size]);
self.used_size = copy_size;
copy_size
}
/// Impide que el buffer se devuelva al pool al destruirse
#[allow(dead_code)]
pub fn do_not_return(mut self) -> Self {
self.return_to_pool = false;
self
}
/// Obtiene el tamaño total del buffer
pub fn capacity(&self) -> usize {
self.buffer.len()
}
/// Obtiene el tamaño usado del buffer
#[allow(dead_code)]
pub fn used_size(&self) -> usize {
self.used_size
}
}
// Cuando se hace drop de un BorrowedBuffer, lo devuelve al pool
impl Drop for BorrowedBuffer {
fn drop(&mut self) {
if self.return_to_pool {
// Tomar posesión del buffer y crear un clone del pool
let buffer = std::mem::take(&mut self.buffer);
let pool = self.pool.clone();
// Spawn del return para que el drop no bloquee
tokio::spawn(async move {
pool.return_buffer(buffer).await;
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_buffer_pooling() {
// Crear pool pequeño para testing
let pool = BufferPool::new(1024, 5, 60);
// Obtener un buffer
let mut buffer1 = pool.get_buffer().await;
buffer1.copy_from_slice(b"test data");
assert_eq!(buffer1.as_slice(), b"test data");
// Obtener otro buffer
let buffer2 = pool.get_buffer().await;
// Verificar stats
let stats = pool.get_stats().await;
assert_eq!(stats.gets, 2);
assert_eq!(stats.hits, 0); // sin hits todavía
assert_eq!(stats.misses, 2); // todos son misses
// Devolver buffer1 al pool (implícitamente por drop)
drop(buffer1);
// Permitir que el return asíncrono ocurra
tokio::time::sleep(Duration::from_millis(10)).await;
// Obtener otro buffer (debería reutilizar el retornado)
let buffer3 = pool.get_buffer().await;
// Verificar stats actualizados
let stats = pool.get_stats().await;
assert_eq!(stats.gets, 3);
assert_eq!(stats.hits, 1); // ahora debería haber un hit
assert_eq!(stats.returns, 1); // un buffer retornado
// Limpiar
drop(buffer2);
drop(buffer3);
}
#[tokio::test]
async fn test_buffer_operations() {
let pool = BufferPool::new(1024, 10, 60);
// Obtener buffer
let mut buffer = pool.get_buffer().await;
// Escribir datos
buffer.copy_from_slice(b"Hello, world!");
assert_eq!(buffer.used_size(), 13);
assert_eq!(buffer.as_slice(), b"Hello, world!");
// Convertir a vec y verificar
let vec = buffer.into_vec(); // Esto impide retornar al pool
assert_eq!(vec, b"Hello, world!");
// Verificar que no se incrementan los returns (buffer no retornado)
tokio::time::sleep(Duration::from_millis(10)).await;
let stats = pool.get_stats().await;
assert_eq!(stats.returns, 0);
}
#[tokio::test]
async fn test_pool_limit() {
// Pool con solo 3 buffers
let pool = BufferPool::new(1024, 3, 60);
// Obtener 3 buffers (alcanza el límite)
let buffer1 = pool.get_buffer().await;
let buffer2 = pool.get_buffer().await;
let buffer3 = pool.get_buffer().await;
// Verificar stats
let stats = pool.get_stats().await;
assert_eq!(stats.gets, 3);
assert_eq!(stats.waits, 0); // sin esperas todavía
// Intentar obtener un 4º buffer en una tarea separada (debería esperar)
let pool_clone = pool.clone();
let handle = tokio::spawn(async move {
let _buffer4 = pool_clone.get_buffer().await;
true
});
// Dar tiempo para que la tarea intente tomar el buffer
tokio::time::sleep(Duration::from_millis(50)).await;
// Verificar que hay una espera
let stats = pool.get_stats().await;
assert_eq!(stats.waits, 1);
// Liberar un buffer
drop(buffer1);
// Dar tiempo para el retorno asíncrono y para que la tarea en espera obtenga su buffer
tokio::time::sleep(Duration::from_millis(50)).await;
// Verificar que la tarea pudo continuar
assert!(handle.await.unwrap());
// Limpiar
drop(buffer2);
drop(buffer3);
}
#[tokio::test]
async fn test_ttl_expiration() {
// Pool con TTL muy corto para testing
let pool = BufferPool::new(1024, 5, 1); // 1 segundo TTL
// Obtener y devolver un buffer
let buffer = pool.get_buffer().await;
drop(buffer);
// Permitir que el return asíncrono ocurra
tokio::time::sleep(Duration::from_millis(50)).await;
// Verificar que hay un buffer en el pool
let stats = pool.get_stats().await;
assert_eq!(stats.returns, 1);
// Esperar a que expire el TTL
tokio::time::sleep(Duration::from_secs(2)).await;
// Limpiar expirados
pool.clean_expired_buffers().await;
// Obtener otro buffer (debería ser un miss ya que el anterior expiró)
let _buffer2 = pool.get_buffer().await;
// Verificar stats
let stats = pool.get_stats().await;
assert_eq!(stats.evictions, 1); // un buffer expirado
assert_eq!(stats.hits, 0); // sin hits (el buffer expiró)
assert_eq!(stats.misses, 2); // dos misses (1er y 3er get)
}
}
@@ -0,0 +1,212 @@
use std::collections::HashMap;
use std::path::PathBuf;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::time;
use futures::future::BoxFuture;
use tokio::sync::RwLock;
/// Representación de metadatos en caché
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct CachedMetadata {
/// Si el archivo o directorio existe
pub exists: bool,
/// Tamaño en bytes (para archivos)
pub size: Option<u64>,
/// Timestamp de creación
pub created_at: Option<u64>,
/// Timestamp de modificación
pub modified_at: Option<u64>,
/// Tiempo de expiración de la caché
expires_at: Instant,
}
/// Estructura para gestionar la caché de metadatos de archivos y directorios
#[allow(dead_code)]
pub struct StorageCacheManager {
/// Caché de existencia y metadatos
cache: RwLock<HashMap<PathBuf, CachedMetadata>>,
/// TTL para entradas de archivos (milisegundos)
file_ttl_ms: u64,
/// TTL para entradas de directorios (milisegundos)
dir_ttl_ms: u64,
/// Tamaño máximo de caché
max_entries: usize,
}
impl StorageCacheManager {
/// Crea una nueva instancia del gestor de caché
#[allow(dead_code)]
pub fn new(file_ttl_ms: u64, dir_ttl_ms: u64, max_entries: usize) -> Self {
Self {
cache: RwLock::new(HashMap::with_capacity(max_entries)),
file_ttl_ms,
dir_ttl_ms,
max_entries,
}
}
/// Crea una instancia por defecto del gestor de caché
#[allow(dead_code)]
pub fn default() -> Self {
Self::new(
60_000, // 1 minuto para archivos
300_000, // 5 minutos para directorios
10_000, // máximo 10,000 entradas
)
}
/// Verifica si un archivo o directorio existe en caché
#[allow(dead_code)]
pub async fn check_exists(&self, path: &PathBuf, _is_dir: bool) -> Result<bool, ()> {
// Intentar obtener de la caché
if let Some(metadata) = self.get_cached_metadata(path).await {
return Ok(metadata.exists);
}
// No está en caché
Err(())
}
/// Obtiene los metadatos de un path desde la caché
#[allow(dead_code)]
async fn get_cached_metadata(&self, path: &PathBuf) -> Option<CachedMetadata> {
let cache = self.cache.read().await;
if let Some(metadata) = cache.get(path) {
// Verificar si la entrada expiró
if Instant::now() < metadata.expires_at {
return Some(metadata.clone());
}
}
None
}
/// Actualiza la caché con los metadatos de un path
#[allow(dead_code)]
pub async fn update_cache(&self, path: &PathBuf, exists: bool, size: Option<u64>,
created_at: Option<u64>, modified_at: Option<u64>, is_dir: bool) {
let mut cache = self.cache.write().await;
// Si la caché está llena, eliminar entradas aleatorias antes de agregar
if cache.len() >= self.max_entries {
self.evict_entries(&mut cache, 100).await;
}
// Determinar TTL basado en si es archivo o directorio
let ttl = if is_dir {
Duration::from_millis(self.dir_ttl_ms)
} else {
Duration::from_millis(self.file_ttl_ms)
};
// Crear metadatos y agregar a la caché
let metadata = CachedMetadata {
exists,
size,
created_at,
modified_at,
expires_at: Instant::now() + ttl,
};
cache.insert(path.clone(), metadata);
}
/// Elimina entradas aleatorias de la caché cuando está llena
#[allow(dead_code)]
async fn evict_entries(&self, cache: &mut HashMap<PathBuf, CachedMetadata>, count: usize) {
// Obtener las entradas más antiguas para eliminar
let mut entries: Vec<_> = cache.keys().cloned().collect();
// Limitar el número de entradas a eliminar
let to_remove = count.min(entries.len() / 10);
if to_remove == 0 {
return;
}
// Eliminar las primeras entradas (implementación simple)
entries.truncate(to_remove);
for path in entries {
cache.remove(&path);
}
}
/// Inicia una tarea de limpieza periódica
#[allow(dead_code)]
pub fn start_cleanup_task(cache_manager: Arc<Self>) -> BoxFuture<'static, ()> {
Box::pin(async move {
let interval = Duration::from_secs(60); // Ejecutar cada minuto
loop {
time::sleep(interval).await;
// Limpiar entradas expiradas
let now = Instant::now();
let mut cache = cache_manager.cache.write().await;
// Encontrar entradas expiradas
let expired: Vec<_> = cache
.iter()
.filter(|(_, metadata)| now > metadata.expires_at)
.map(|(path, _)| path.clone())
.collect();
// Eliminar entradas expiradas
for path in expired {
cache.remove(&path);
}
// Registrar estadísticas
let cache_size = cache.len();
drop(cache);
tracing::debug!("Cache cleanup completed. Entries remaining: {}", cache_size);
}
})
}
/// Invalida una entrada específica de la caché
#[allow(dead_code)]
pub async fn invalidate(&self, path: &PathBuf) {
let mut cache = self.cache.write().await;
cache.remove(path);
}
/// Invalida todas las entradas de la caché relacionadas con una carpeta
#[allow(dead_code)]
pub async fn invalidate_folder(&self, folder_path: &PathBuf) {
let mut cache = self.cache.write().await;
// Eliminar entradas que sean descendientes de la carpeta
let folder_str = folder_path.to_string_lossy().to_string();
// Encontrar entradas a eliminar
let to_remove: Vec<_> = cache
.keys()
.filter_map(|path| {
let path_str = path.to_string_lossy().to_string();
if path_str.starts_with(&folder_str) {
Some(path.clone())
} else {
None
}
})
.collect();
// Eliminar las entradas
for path in to_remove {
cache.remove(&path);
}
}
/// Obtiene el número actual de entradas en la caché
#[allow(dead_code)]
pub async fn cache_size(&self) -> usize {
let cache = self.cache.read().await;
cache.len()
}
}
@@ -0,0 +1,425 @@
use std::io::{Read};
use std::sync::Arc;
use async_trait::async_trait;
use bytes::Bytes;
use futures::{Stream, StreamExt};
use tracing::error;
use std::io;
use flate2::Compression;
use flate2::read::GzEncoder as GzEncoderRead;
use flate2::bufread::GzDecoder;
use crate::infrastructure::services::buffer_pool::BufferPool;
/// Nivel de compresión para ficheros
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CompressionLevel {
/// Sin compresión (solo para transferencia)
None = 0,
/// Compresión rápida con menor ratio
Fast = 1,
/// Compresión balanceada (por defecto)
Default = 6,
/// Compresión máxima (más lenta)
Best = 9,
}
impl From<CompressionLevel> for Compression {
fn from(level: CompressionLevel) -> Self {
match level {
CompressionLevel::None => Compression::none(),
CompressionLevel::Fast => Compression::fast(),
CompressionLevel::Default => Compression::default(),
CompressionLevel::Best => Compression::best(),
}
}
}
/// Umbral de tamaño para decidir si se comprime o no
const COMPRESSION_SIZE_THRESHOLD: u64 = 1024 * 50; // 50KB
/// Interfaz para servicios de compresión
#[async_trait]
pub trait CompressionService: Send + Sync {
/// Comprime datos en memoria
async fn compress_data(&self, data: &[u8], level: CompressionLevel) -> io::Result<Vec<u8>>;
/// Descomprime datos en memoria
async fn decompress_data(&self, compressed_data: &[u8]) -> io::Result<Vec<u8>>;
/// Comprime un stream de datos
#[allow(dead_code)]
fn compress_stream<S>(&self, stream: S, level: CompressionLevel)
-> impl Stream<Item = io::Result<Bytes>> + Send
where
S: Stream<Item = io::Result<Bytes>> + Send + 'static + Unpin;
/// Descomprime un stream de datos
#[allow(dead_code)]
fn decompress_stream<S>(&self, compressed_stream: S)
-> impl Stream<Item = io::Result<Bytes>> + Send
where
S: Stream<Item = io::Result<Bytes>> + Send + 'static + Unpin;
/// Determina si un archivo debe ser comprimido basado en su tipo MIME y tamaño
fn should_compress(&self, mime_type: &str, size: u64) -> bool;
}
/// Implementación de servicios de compresión usando Gzip
pub struct GzipCompressionService {
/// Pool de buffers para optimización de memoria
buffer_pool: Option<Arc<BufferPool>>,
}
impl GzipCompressionService {
/// Crea una nueva instancia del servicio
pub fn new() -> Self {
Self {
buffer_pool: None,
}
}
/// Crea una nueva instancia del servicio con buffer pool
pub fn new_with_buffer_pool(buffer_pool: Arc<BufferPool>) -> Self {
Self {
buffer_pool: Some(buffer_pool),
}
}
}
#[async_trait]
impl CompressionService for GzipCompressionService {
/// Comprime datos en memoria usando Gzip
async fn compress_data(&self, data: &[u8], level: CompressionLevel) -> io::Result<Vec<u8>> {
// Si tenemos un buffer pool, usar un buffer prestado para la compresión
if let Some(pool) = &self.buffer_pool {
// Estimar el tamaño de la compresión (aproximadamente 80% del original para casos típicos)
let estimated_size = (data.len() as f64 * 0.8) as usize;
// Obtener un buffer del pool
let buffer = pool.get_buffer().await;
// Comprobar si el buffer es suficientemente grande
if buffer.capacity() >= estimated_size {
// Ejecutar la compresión en un worker thread usando el buffer
let buffer_ptr = Arc::new(tokio::sync::Mutex::new(buffer));
let buffer_clone = buffer_ptr.clone();
// Comprimir datos
// Clonar los datos para evitar problemas de lifetime
let data_owned = data.to_vec();
let result = tokio::task::spawn_blocking(move || {
let mut encoder = GzEncoderRead::new(&data_owned[..], level.into());
// Intentar bloquear el mutex (no debería fallar ya que estamos en un hilo separado)
let mut buffer_guard = match futures::executor::block_on(buffer_clone.lock()) {
buffer => buffer,
};
// Leer directamente en el buffer
let read_bytes = encoder.read(buffer_guard.as_mut_slice())?;
buffer_guard.set_used(read_bytes);
Ok(()) as io::Result<()>
}).await;
// Verificar resultado
match result {
Ok(Ok(())) => {
// Obtener el buffer y convertirlo a Vec<u8>
let buffer = buffer_ptr.lock().await;
let cloned_buffer = buffer.clone();
drop(buffer); // Liberar el mutex primero
return Ok(cloned_buffer.into_vec());
},
Ok(Err(e)) => {
error!("Error en compresión con buffer pool: {}", e);
// Continuar con implementación estándar
},
Err(e) => {
error!("Error en task de compresión con buffer pool: {}", e);
// Continuar con implementación estándar
}
}
}
}
// Implementación estándar si no hay buffer pool o el buffer es insuficiente
// Clonar los datos para evitar problemas de lifetime
let data_owned = data.to_vec();
tokio::task::spawn_blocking(move || {
let mut encoder = GzEncoderRead::new(&data_owned[..], level.into());
let mut compressed = Vec::new();
encoder.read_to_end(&mut compressed)?;
Ok(compressed)
}).await.unwrap_or_else(|e| {
error!("Error en task de compresión: {}", e);
Err(io::Error::new(io::ErrorKind::Other, e.to_string()))
})
}
/// Descomprime datos en memoria
async fn decompress_data(&self, compressed_data: &[u8]) -> io::Result<Vec<u8>> {
// Si tenemos un buffer pool, usar un buffer prestado para la descompresión
if let Some(pool) = &self.buffer_pool {
// Estimar el tamaño de la descompresión (aproximadamente 5x del comprimido para casos típicos)
let estimated_size = compressed_data.len() * 5;
// Obtener un buffer del pool
let buffer = pool.get_buffer().await;
// Comprobar si el buffer es suficientemente grande
if buffer.capacity() >= estimated_size {
// Clonar datos comprimidos para mover al worker
let data = compressed_data.to_vec();
let buffer_ptr = Arc::new(tokio::sync::Mutex::new(buffer));
let buffer_clone = buffer_ptr.clone();
// Descomprimir datos
let result = tokio::task::spawn_blocking(move || {
let mut decoder = GzDecoder::new(&data[..]);
// Intentar bloquear el mutex
let mut buffer_guard = match futures::executor::block_on(buffer_clone.lock()) {
buffer => buffer,
};
// Leer directamente en el buffer
let read_bytes = decoder.read(buffer_guard.as_mut_slice())?;
buffer_guard.set_used(read_bytes);
Ok(()) as io::Result<()>
}).await;
// Verificar resultado
match result {
Ok(Ok(())) => {
// Obtener el buffer y convertirlo a Vec<u8>
let buffer = buffer_ptr.lock().await;
let cloned_buffer = buffer.clone();
drop(buffer); // Liberar el mutex primero
return Ok(cloned_buffer.into_vec());
},
Ok(Err(e)) => {
error!("Error en descompresión con buffer pool: {}", e);
// Continuar con implementación estándar
},
Err(e) => {
error!("Error en task de descompresión con buffer pool: {}", e);
// Continuar con implementación estándar
}
}
}
}
// Implementación estándar si no hay buffer pool o el buffer es insuficiente
let data = compressed_data.to_vec(); // Clonar para mover al worker
tokio::task::spawn_blocking(move || {
let mut decoder = GzDecoder::new(&data[..]);
let mut decompressed = Vec::new();
decoder.read_to_end(&mut decompressed)?;
Ok(decompressed)
}).await.unwrap_or_else(|e| {
error!("Error en task de descompresión: {}", e);
Err(io::Error::new(io::ErrorKind::Other, e.to_string()))
})
}
/// Comprime un stream de bytes
fn compress_stream<S>(&self, stream: S, level: CompressionLevel)
-> impl Stream<Item = io::Result<Bytes>> + Send
where
S: Stream<Item = io::Result<Bytes>> + Send + 'static + Unpin
{
// For now, simplify the implementation to avoid complex pinning issues
// This implementation collects all stream data and then compresses it at once
// Future optimization would be to implement true streaming compression
let compression_level = level;
Box::pin(async_stream::stream! {
let mut data = Vec::new();
// Collect all bytes from the stream
let mut stream = Box::pin(stream);
while let Some(result) = stream.next().await {
match result {
Ok(bytes) => {
data.extend_from_slice(&bytes);
},
Err(e) => {
yield Err(e);
return;
}
}
}
// Compress collected data
match self.compress_data(&data, compression_level).await {
Ok(compressed) => {
// Return compressed data as a single chunk
yield Ok(Bytes::from(compressed));
},
Err(e) => {
yield Err(e);
}
}
})
}
/// Descomprime un stream de bytes
fn decompress_stream<S>(&self, compressed_stream: S)
-> impl Stream<Item = io::Result<Bytes>> + Send
where
S: Stream<Item = io::Result<Bytes>> + Send + 'static + Unpin
{
// For now, simplify the implementation to avoid complex pinning issues
// This implementation collects all stream data and then decompresses it at once
// Future optimization would be to implement streaming decompression correctly
Box::pin(async_stream::stream! {
let mut compressed_data = Vec::new();
// Collect all bytes from the stream
let mut stream = Box::pin(compressed_stream);
while let Some(result) = stream.next().await {
match result {
Ok(bytes) => {
compressed_data.extend_from_slice(&bytes);
},
Err(e) => {
yield Err(e);
return;
}
}
}
// Decompress collected data
match self.decompress_data(&compressed_data).await {
Ok(decompressed) => {
// Return decompressed data as a single chunk
yield Ok(Bytes::from(decompressed));
},
Err(e) => {
yield Err(e);
}
}
})
}
/// Determina si un archivo debe ser comprimido basado en su tipo MIME y tamaño
fn should_compress(&self, mime_type: &str, size: u64) -> bool {
// No comprimir archivos muy pequeños (overhead)
if size < COMPRESSION_SIZE_THRESHOLD {
return false;
}
// No comprimir archivos ya comprimidos
if mime_type.starts_with("image/")
&& !mime_type.contains("svg")
&& !mime_type.contains("bmp") {
return false;
}
if mime_type.starts_with("audio/")
|| mime_type.starts_with("video/")
|| mime_type.contains("zip")
|| mime_type.contains("gzip")
|| mime_type.contains("compressed")
|| mime_type.contains("7z")
|| mime_type.contains("rar")
|| mime_type.contains("bz2")
|| mime_type.contains("xz")
|| mime_type.contains("jpg")
|| mime_type.contains("jpeg")
|| mime_type.contains("png")
|| mime_type.contains("gif")
|| mime_type.contains("webp")
|| mime_type.contains("mp3")
|| mime_type.contains("mp4")
|| mime_type.contains("ogg")
|| mime_type.contains("webm") {
return false;
}
// Comprimir archivos de texto, documentos, y otros tipos compresibles
true
}
}
#[cfg(test)]
mod tests {
use super::*;
use tokio_stream::StreamExt;
use futures::TryStreamExt;
#[tokio::test]
async fn test_compress_decompress_data() {
let service = GzipCompressionService::new();
// Datos de prueba
let data = "Hello, world! ".repeat(1000).into_bytes();
// Comprimir
let compressed = service.compress_data(&data, CompressionLevel::Default).await.unwrap();
// Verificar que la compresión reduce el tamaño
assert!(compressed.len() < data.len());
// Descomprimir
let decompressed = service.decompress_data(&compressed).await.unwrap();
// Verificar que los datos originales se recuperan correctamente
assert_eq!(decompressed, data);
}
#[tokio::test]
async fn test_compress_decompress_stream() {
let service = GzipCompressionService::new();
// Crear datos de prueba
let chunks = vec![
Ok(Bytes::from("Hello, ")),
Ok(Bytes::from("world! ")),
Ok(Bytes::from("This is a test of streaming compression.")),
];
// Convertir a stream
let input_stream = futures::stream::iter(chunks);
// Comprimir el stream
let compressed_stream = service.compress_stream(input_stream, CompressionLevel::Default);
// Recolectar los bytes comprimidos
let compressed_bytes = compressed_stream
.try_fold(Vec::new(), |mut acc, chunk| async move {
acc.extend_from_slice(&chunk);
Ok(acc)
}).await.unwrap();
// Descomprimir los datos
let decompressed = service.decompress_data(&compressed_bytes).await.unwrap();
// Verificar resultado
let expected = "Hello, world! This is a test of streaming compression.";
assert_eq!(String::from_utf8(decompressed).unwrap(), expected);
}
#[test]
fn test_should_compress() {
let service = GzipCompressionService::new();
// Casos que no deberían comprimirse
assert!(!service.should_compress("image/jpeg", 100 * 1024));
assert!(!service.should_compress("video/mp4", 10 * 1024 * 1024));
assert!(!service.should_compress("application/zip", 5 * 1024 * 1024));
// Casos que sí deberían comprimirse
assert!(service.should_compress("text/html", 100 * 1024));
assert!(service.should_compress("application/json", 200 * 1024));
assert!(service.should_compress("text/plain", 1024 * 1024));
// Archivos pequeños no deberían comprimirse independientemente del tipo
assert!(!service.should_compress("text/html", 10 * 1024));
}
}
@@ -0,0 +1,659 @@
use std::collections::{HashMap, VecDeque};
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant, UNIX_EPOCH};
use tokio::fs;
use tokio::sync::RwLock;
use tokio::time;
use futures::future::BoxFuture;
use tracing::debug;
use mime_guess::from_path;
use crate::common::config::AppConfig;
/// Tipos de entradas en caché
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CacheEntryType {
/// Archivo
File,
/// Directorio
Directory,
/// Tipo desconocido
Unknown,
}
/// Estadísticas de caché para monitoreo
#[derive(Debug, Clone, Default)]
pub struct CacheStats {
/// Número de hits en caché
pub hits: usize,
/// Número de misses en caché
pub misses: usize,
/// Número de invalidaciones manuales
pub invalidations: usize,
/// Número de expiraciones automáticas
pub expirations: usize,
/// Número de inserciones en caché
pub inserts: usize,
/// Tiempo total ahorrado (milisegundos)
pub time_saved_ms: u64,
}
/// Metadatos completos de archivo en caché
#[derive(Debug, Clone)]
pub struct FileMetadata {
/// Ruta absoluta del archivo
pub path: PathBuf,
/// Si el archivo existe físicamente
#[allow(dead_code)]
pub exists: bool,
/// Tipo de entrada (archivo, directorio)
pub entry_type: CacheEntryType,
/// Tamaño en bytes (para archivos)
pub size: Option<u64>,
/// Tipo MIME (para archivos)
#[allow(dead_code)]
pub mime_type: Option<String>,
/// Timestamp de creación (UNIX epoch seconds)
pub created_at: Option<u64>,
/// Timestamp de modificación (UNIX epoch seconds)
pub modified_at: Option<u64>,
/// Acceso previo (usado para LRU)
pub last_access: Instant,
/// Tiempo de expiración de la caché
pub expires_at: Instant,
/// Número de accesos a esta entrada
pub access_count: usize,
}
impl FileMetadata {
/// Crea una nueva entrada de metadatos
pub fn new(
path: PathBuf,
exists: bool,
entry_type: CacheEntryType,
size: Option<u64>,
mime_type: Option<String>,
created_at: Option<u64>,
modified_at: Option<u64>,
ttl: Duration,
) -> Self {
let now = Instant::now();
Self {
path,
exists,
entry_type,
size,
mime_type,
created_at,
modified_at,
last_access: now,
expires_at: now + ttl,
access_count: 1,
}
}
/// Actualiza el tiempo de último acceso
pub fn touch(&mut self) {
self.last_access = Instant::now();
self.access_count += 1;
}
/// Verifica si la entrada ha expirado
pub fn is_expired(&self) -> bool {
Instant::now() > self.expires_at
}
/// Actualiza el tiempo de expiración con un nuevo TTL
pub fn update_expiry(&mut self, ttl: Duration) {
self.expires_at = Instant::now() + ttl;
}
}
/// Caché avanzada de metadatos de archivos
pub struct FileMetadataCache {
/// Caché principal de metadatos
metadata_cache: RwLock<HashMap<PathBuf, FileMetadata>>,
/// Cola LRU para administración de caché
lru_queue: RwLock<VecDeque<PathBuf>>,
/// Estadísticas de uso del caché
stats: RwLock<CacheStats>,
/// Configuración global de la aplicación
config: AppConfig,
/// TTL adaptativo para entradas populares
ttl_multiplier: f64,
/// Umbral de popularidad para TTL extendido
popularity_threshold: usize,
/// Tamaño máximo de caché
max_entries: usize,
}
impl FileMetadataCache {
/// Crea una nueva instancia de caché de metadatos
pub fn new(config: AppConfig, max_entries: usize) -> Self {
Self {
metadata_cache: RwLock::new(HashMap::with_capacity(max_entries)),
lru_queue: RwLock::new(VecDeque::with_capacity(max_entries)),
stats: RwLock::new(CacheStats::default()),
config,
ttl_multiplier: 5.0, // Entradas populares tienen 5x TTL
popularity_threshold: 10, // Después de 10 accesos se considera popular
max_entries,
}
}
/// Crea una instancia de caché con configuración por defecto
pub fn default_with_config(config: AppConfig) -> Self {
Self::new(config, 50_000) // Caché más grande para sistema en producción
}
/// Obtiene los metadatos de un archivo si están en caché
pub async fn get_metadata(&self, path: &Path) -> Option<FileMetadata> {
let start_time = Instant::now();
let mut cache = self.metadata_cache.write().await;
if let Some(metadata) = cache.get_mut(path) {
// Verificar si ha expirado
if metadata.is_expired() {
// Eliminar de caché si expiró
cache.remove(path);
// Actualizar estadísticas
let mut stats = self.stats.write().await;
stats.misses += 1;
stats.expirations += 1;
debug!("Cache entry expired for: {}", path.display());
return None;
}
// Actualizar tiempo de acceso
metadata.touch();
// Para entradas populares, extender TTL
if metadata.access_count >= self.popularity_threshold {
let new_ttl = match metadata.entry_type {
CacheEntryType::File => Duration::from_millis(
(self.config.timeouts.file_operation_ms as f64 * self.ttl_multiplier) as u64
),
CacheEntryType::Directory => Duration::from_millis(
(self.config.timeouts.dir_operation_ms as f64 * self.ttl_multiplier) as u64
),
_ => Duration::from_secs(60), // 1 minuto por defecto
};
metadata.update_expiry(new_ttl);
debug!("Extended TTL for popular entry: {}", path.display());
}
// Calcular tiempo ahorrado aproximado
let elapsed = start_time.elapsed().as_millis() as u64;
let estimated_io_time: u64 = 10; // Asumimos 10ms mínimo para operación de IO
let time_saved = estimated_io_time.saturating_sub(elapsed);
// Actualizar estadísticas
let mut stats = self.stats.write().await;
stats.hits += 1;
stats.time_saved_ms += time_saved;
debug!("Cache hit for: {}", path.display());
// Mantener también la cola LRU actualizada
self.update_lru(path.to_path_buf()).await;
// Clonar para retornar
return Some(metadata.clone());
}
// No encontrado en caché
let mut stats = self.stats.write().await;
stats.misses += 1;
debug!("Cache miss for: {}", path.display());
None
}
/// Actualiza la cola LRU
async fn update_lru(&self, path: PathBuf) {
let mut lru = self.lru_queue.write().await;
// Eliminar si ya existe
if let Some(pos) = lru.iter().position(|p| p == &path) {
lru.remove(pos);
}
// Agregar al final (más reciente)
lru.push_back(path);
}
/// Verifica si un archivo existe
#[allow(dead_code)]
pub async fn exists(&self, path: &Path) -> Option<bool> {
if let Some(metadata) = self.get_metadata(path).await {
return Some(metadata.exists);
}
None
}
/// Verifica si un path es un directorio
#[allow(dead_code)]
pub async fn is_dir(&self, path: &Path) -> Option<bool> {
if let Some(metadata) = self.get_metadata(path).await {
return Some(metadata.entry_type == CacheEntryType::Directory);
}
None
}
/// Verifica si un path es un archivo
pub async fn is_file(&self, path: &Path) -> Option<bool> {
if let Some(metadata) = self.get_metadata(path).await {
return Some(metadata.entry_type == CacheEntryType::File);
}
None
}
/// Obtiene el tamaño de un archivo
#[allow(dead_code)]
pub async fn get_size(&self, path: &Path) -> Option<u64> {
if let Some(metadata) = self.get_metadata(path).await {
return metadata.size;
}
None
}
/// Obtiene el tipo MIME de un archivo
#[allow(dead_code)]
pub async fn get_mime_type(&self, path: &Path) -> Option<String> {
if let Some(metadata) = self.get_metadata(path).await {
return metadata.mime_type;
}
None
}
/// Refresca los metadatos de un path
pub async fn refresh_metadata(&self, path: &Path) -> Result<FileMetadata, std::io::Error> {
// Realizar lectura real del sistema de archivos
let metadata = fs::metadata(path).await?;
// Determinar tipo de entrada
let entry_type = if metadata.is_dir() {
CacheEntryType::Directory
} else if metadata.is_file() {
CacheEntryType::File
} else {
CacheEntryType::Unknown
};
// Obtener tamaño para archivos
let size = if metadata.is_file() {
Some(metadata.len())
} else {
None
};
// Obtener tipo MIME para archivos
let mime_type = if metadata.is_file() {
Some(from_path(path).first_or_octet_stream().to_string())
} else {
None
};
// Obtener timestamps
let created_at = metadata.created()
.map(|time| time.duration_since(UNIX_EPOCH).unwrap_or_default().as_secs())
.ok();
let modified_at = metadata.modified()
.map(|time| time.duration_since(UNIX_EPOCH).unwrap_or_default().as_secs())
.ok();
// Determinar TTL apropiado
let ttl = if metadata.is_dir() {
Duration::from_millis(self.config.timeouts.dir_operation_ms)
} else {
Duration::from_millis(self.config.timeouts.file_operation_ms)
};
// Crear entrada de metadatos
let file_metadata = FileMetadata::new(
path.to_path_buf(),
true,
entry_type,
size,
mime_type,
created_at,
modified_at,
ttl,
);
// Actualizar caché
self.update_cache(file_metadata.clone()).await;
Ok(file_metadata)
}
/// Actualiza la caché con nuevos metadatos
pub async fn update_cache(&self, metadata: FileMetadata) {
// Evitar caché llena antes de insertar
self.ensure_capacity().await;
let path = metadata.path.clone();
// Insertar en caché
{
let mut cache = self.metadata_cache.write().await;
cache.insert(path.clone(), metadata);
// Actualizar estadísticas
let mut stats = self.stats.write().await;
stats.inserts += 1;
}
// Actualizar la cola LRU
self.update_lru(path).await;
}
/// Asegura que hay espacio en la caché
async fn ensure_capacity(&self) {
let cache_size = {
let cache = self.metadata_cache.read().await;
cache.len()
};
if cache_size >= self.max_entries {
self.evict_lru_entries(cache_size / 10).await; // Liberar 10%
}
}
/// Elimina entradas menos recientemente usadas
async fn evict_lru_entries(&self, count: usize) {
let mut paths_to_remove = Vec::with_capacity(count);
// Obtener entries a eliminar de la cola LRU
{
let mut lru = self.lru_queue.write().await;
for _ in 0..count {
if let Some(path) = lru.pop_front() {
paths_to_remove.push(path);
} else {
break;
}
}
}
// Eliminar de la caché principal
{
let mut cache = self.metadata_cache.write().await;
for path in paths_to_remove {
cache.remove(&path);
}
}
debug!("Evicted {} LRU entries from cache", count);
}
/// Invalidar una entrada específica de caché
pub async fn invalidate(&self, path: &Path) {
// Eliminar de la caché principal
{
let mut cache = self.metadata_cache.write().await;
cache.remove(path);
// Actualizar estadísticas
let mut stats = self.stats.write().await;
stats.invalidations += 1;
}
// Eliminar de la cola LRU
let path_buf = path.to_path_buf();
{
let mut lru = self.lru_queue.write().await;
if let Some(pos) = lru.iter().position(|p| p == &path_buf) {
lru.remove(pos);
}
}
debug!("Invalidated cache entry for: {}", path.display());
}
/// Invalidar recursivamente entradas bajo un directorio
pub async fn invalidate_directory(&self, dir_path: &Path) {
let dir_str = dir_path.to_string_lossy().to_string();
let mut paths_to_remove = Vec::new();
// Encontrar todos los paths que comienzan con el directorio
{
let cache = self.metadata_cache.read().await;
for path in cache.keys() {
let path_str = path.to_string_lossy().to_string();
if path_str.starts_with(&dir_str) {
paths_to_remove.push(path.clone());
}
}
}
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.invalidations += paths_to_remove.len();
}
// Eliminar cada path encontrado
for path in paths_to_remove {
self.invalidate(&path).await;
}
debug!("Invalidated directory and contents: {}", dir_path.display());
}
/// Obtener estadísticas actuales de la caché
pub async fn get_stats(&self) -> CacheStats {
let stats = self.stats.read().await;
stats.clone()
}
/// Limpia todas las entradas expiradas de la caché
pub async fn clear_expired(&self) {
let now = Instant::now();
let mut paths_to_remove = Vec::new();
// Encontrar entradas expiradas
{
let cache = self.metadata_cache.read().await;
for (path, metadata) in cache.iter() {
if now > metadata.expires_at {
paths_to_remove.push(path.clone());
}
}
}
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.expirations += paths_to_remove.len();
}
// Guardar la cantidad de entradas para el logging
let num_paths = paths_to_remove.len();
// Eliminar entradas expiradas
for path in paths_to_remove {
self.invalidate(&path).await;
}
debug!("Cleared {} expired entries from cache", num_paths);
}
/// Inicia el proceso de limpieza periódica
pub fn start_cleanup_task(cache: Arc<Self>) -> BoxFuture<'static, ()> {
Box::pin(async move {
let cleanup_interval = Duration::from_secs(60); // Cada minuto
loop {
// Esperar el intervalo
time::sleep(cleanup_interval).await;
// Limpiar entradas expiradas
cache.clear_expired().await;
// Registrar estadísticas
let stats = cache.get_stats().await;
let cache_size = {
let cache_map = cache.metadata_cache.read().await;
cache_map.len()
};
debug!(
"Cache stats: size={}, hits={}, misses={}, hit_ratio={:.2}%, time_saved={}ms",
cache_size,
stats.hits,
stats.misses,
if stats.hits + stats.misses > 0 {
(stats.hits as f64 * 100.0) / (stats.hits + stats.misses) as f64
} else {
0.0
},
stats.time_saved_ms
);
}
})
}
/// Precarga metadatos de directorios completos (útil para inicialización)
pub async fn preload_directory(&self, dir_path: &Path, recursive: bool, max_depth: usize) -> Result<usize, std::io::Error> {
self._preload_directory_internal(dir_path, recursive, max_depth, 0).await
}
/// Implementación interna de precarga con seguimiento de profundidad
async fn _preload_directory_internal(
&self,
dir_path: &Path,
recursive: bool,
max_depth: usize,
current_depth: usize
) -> Result<usize, std::io::Error> {
Box::pin(async move {
if current_depth > max_depth {
return Ok(0);
}
// Obtener entradas del directorio
let mut entries = fs::read_dir(dir_path).await?;
let mut count = 0;
// Procesar cada entrada
while let Some(entry) = entries.next_entry().await? {
let path = entry.path();
let metadata = fs::metadata(&path).await?;
// Refrescar metadatos de esta entrada
self.refresh_metadata(&path).await?;
count += 1;
// Recursivamente procesar subdirectorios si es necesario
if recursive && metadata.is_dir() {
// Box to break recursion
count += self._preload_directory_internal(
&path,
recursive,
max_depth,
current_depth + 1
).await?;
}
}
Ok(count)
}).await
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
#[tokio::test]
async fn test_cache_operations() {
// Crear directorio temporal para pruebas
let temp_dir = tempdir().unwrap();
let file_path = temp_dir.path().join("test_file.txt");
// Crear un archivo de prueba
let mut file = File::create(&file_path).await.unwrap();
file.write_all(b"test content").await.unwrap();
file.flush().await.unwrap();
drop(file);
// Crear caché
let config = AppConfig::default();
let cache = FileMetadataCache::new(config, 1000);
// Verificar miss inicial
assert!(cache.exists(&file_path).await.is_none());
// Refrescar y verificar hit
let metadata = cache.refresh_metadata(&file_path).await.unwrap();
assert_eq!(metadata.entry_type, CacheEntryType::File);
assert_eq!(metadata.size, Some(12)); // "test content" = 12 bytes
// Verificar que ahora existe en caché
assert_eq!(cache.exists(&file_path).await, Some(true));
assert_eq!(cache.is_file(&file_path).await, Some(true));
// Invalidar y verificar que ya no existe en caché
cache.invalidate(&file_path).await;
assert!(cache.exists(&file_path).await.is_none());
// Verificar estadísticas
let stats = cache.get_stats().await;
assert_eq!(stats.inserts, 1);
assert_eq!(stats.invalidations, 1);
assert!(stats.hits > 0);
}
#[tokio::test]
async fn test_directory_operations() {
// Crear estructura de directorios para pruebas
let temp_dir = tempdir().unwrap();
let sub_dir = temp_dir.path().join("subdir");
fs::create_dir(&sub_dir).await.unwrap();
let file1 = temp_dir.path().join("file1.txt");
let file2 = sub_dir.join("file2.txt");
File::create(&file1).await.unwrap();
File::create(&file2).await.unwrap();
// Crear caché
let config = AppConfig::default();
let cache = FileMetadataCache::new(config, 1000);
// Precargar directorio recursivamente
let count = cache.preload_directory(temp_dir.path(), true, 2).await.unwrap();
assert_eq!(count, 3); // dir, subdir, 2 files
// Verificar existencia en caché
assert_eq!(cache.is_dir(temp_dir.path()).await, Some(true));
assert_eq!(cache.is_dir(&sub_dir).await, Some(true));
assert_eq!(cache.is_file(&file1).await, Some(true));
assert_eq!(cache.is_file(&file2).await, Some(true));
// Invalidar directorio y contenido
cache.invalidate_directory(temp_dir.path()).await;
// Verificar que nada existe en caché
assert!(cache.exists(temp_dir.path()).await.is_none());
assert!(cache.exists(&sub_dir).await.is_none());
assert!(cache.exists(&file1).await.is_none());
assert!(cache.exists(&file2).await.is_none());
}
}
@@ -0,0 +1,643 @@
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::{Mutex, RwLock, Semaphore};
use tracing::{debug, error, info, warn};
use async_trait::async_trait;
use crate::domain::services::path_service::StoragePath;
use crate::infrastructure::services::id_mapping_service::{IdMappingService, IdMappingError};
use crate::common::errors::DomainError;
use crate::application::ports::outbound::IdMappingPort;
/// Tamaño máximo de entradas en el caché
const MAX_CACHE_SIZE: usize = 10_000;
/// Tiempo de vida del caché (en segundos)
const CACHE_TTL_SECONDS: u64 = 60 * 5; // 5 minutos
/// Optimizador para operaciones masivas de mapeo de IDs
pub struct IdMappingOptimizer {
/// Servicio base de mapeo de IDs
base_service: Arc<IdMappingService>,
/// Caché de ID por ruta (path -> id)
path_to_id_cache: RwLock<HashMap<String, (String, Instant)>>,
/// Caché de ruta por ID (id -> path)
id_to_path_cache: RwLock<HashMap<String, (String, Instant)>>,
/// Contador de hits
stats: RwLock<OptimizerStats>,
/// Semáforo para limitar operaciones de batch
batch_limiter: Semaphore,
/// Cola de batch pendientes
pending_batch: Mutex<BatchQueue>,
}
/// Estadísticas del optimizador
#[derive(Debug, Default, Clone)]
pub struct OptimizerStats {
/// Número total de consultas get_path_by_id
pub path_by_id_queries: usize,
/// Número de hits en caché get_path_by_id
pub path_by_id_hits: usize,
/// Número total de consultas get_or_create_id
pub get_id_queries: usize,
/// Número de hits en caché get_or_create_id
pub get_id_hits: usize,
/// Número de batch realizados
pub batch_operations: usize,
/// Número total de IDs procesados en batch
pub batch_items_processed: usize,
/// Último momento de limpieza de caché
pub last_cleanup: Option<Instant>,
}
/// Cola para operaciones batch
struct BatchQueue {
/// Rutas pendientes para obtener/crear ID
path_to_id_requests: HashSet<String>,
/// IDs pendientes para obtener ruta
id_to_path_requests: HashSet<String>,
}
impl Default for BatchQueue {
fn default() -> Self {
Self {
path_to_id_requests: HashSet::new(),
id_to_path_requests: HashSet::new(),
}
}
}
/// Resultado de una operación batch
struct BatchResult {
/// Mapeo de ruta a ID
path_to_id: HashMap<String, String>,
/// Mapeo de ID a ruta
id_to_path: HashMap<String, String>,
}
impl IdMappingOptimizer {
/// Crea un nuevo optimizador para el servicio de mapeo de IDs
pub fn new(base_service: Arc<IdMappingService>) -> Self {
Self {
base_service,
path_to_id_cache: RwLock::new(HashMap::with_capacity(1000)),
id_to_path_cache: RwLock::new(HashMap::with_capacity(1000)),
stats: RwLock::new(OptimizerStats::default()),
batch_limiter: Semaphore::new(2), // Limitar a 2 operaciones batch concurrentes
pending_batch: Mutex::new(BatchQueue::default()),
}
}
/// Obtiene estadísticas del optimizador
pub async fn get_stats(&self) -> OptimizerStats {
self.stats.read().await.clone()
}
/// Limpia entradas expiradas del caché
pub async fn cleanup_cache(&self) {
let now = Instant::now();
let ttl = Duration::from_secs(CACHE_TTL_SECONDS);
// Limpiar caché path_to_id
{
let mut cache = self.path_to_id_cache.write().await;
let initial_size = cache.len();
// Retener solo entradas no expiradas
cache.retain(|_, (_, timestamp)| {
now.duration_since(*timestamp) < ttl
});
let removed = initial_size - cache.len();
if removed > 0 {
debug!("Cleaned {} expired entries from path_to_id cache", removed);
}
}
// Limpiar caché id_to_path
{
let mut cache = self.id_to_path_cache.write().await;
let initial_size = cache.len();
// Retener solo entradas no expiradas
cache.retain(|_, (_, timestamp)| {
now.duration_since(*timestamp) < ttl
});
let removed = initial_size - cache.len();
if removed > 0 {
debug!("Cleaned {} expired entries from id_to_path cache", removed);
}
}
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.last_cleanup = Some(now);
}
}
/// Inicia tarea de limpieza periódica
pub fn start_cleanup_task(optimizer: Arc<Self>) {
tokio::spawn(async move {
let cleanup_interval = Duration::from_secs(CACHE_TTL_SECONDS / 2);
loop {
tokio::time::sleep(cleanup_interval).await;
optimizer.cleanup_cache().await;
// Loguear estadísticas periódicamente
let stats = optimizer.get_stats().await;
info!("ID Mapping Optimizer stats - Path queries: {}, hits: {} ({}%), ID queries: {}, hits: {} ({}%), Batch ops: {}, items: {}",
stats.path_by_id_queries,
stats.path_by_id_hits,
if stats.path_by_id_queries > 0 { stats.path_by_id_hits as f64 * 100.0 / stats.path_by_id_queries as f64 } else { 0.0 },
stats.get_id_queries,
stats.get_id_hits,
if stats.get_id_queries > 0 { stats.get_id_hits as f64 * 100.0 / stats.get_id_queries as f64 } else { 0.0 },
stats.batch_operations,
stats.batch_items_processed
);
}
});
}
/// Agrega una solicitud a la cola pendiente para procesamiento batch
async fn queue_path_to_id_request(&self, path: &StoragePath) -> Result<Option<String>, IdMappingError> {
let path_str = path.to_string();
// Verificar primero en el caché
{
let cache = self.path_to_id_cache.read().await;
if let Some((id, _)) = cache.get(&path_str) {
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.get_id_hits += 1;
}
return Ok(Some(id.clone()));
}
}
// Si no está en caché, agregar a la cola de batch
{
let mut batch_queue = self.pending_batch.lock().await;
batch_queue.path_to_id_requests.insert(path_str);
}
// No encontrado en caché, debe procesarse en batch
Ok(None)
}
/// Procesa las solicitudes pendientes en batch
async fn process_batch(&self) -> Result<BatchResult, IdMappingError> {
// Adquirir permiso para operación batch
let _permit = self.batch_limiter.acquire().await.unwrap();
// Obtener las solicitudes pendientes
let (path_requests, id_requests) = {
let mut batch_queue = self.pending_batch.lock().await;
let paths = std::mem::take(&mut batch_queue.path_to_id_requests);
let ids = std::mem::take(&mut batch_queue.id_to_path_requests);
(paths, ids)
};
// Crear resultados
let mut result = BatchResult {
path_to_id: HashMap::with_capacity(path_requests.len()),
id_to_path: HashMap::with_capacity(id_requests.len()),
};
// Procesar solicitudes path->id en batch
for path_str in path_requests {
let path = StoragePath::from_string(&path_str);
match self.base_service.get_or_create_id(&path).await {
Ok(id) => {
result.path_to_id.insert(path_str.clone(), id.clone());
result.id_to_path.insert(id, path_str);
},
Err(e) => {
error!("Error batch-processing path {}: {}", path_str, e);
// Continuar con las demás solicitudes
}
}
}
// Procesar solicitudes id->path en batch
for id in id_requests {
match self.base_service.get_path_by_id(&id).await {
Ok(path) => {
let path_str = path.to_string();
result.id_to_path.insert(id.clone(), path_str.clone());
result.path_to_id.insert(path_str, id);
},
Err(e) => {
error!("Error batch-processing ID {}: {}", id, e);
// Continuar con las demás solicitudes
}
}
}
// Actualizar caché con los resultados del batch
{
let mut path_cache = self.path_to_id_cache.write().await;
let mut id_cache = self.id_to_path_cache.write().await;
let now = Instant::now();
for (path, id) in &result.path_to_id {
path_cache.insert(path.clone(), (id.clone(), now));
}
for (id, path) in &result.id_to_path {
id_cache.insert(id.clone(), (path.clone(), now));
}
}
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.batch_operations += 1;
stats.batch_items_processed += result.path_to_id.len() + result.id_to_path.len();
}
// Guardar los cambios al disco en segundo plano
let service_clone = self.base_service.clone();
tokio::spawn(async move {
if let Err(e) = service_clone.save_pending_changes().await {
error!("Error saving ID mapping changes: {}", e);
}
});
Ok(result)
}
/// Fuerza el procesamiento de solicitudes pendientes si hay suficientes
async fn trigger_batch_if_needed(&self, min_batch_size: usize) -> Result<(), IdMappingError> {
// Verificar si hay suficientes solicitudes pendientes
let should_process = {
let batch_queue = self.pending_batch.lock().await;
batch_queue.path_to_id_requests.len() + batch_queue.id_to_path_requests.len() >= min_batch_size
};
// Procesar si es necesario
if should_process {
self.process_batch().await?;
}
Ok(())
}
/// Precargar un conjunto de rutas para obtener sus IDs en batch
#[allow(dead_code)]
pub async fn preload_paths(&self, paths: Vec<StoragePath>) -> Result<(), IdMappingError> {
// Solo proceder si hay rutas para cargar
if paths.is_empty() {
return Ok(());
}
// Rutas que debemos cargar (las que no están en caché)
let mut paths_to_load = Vec::new();
// Verificar primero el caché
{
let cache = self.path_to_id_cache.read().await;
for path in paths {
let path_str = path.to_string();
if !cache.contains_key(&path_str) {
paths_to_load.push(path_str);
}
}
}
// Si todos estaban en caché, terminar
if paths_to_load.is_empty() {
return Ok(());
}
// Agregar rutas a la cola para procesamiento batch
{
let mut batch_queue = self.pending_batch.lock().await;
for path in paths_to_load {
batch_queue.path_to_id_requests.insert(path);
}
}
// Ejecutar procesamiento batch inmediatamente
self.process_batch().await?;
Ok(())
}
/// Precargar un conjunto de IDs para obtener sus rutas en batch
#[allow(dead_code)]
pub async fn preload_ids(&self, ids: Vec<String>) -> Result<(), IdMappingError> {
// Solo proceder si hay IDs para cargar
if ids.is_empty() {
return Ok(());
}
// IDs que debemos cargar (los que no están en caché)
let mut ids_to_load = Vec::new();
// Verificar primero el caché
{
let cache = self.id_to_path_cache.read().await;
for id in ids {
if !cache.contains_key(&id) {
ids_to_load.push(id);
}
}
}
// Si todos estaban en caché, terminar
if ids_to_load.is_empty() {
return Ok(());
}
// Agregar IDs a la cola para procesamiento batch
{
let mut batch_queue = self.pending_batch.lock().await;
for id in ids_to_load {
batch_queue.id_to_path_requests.insert(id);
}
}
// Ejecutar procesamiento batch inmediatamente
self.process_batch().await?;
Ok(())
}
}
#[async_trait]
impl IdMappingPort for IdMappingOptimizer {
async fn get_or_create_id(&self, path: &StoragePath) -> Result<String, DomainError> {
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.get_id_queries += 1;
}
let path_str = path.to_string();
// Verificar primero en el caché
{
let cache = self.path_to_id_cache.read().await;
if let Some((id, _)) = cache.get(&path_str) {
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.get_id_hits += 1;
}
return Ok(id.clone());
}
}
// Si no está en caché, intentar agregar a cola de batch primero
let queued_result = self.queue_path_to_id_request(path).await?;
if let Some(id) = queued_result {
return Ok(id);
}
// Trigger batch processing if enough items accumulated
self.trigger_batch_if_needed(20).await?;
// Intentar obtener del servicio base
let id = self.base_service.get_or_create_id(path).await?;
// Actualizar caché con el nuevo ID
{
let mut path_cache = self.path_to_id_cache.write().await;
let mut id_cache = self.id_to_path_cache.write().await;
let now = Instant::now();
// Controlar tamaño del caché
if path_cache.len() >= MAX_CACHE_SIZE {
warn!("Path-to-ID cache size reached limit ({}), clearing oldest entries", MAX_CACHE_SIZE);
path_cache.clear();
}
if id_cache.len() >= MAX_CACHE_SIZE {
warn!("ID-to-path cache size reached limit ({}), clearing oldest entries", MAX_CACHE_SIZE);
id_cache.clear();
}
path_cache.insert(path_str.clone(), (id.clone(), now));
id_cache.insert(id.clone(), (path_str, now));
}
Ok(id)
}
async fn get_path_by_id(&self, id: &str) -> Result<StoragePath, DomainError> {
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.path_by_id_queries += 1;
}
// Verificar primero en el caché
{
let cache = self.id_to_path_cache.read().await;
if let Some((path_str, _)) = cache.get(id) {
// Actualizar estadísticas
{
let mut stats = self.stats.write().await;
stats.path_by_id_hits += 1;
}
return Ok(StoragePath::from_string(path_str));
}
}
// Obtener del servicio base
let path = self.base_service.get_path_by_id(id).await?;
// Actualizar caché
{
let mut id_cache = self.id_to_path_cache.write().await;
let mut path_cache = self.path_to_id_cache.write().await;
let now = Instant::now();
let path_str = path.to_string();
// Controlar tamaño del caché
if id_cache.len() >= MAX_CACHE_SIZE {
warn!("ID-to-path cache size reached limit ({}), clearing oldest entries", MAX_CACHE_SIZE);
id_cache.clear();
}
if path_cache.len() >= MAX_CACHE_SIZE {
warn!("Path-to-ID cache size reached limit ({}), clearing oldest entries", MAX_CACHE_SIZE);
path_cache.clear();
}
id_cache.insert(id.to_string(), (path_str.clone(), now));
path_cache.insert(path_str, (id.to_string(), now));
}
Ok(path)
}
async fn update_path(&self, id: &str, new_path: &StoragePath) -> Result<(), DomainError> {
// Invalidar caché para este ID
{
let mut id_cache = self.id_to_path_cache.write().await;
let mut path_cache = self.path_to_id_cache.write().await;
// Eliminar la entrada del ID
if let Some((old_path, _)) = id_cache.remove(id) {
path_cache.remove(&old_path);
}
}
// Actualizar en el servicio base
let result = self.base_service.update_path(id, new_path).await?;
// Actualizar caché con el nuevo mapeo
{
let mut id_cache = self.id_to_path_cache.write().await;
let mut path_cache = self.path_to_id_cache.write().await;
let now = Instant::now();
let path_str = new_path.to_string();
id_cache.insert(id.to_string(), (path_str.clone(), now));
path_cache.insert(path_str, (id.to_string(), now));
}
Ok(result)
}
async fn remove_id(&self, id: &str) -> Result<(), DomainError> {
// Invalidar caché para este ID
{
let mut id_cache = self.id_to_path_cache.write().await;
let mut path_cache = self.path_to_id_cache.write().await;
// Eliminar la entrada del ID
if let Some((path, _)) = id_cache.remove(id) {
path_cache.remove(&path);
}
}
// Eliminar en el servicio base
self.base_service.remove_id(id).await?;
Ok(())
}
async fn save_changes(&self) -> Result<(), DomainError> {
// Delegar al servicio base
self.base_service.save_changes().await?;
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
async fn create_test_service() -> (Arc<IdMappingService>, Arc<IdMappingOptimizer>) {
let temp_dir = tempdir().unwrap();
let map_path = temp_dir.path().join("id_map.json");
let base_service = Arc::new(IdMappingService::new(map_path).await.unwrap());
let optimizer = Arc::new(IdMappingOptimizer::new(base_service.clone()));
(base_service, optimizer)
}
#[tokio::test]
async fn test_basic_caching() {
let (_, optimizer) = create_test_service().await;
let path = StoragePath::from_string("/test/file.txt");
// Primera llamada debería usar el servicio base
let id = optimizer.get_or_create_id(&path).await.unwrap();
assert!(!id.is_empty(), "ID should not be empty");
// Segunda llamada debería usar caché
let id2 = optimizer.get_or_create_id(&path).await.unwrap();
assert_eq!(id, id2, "Same path should return same ID");
// Verificar estadísticas de caché
let stats = optimizer.get_stats().await;
assert_eq!(stats.get_id_queries, 2, "Should have 2 queries");
assert_eq!(stats.get_id_hits, 1, "Should have 1 hit");
}
#[tokio::test]
async fn test_batch_processing() {
let (_, optimizer) = create_test_service().await;
// Crear un lote de rutas
let mut paths = Vec::new();
for i in 0..50 {
paths.push(StoragePath::from_string(&format!("/test/batch/file{}.txt", i)));
}
// Precargar las rutas
optimizer.preload_paths(paths.clone()).await.unwrap();
// Verificar que todas están en caché
for path in &paths {
let id = optimizer.get_or_create_id(path).await.unwrap();
assert!(!id.is_empty(), "ID should be available for path");
}
// Verificar estadísticas
let stats = optimizer.get_stats().await;
assert_eq!(stats.batch_operations, 1, "Should have 1 batch operation");
assert!(stats.batch_items_processed >= 50, "Should have processed at least 50 items");
// Verificar que todas las consultas posteriores son hits en caché
assert_eq!(stats.get_id_hits, 50, "All subsequente queries should be cache hits");
}
#[tokio::test]
async fn test_cache_cleanup() {
let (_, optimizer) = create_test_service().await;
// Crear algunas entradas
let path = StoragePath::from_string("/test/cleanup.txt");
let id = optimizer.get_or_create_id(&path).await.unwrap();
// Verificar estadísticas iniciales
{
let stats = optimizer.get_stats().await;
assert_eq!(stats.get_id_queries, 1, "Should have 1 query");
assert_eq!(stats.get_id_hits, 0, "Should have 0 hits");
}
// Ejecutar limpieza (no debería eliminar nada todavía)
optimizer.cleanup_cache().await;
// Verificar que el caché sigue funcionando
let id2 = optimizer.get_or_create_id(&path).await.unwrap();
assert_eq!(id, id2, "Cache should still work after cleanup");
{
let stats = optimizer.get_stats().await;
assert_eq!(stats.get_id_hits, 1, "Should have 1 hit after cleanup");
}
}
}
@@ -0,0 +1,534 @@
use std::path::PathBuf;
use std::collections::HashMap;
use std::time::Duration;
use tokio::sync::{RwLock, Mutex};
use tokio::fs;
use tokio::time;
use uuid::Uuid;
use serde::{Serialize, Deserialize};
use async_trait::async_trait;
use crate::domain::services::path_service::StoragePath;
use crate::common::errors::{DomainError, ErrorKind, ErrorContext};
use crate::application::ports::outbound::IdMappingPort;
use crate::common::config::TimeoutConfig;
/// Error específico para el servicio de mapeo de IDs
#[derive(Debug, thiserror::Error)]
pub enum IdMappingError {
#[error("ID not found: {0}")]
NotFound(String),
#[error("IO error: {0}")]
IoError(#[from] std::io::Error),
#[error("Timeout error: {0}")]
Timeout(String),
#[error("Serialization error: {0}")]
SerializationError(#[from] serde_json::Error),
#[error("Other error: {0}")]
#[allow(dead_code)]
Other(String),
}
// Implementar conversión de IdMappingError a DomainError
impl From<IdMappingError> for DomainError {
fn from(err: IdMappingError) -> Self {
match err {
IdMappingError::NotFound(id) => DomainError::not_found("IdMapping", id),
IdMappingError::IoError(e) => DomainError::new(
ErrorKind::InternalError,
"IdMapping",
format!("IO error: {}", e)
).with_source(e),
IdMappingError::Timeout(msg) => DomainError::timeout(
"IdMapping",
format!("Timeout: {}", msg)
),
IdMappingError::SerializationError(e) => DomainError::new(
ErrorKind::InternalError,
"IdMapping",
format!("Serialization error: {}", e)
).with_source(e),
IdMappingError::Other(msg) => DomainError::new(
ErrorKind::InternalError,
"IdMapping",
format!("Other error: {}", msg)
),
}
}
}
/// Estructura para almacenar IDs mapeados a sus rutas
#[derive(Serialize, Deserialize, Debug, Default)]
struct IdMap {
path_to_id: HashMap<String, String>,
id_to_path: HashMap<String, String>, // Campo para búsqueda bidireccional eficiente
version: u32, // Versión para detectar cambios
}
/// Constantes para configuración
const SAVE_DEBOUNCE_MS: u64 = 300; // Tiempo para agrupar operaciones de guardado
/// Servicio para gestionar mapeos entre rutas y IDs únicos
pub struct IdMappingService {
map_path: PathBuf,
id_map: RwLock<IdMap>,
save_mutex: Mutex<()>, // Para evitar múltiples guardados concurrentes
timeouts: TimeoutConfig,
pending_save: RwLock<bool>, // Indica si hay cambios pendientes
}
impl IdMappingService {
/// Crea un nuevo servicio de mapeo de IDs
pub async fn new(map_path: PathBuf) -> Result<Self, DomainError> {
let timeouts = TimeoutConfig::default();
let id_map = Self::load_id_map(&map_path, &timeouts).await?;
Ok(Self {
map_path,
id_map: RwLock::new(id_map),
save_mutex: Mutex::new(()),
timeouts,
pending_save: RwLock::new(false),
})
}
/// Carga el mapa de IDs desde disco con manejo robusto de errores
async fn load_id_map(map_path: &PathBuf, timeouts: &TimeoutConfig) -> Result<IdMap, DomainError> {
if map_path.exists() {
// Intentar leer con timeout para evitar bloqueos indefinidos
let read_result = time::timeout(
timeouts.lock_timeout(),
fs::read_to_string(map_path)
).await
.with_context(|| format!("Timeout reading ID map from {}", map_path.display()))?;
let content = read_result.with_context(|| format!("Failed to read ID map from {}", map_path.display()))?;
// Parsear el JSON
match serde_json::from_str::<IdMap>(&content) {
Ok(mut map) => {
// Reconstruir el mapa inverso si es necesario
if map.id_to_path.is_empty() && !map.path_to_id.is_empty() {
let mut rebuild_count = 0;
for (path, id) in &map.path_to_id {
map.id_to_path.insert(id.clone(), path.clone());
rebuild_count += 1;
}
tracing::info!("Rebuilt inverse mapping with {} entries", rebuild_count);
}
tracing::info!("Loaded ID map with {} entries (version: {})",
map.path_to_id.len(), map.version);
return Ok(map);
},
Err(e) => {
tracing::error!("Error parsing ID map: {}", e);
// Intentar hacer un respaldo del archivo corrupto
let backup_path = map_path.with_extension("json.bak");
if let Err(copy_err) = tokio::fs::copy(map_path, &backup_path).await {
tracing::error!("Failed to backup corrupted map file: {}", copy_err);
} else {
tracing::info!("Backed up corrupted ID map to {}", backup_path.display());
}
return Err(DomainError::new(
ErrorKind::InternalError,
"IdMapping",
format!("Error parsing ID map: {}", e)
).with_source(e));
}
}
}
// Devolver un mapa vacío si el archivo no existe
tracing::info!("No existing ID map found, creating new empty map");
Ok(IdMap {
path_to_id: HashMap::new(),
id_to_path: HashMap::new(),
version: 1, // Iniciar con versión 1
})
}
/// Guarda el mapa de IDs en disco de manera segura
async fn save_id_map(&self) -> Result<(), DomainError> {
// Adquirir bloqueo exclusivo para salvar
let _lock = time::timeout(
self.timeouts.lock_timeout(),
self.save_mutex.lock()
).await
.with_context(|| "Timeout acquiring save lock for ID mapping")?;
// Crear JSON con el lock de lectura para minimizar el tiempo de bloqueo
let json = {
let mut map = time::timeout(
self.timeouts.lock_timeout(),
self.id_map.write()
).await
.with_context(|| "Timeout acquiring write lock for ID mapping")?;
// Incrementar versión sólo si hay cambios por guardar
let pending = *self.pending_save.read().await;
if pending {
map.version += 1;
tracing::debug!("Incrementing ID map version to {}", map.version);
}
// Use serde with reasonably safe defaults
serde_json::to_string_pretty(&*map)
.with_context(|| "Failed to serialize ID map to JSON")?
};
// Escribir a un archivo temporal primero para evitar corrupción
let temp_path = self.map_path.with_extension("json.tmp");
fs::write(&temp_path, &json).await
.with_context(|| format!("Failed to write temporary ID map to {}", temp_path.display()))?;
// Realizar el rename atómico
fs::rename(&temp_path, &self.map_path).await
.with_context(|| format!("Failed to rename temporary ID map to {}", self.map_path.display()))?;
// Resetear flag de pendientes
{
let mut pending = self.pending_save.write().await;
*pending = false;
}
tracing::info!("Saved ID map successfully to {}", self.map_path.display());
Ok(())
}
/// Genera un ID único
fn generate_id(&self) -> String {
Uuid::new_v4().to_string()
}
/// Marca cambios como pendientes
async fn mark_pending(&self) {
let mut pending = self.pending_save.write().await;
*pending = true;
}
/// Obtiene el ID para una ruta o genera uno nuevo si no existe
pub async fn get_or_create_id(&self, path: &StoragePath) -> Result<String, IdMappingError> {
let path_str = path.to_string();
// Primer intento con lock de lectura (más eficiente)
{
let read_result = match time::timeout(
self.timeouts.lock_timeout(),
self.id_map.read()
).await {
Ok(guard) => guard,
Err(_) => return Err(IdMappingError::Timeout("Timeout acquiring read lock for ID mapping".to_string())),
};
if let Some(id) = read_result.path_to_id.get(&path_str) {
return Ok(id.clone());
}
}
// Si no se encuentra, adquirir lock de escritura
let write_result = match time::timeout(
self.timeouts.lock_timeout(),
self.id_map.write()
).await {
Ok(guard) => guard,
Err(_) => return Err(IdMappingError::Timeout("Timeout acquiring write lock for ID mapping".to_string())),
};
let mut map = write_result;
// Verificar nuevamente (podría haberse agregado mientras esperábamos el lock)
if let Some(id) = map.path_to_id.get(&path_str) {
return Ok(id.clone());
}
// Generar un nuevo ID y almacenarlo
let id = self.generate_id();
map.path_to_id.insert(path_str.clone(), id.clone());
map.id_to_path.insert(id.clone(), path_str);
// Marcar como pendiente para guardar
drop(map); // Liberar el write lock antes de adquirir otro
self.mark_pending().await;
tracing::debug!("Created new ID mapping: {} -> {}", path.to_string(), id);
Ok(id)
}
/// Obtiene una ruta por su ID con manejo de timeout
pub async fn get_path_by_id(&self, id: &str) -> Result<StoragePath, IdMappingError> {
let read_result = match time::timeout(
self.timeouts.lock_timeout(),
self.id_map.read()
).await {
Ok(guard) => guard,
Err(_) => return Err(IdMappingError::Timeout("Timeout acquiring read lock for ID lookup".to_string())),
};
if let Some(path_str) = read_result.id_to_path.get(id) {
return Ok(StoragePath::from_string(path_str));
}
Err(IdMappingError::NotFound(id.to_string()))
}
/// Actualiza el mapeo de un ID existente a una nueva ruta
pub async fn update_path(&self, id: &str, new_path: &StoragePath) -> Result<(), IdMappingError> {
let write_result = match time::timeout(
self.timeouts.lock_timeout(),
self.id_map.write()
).await {
Ok(guard) => guard,
Err(_) => return Err(IdMappingError::Timeout("Timeout acquiring write lock for ID update".to_string())),
};
let mut map = write_result;
// Buscar la ruta anterior para eliminarla
if let Some(old_path) = map.id_to_path.get(id).cloned() {
map.path_to_id.remove(&old_path);
// Registrar la nueva ruta
let new_path_str = new_path.to_string();
map.path_to_id.insert(new_path_str.clone(), id.to_string());
map.id_to_path.insert(id.to_string(), new_path_str);
// Marcar como pendiente
drop(map); // Liberar el write lock antes de adquirir otro
self.mark_pending().await;
tracing::debug!("Updated path mapping for ID {}: {} -> {}",
id, old_path, new_path.to_string());
Ok(())
} else {
Err(IdMappingError::NotFound(id.to_string()))
}
}
/// Elimina un ID del mapa
pub async fn remove_id(&self, id: &str) -> Result<(), IdMappingError> {
let write_result = match time::timeout(
self.timeouts.lock_timeout(),
self.id_map.write()
).await {
Ok(guard) => guard,
Err(_) => return Err(IdMappingError::Timeout("Timeout acquiring write lock for ID removal".to_string())),
};
let mut map = write_result;
// Buscar la ruta para eliminarla
if let Some(path) = map.id_to_path.remove(id) {
map.path_to_id.remove(&path);
// Marcar como pendiente
drop(map); // Liberar el write lock antes de adquirir otro
self.mark_pending().await;
tracing::debug!("Removed ID mapping: {} -> {}", id, path);
Ok(())
} else {
Err(IdMappingError::NotFound(id.to_string()))
}
}
/// Guarda cambios pendientes al disco
pub async fn save_pending_changes(&self) -> Result<(), IdMappingError> {
// Verificar si hay cambios pendientes
{
let pending = self.pending_save.read().await;
if !*pending {
return Ok(());
}
}
// Implementar debounce para agrupación de guardados
let map_path = self.map_path.clone();
let self_clone = self.clone();
tokio::spawn(async move {
// Esperar un poco para permitir la agrupación de operaciones
time::sleep(Duration::from_millis(SAVE_DEBOUNCE_MS)).await;
if let Err(e) = self_clone.save_id_map().await {
tracing::error!("Failed to save ID map to {}: {}", map_path.display(), e);
}
});
Ok(())
}
}
#[async_trait]
impl IdMappingPort for IdMappingService {
/// Obtiene el ID para una ruta o genera uno nuevo si no existe
async fn get_or_create_id(&self, path: &StoragePath) -> Result<String, DomainError> {
self.get_or_create_id(path).await
.with_context(|| format!("Failed to get or create ID for path: {}", path.to_string()))
}
/// Obtiene una ruta por su ID con manejo de timeout
async fn get_path_by_id(&self, id: &str) -> Result<StoragePath, DomainError> {
self.get_path_by_id(id).await
.with_context(|| format!("Failed to get path for ID: {}", id))
}
/// Actualiza el mapeo de un ID existente a una nueva ruta
async fn update_path(&self, id: &str, new_path: &StoragePath) -> Result<(), DomainError> {
self.update_path(id, new_path).await
.with_context(|| format!("Failed to update path for ID: {} to {}", id, new_path.to_string()))
}
/// Elimina un ID del mapa
async fn remove_id(&self, id: &str) -> Result<(), DomainError> {
self.remove_id(id).await
.with_context(|| format!("Failed to remove ID: {}", id))
}
/// Guarda cambios pendientes al disco
async fn save_changes(&self) -> Result<(), DomainError> {
self.save_pending_changes().await
.with_context(|| "Failed to save pending ID mapping changes")
}
}
// Implementar Clone para poder usar en tokio::spawn
/// Synchronous helper for contexts where we can't use async
impl IdMappingService {
/// Create a new service synchronously (only for stubs and initialization)
#[allow(dead_code)]
pub fn new_sync(map_path: PathBuf) -> Self {
// Create a minimal implementation for initialization purposes
Self {
map_path,
id_map: RwLock::new(IdMap::default()),
save_mutex: Mutex::new(()),
timeouts: TimeoutConfig::default(),
pending_save: RwLock::new(false),
}
}
}
impl Clone for IdMappingService {
fn clone(&self) -> Self {
// No podemos clonar directamente los RwLock/Mutex,
// pero podemos crear nuevas instancias que apunten al mismo Arc interno
// Sin embargo, en este caso simplemente necesitamos la map_path
Self {
map_path: self.map_path.clone(),
id_map: RwLock::new(IdMap::default()), // Esto no se usa en el task asíncrono
save_mutex: Mutex::new(()), // Esto tampoco
timeouts: self.timeouts.clone(),
pending_save: RwLock::new(false),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
#[tokio::test]
async fn test_get_or_create_id() {
let temp_dir = tempdir().unwrap();
let map_path = temp_dir.path().join("id_map.json");
let service = IdMappingService::new(map_path).await.unwrap();
let path = StoragePath::from_string("/test/file.txt");
let id = service.get_or_create_id(&path).await.unwrap();
assert!(!id.is_empty(), "ID should not be empty");
// Verificar que el mismo ID se devuelve para la misma ruta
let id2 = service.get_or_create_id(&path).await.unwrap();
assert_eq!(id, id2, "Same path should return same ID");
}
#[tokio::test]
async fn test_update_path() {
let temp_dir = tempdir().unwrap();
let map_path = temp_dir.path().join("id_map.json");
let service = IdMappingService::new(map_path).await.unwrap();
let old_path = StoragePath::from_string("/test/old.txt");
let id = service.get_or_create_id(&old_path).await.unwrap();
let new_path = StoragePath::from_string("/test/new.txt");
service.update_path(&id, &new_path).await.unwrap();
let retrieved_path = service.get_path_by_id(&id).await.unwrap();
assert_eq!(retrieved_path, new_path, "Path should be updated");
}
#[tokio::test]
async fn test_save_and_load() {
let temp_dir = tempdir().unwrap();
let map_path = temp_dir.path().join("id_map.json");
// Crear y poblar el servicio
let service = IdMappingService::new(map_path.clone()).await.unwrap();
let path1 = StoragePath::from_string("/test/file1.txt");
let path2 = StoragePath::from_string("/test/file2.txt");
let id1 = service.get_or_create_id(&path1).await.unwrap();
let id2 = service.get_or_create_id(&path2).await.unwrap();
// Guardar cambios
service.save_pending_changes().await.unwrap();
// Esperar para asegurar que el guardado asíncrono termine
tokio::time::sleep(Duration::from_millis(500)).await;
// Crear un nuevo servicio que debería cargar el mismo mapa
let service2 = IdMappingService::new(map_path).await.unwrap();
// Verificar que los IDs coinciden
let loaded_id1 = service2.get_or_create_id(&path1).await.unwrap();
let loaded_id2 = service2.get_or_create_id(&path2).await.unwrap();
assert_eq!(id1, loaded_id1, "ID1 should be preserved");
assert_eq!(id2, loaded_id2, "ID2 should be preserved");
}
#[tokio::test]
async fn test_concurrent_operations() {
use futures::future::join_all;
let temp_dir = tempdir().unwrap();
let map_path = temp_dir.path().join("id_map.json");
let service = std::sync::Arc::new(IdMappingService::new(map_path).await.unwrap());
// Crear múltiples tareas que intentan acceder simultáneamente
let mut tasks = Vec::new();
for i in 0..100 {
let path = StoragePath::from_string(&format!("/test/concurrent/file{}.txt", i));
let service_clone = service.clone();
tasks.push(tokio::spawn(async move {
service_clone.get_or_create_id(&path).await
}));
}
// Esperar a que todas terminen
let results = join_all(tasks).await;
// Verificar que todas tuvieron éxito
for result in results {
assert!(result.unwrap().is_ok(), "Concurrent operations should succeed");
}
// Guardar cambios
service.save_pending_changes().await.unwrap();
}
}
+6
View File
@@ -1 +1,7 @@
pub mod file_system_i18n_service;
pub mod id_mapping_service;
pub mod id_mapping_optimizer;
pub mod cache_manager;
pub mod file_metadata_cache;
pub mod compression_service;
pub mod buffer_pool;
@@ -0,0 +1,410 @@
use std::sync::Arc;
use axum::{
extract::{State, Json},
response::IntoResponse,
http::StatusCode,
};
use serde::{Deserialize, Serialize};
use crate::application::services::batch_operations::{
BatchOperationService, BatchResult, BatchStats
};
use crate::application::dtos::file_dto::FileDto;
use crate::application::dtos::folder_dto::FolderDto;
use crate::interfaces::api::handlers::ApiResult;
/// Estado compartido para el handler de batch
#[derive(Clone)]
pub struct BatchHandlerState {
pub batch_service: Arc<BatchOperationService>,
}
/// DTO para las solicitudes de operaciones en lote de archivos
#[derive(Debug, Deserialize)]
pub struct BatchFileOperationRequest {
/// IDs de los archivos a procesar
pub file_ids: Vec<String>,
/// ID de la carpeta destino (opcional)
#[serde(skip_serializing_if = "Option::is_none")]
pub target_folder_id: Option<String>,
}
/// DTO para las solicitudes de operaciones en lote de carpetas
#[derive(Debug, Deserialize)]
pub struct BatchFolderOperationRequest {
/// IDs de las carpetas a procesar
pub folder_ids: Vec<String>,
/// Si la operación debe ser recursiva
#[serde(default)]
pub recursive: bool,
/// ID de la carpeta destino (opcional)
#[serde(skip_serializing_if = "Option::is_none")]
#[allow(dead_code)]
pub target_folder_id: Option<String>,
}
/// DTO para las solicitudes de creación en lote de carpetas
#[derive(Debug, Deserialize)]
pub struct BatchCreateFoldersRequest {
/// Detalles de las carpetas a crear
pub folders: Vec<CreateFolderDetail>,
}
/// Detalle para creación de una carpeta
#[derive(Debug, Deserialize)]
pub struct CreateFolderDetail {
/// Nombre de la carpeta
pub name: String,
/// ID de la carpeta padre (opcional)
#[serde(skip_serializing_if = "Option::is_none")]
pub parent_id: Option<String>,
}
/// DTO para los resultados de operaciones en lote
#[derive(Debug, Serialize)]
pub struct BatchOperationResponse<T> {
/// Entidades procesadas exitosamente
pub successful: Vec<T>,
/// Operaciones fallidas con sus mensajes de error
pub failed: Vec<FailedOperation>,
/// Estadísticas de la operación
pub stats: BatchOperationStats,
}
/// Operación fallida en un lote
#[derive(Debug, Serialize)]
pub struct FailedOperation {
/// Identificador de la entidad que falló
pub id: String,
/// Mensaje de error
pub error: String,
}
/// Estadísticas de una operación por lotes
#[derive(Debug, Serialize)]
pub struct BatchOperationStats {
/// Número total de operaciones
pub total: usize,
/// Número de operaciones exitosas
pub successful: usize,
/// Número de operaciones fallidas
pub failed: usize,
/// Tiempo total de ejecución en milisegundos
pub execution_time_ms: u128,
}
/// Convierte BatchStats del dominio a DTO
impl From<BatchStats> for BatchOperationStats {
fn from(stats: BatchStats) -> Self {
Self {
total: stats.total,
successful: stats.successful,
failed: stats.failed,
execution_time_ms: stats.execution_time_ms,
}
}
}
/// Convierte BatchResult<T> del dominio a DTO
impl<T, U> From<BatchResult<T>> for BatchOperationResponse<U>
where
U: From<T>,
{
fn from(result: BatchResult<T>) -> Self {
let successful = result.successful.into_iter().map(U::from).collect();
let failed = result.failed.into_iter()
.map(|(id, error)| FailedOperation { id, error })
.collect();
Self {
successful,
failed,
stats: result.stats.into(),
}
}
}
/// Handler para mover múltiples archivos en lote
pub async fn move_files_batch(
State(state): State<BatchHandlerState>,
Json(request): Json<BatchFileOperationRequest>,
) -> ApiResult<impl IntoResponse> {
// Verificar que hay archivos para procesar
if request.file_ids.is_empty() {
return Ok((
StatusCode::BAD_REQUEST,
Json(serde_json::json!({
"error": "No file IDs provided"
}))
).into_response());
}
// Ejecutar operación de lote
let result = state.batch_service
.move_files(request.file_ids, request.target_folder_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Convertir resultado a DTO
let response: BatchOperationResponse<FileDto> = result.into();
// Determinar código de estado basado en los resultados
let status_code = if response.stats.failed > 0 {
if response.stats.successful > 0 {
StatusCode::PARTIAL_CONTENT // Algunas operaciones exitosas, otras fallidas
} else {
StatusCode::BAD_REQUEST // Todas fallaron
}
} else {
StatusCode::OK // Todas exitosas
};
Ok((status_code, Json(response)).into_response())
}
/// Handler para copiar múltiples archivos en lote
pub async fn copy_files_batch(
State(state): State<BatchHandlerState>,
Json(request): Json<BatchFileOperationRequest>,
) -> ApiResult<impl IntoResponse> {
// Verificar que hay archivos para procesar
if request.file_ids.is_empty() {
return Ok((
StatusCode::BAD_REQUEST,
Json(serde_json::json!({
"error": "No file IDs provided"
}))
).into_response());
}
// Ejecutar operación de lote
let result = state.batch_service
.copy_files(request.file_ids, request.target_folder_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Convertir resultado a DTO
let response: BatchOperationResponse<FileDto> = result.into();
// Determinar código de estado basado en los resultados
let status_code = if response.stats.failed > 0 {
if response.stats.successful > 0 {
StatusCode::PARTIAL_CONTENT // Algunas operaciones exitosas, otras fallidas
} else {
StatusCode::BAD_REQUEST // Todas fallaron
}
} else {
StatusCode::OK // Todas exitosas
};
Ok((status_code, Json(response)).into_response())
}
/// Handler para eliminar múltiples archivos en lote
pub async fn delete_files_batch(
State(state): State<BatchHandlerState>,
Json(request): Json<BatchFileOperationRequest>,
) -> ApiResult<impl IntoResponse> {
// Verificar que hay archivos para procesar
if request.file_ids.is_empty() {
return Ok((
StatusCode::BAD_REQUEST,
Json(serde_json::json!({
"error": "No file IDs provided"
}))
).into_response());
}
// Ejecutar operación de lote
let result = state.batch_service
.delete_files(request.file_ids)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Crear respuesta personalizada para IDs de string
let response = BatchOperationResponse {
successful: result.successful,
failed: result.failed.into_iter()
.map(|(id, error)| FailedOperation { id, error })
.collect(),
stats: result.stats.into(),
};
// Determinar código de estado basado en los resultados
let status_code = if response.stats.failed > 0 {
if response.stats.successful > 0 {
StatusCode::PARTIAL_CONTENT // Algunas operaciones exitosas, otras fallidas
} else {
StatusCode::BAD_REQUEST // Todas fallaron
}
} else {
StatusCode::OK // Todas exitosas
};
Ok((status_code, Json(response)).into_response())
}
/// Handler para eliminar múltiples carpetas en lote
pub async fn delete_folders_batch(
State(state): State<BatchHandlerState>,
Json(request): Json<BatchFolderOperationRequest>,
) -> ApiResult<impl IntoResponse> {
// Verificar que hay carpetas para procesar
if request.folder_ids.is_empty() {
return Ok((
StatusCode::BAD_REQUEST,
Json(serde_json::json!({
"error": "No folder IDs provided"
}))
).into_response());
}
// Ejecutar operación de lote
let result = state.batch_service
.delete_folders(request.folder_ids, request.recursive)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Crear respuesta personalizada para IDs de string
let response = BatchOperationResponse {
successful: result.successful,
failed: result.failed.into_iter()
.map(|(id, error)| FailedOperation { id, error })
.collect(),
stats: result.stats.into(),
};
// Determinar código de estado basado en los resultados
let status_code = if response.stats.failed > 0 {
if response.stats.successful > 0 {
StatusCode::PARTIAL_CONTENT // Algunas operaciones exitosas, otras fallidas
} else {
StatusCode::BAD_REQUEST // Todas fallaron
}
} else {
StatusCode::OK // Todas exitosas
};
Ok((status_code, Json(response)).into_response())
}
/// Handler para crear múltiples carpetas en lote
pub async fn create_folders_batch(
State(state): State<BatchHandlerState>,
Json(request): Json<BatchCreateFoldersRequest>,
) -> ApiResult<impl IntoResponse> {
// Verificar que hay carpetas para procesar
if request.folders.is_empty() {
return Ok((
StatusCode::BAD_REQUEST,
Json(serde_json::json!({
"error": "No folders provided"
}))
).into_response());
}
// Transformar el formato para el servicio
let folders = request.folders
.into_iter()
.map(|detail| (detail.name, detail.parent_id))
.collect();
// Ejecutar operación de lote
let result = state.batch_service
.create_folders(folders)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Convertir resultado a DTO
let response: BatchOperationResponse<FolderDto> = result.into();
// Determinar código de estado basado en los resultados
let status_code = if response.stats.failed > 0 {
if response.stats.successful > 0 {
StatusCode::PARTIAL_CONTENT // Algunas operaciones exitosas, otras fallidas
} else {
StatusCode::BAD_REQUEST // Todas fallaron
}
} else {
StatusCode::CREATED // Todas exitosas
};
Ok((status_code, Json(response)).into_response())
}
/// Handler para obtener múltiples archivos en lote
pub async fn get_files_batch(
State(state): State<BatchHandlerState>,
Json(request): Json<BatchFileOperationRequest>,
) -> ApiResult<impl IntoResponse> {
// Verificar que hay archivos para procesar
if request.file_ids.is_empty() {
return Ok((
StatusCode::BAD_REQUEST,
Json(serde_json::json!({
"error": "No file IDs provided"
}))
).into_response());
}
// Ejecutar operación de lote
let result = state.batch_service
.get_multiple_files(request.file_ids)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Convertir resultado a DTO
let response: BatchOperationResponse<FileDto> = result.into();
// Determinar código de estado basado en los resultados
let status_code = if response.stats.failed > 0 {
if response.stats.successful > 0 {
StatusCode::PARTIAL_CONTENT // Algunas operaciones exitosas, otras fallidas
} else {
StatusCode::BAD_REQUEST // Todas fallaron
}
} else {
StatusCode::OK // Todas exitosas
};
Ok((status_code, Json(response)).into_response())
}
/// Handler para obtener múltiples carpetas en lote
pub async fn get_folders_batch(
State(state): State<BatchHandlerState>,
Json(request): Json<BatchFolderOperationRequest>,
) -> ApiResult<impl IntoResponse> {
// Verificar que hay carpetas para procesar
if request.folder_ids.is_empty() {
return Ok((
StatusCode::BAD_REQUEST,
Json(serde_json::json!({
"error": "No folder IDs provided"
}))
).into_response());
}
// Ejecutar operación de lote
let result = state.batch_service
.get_multiple_folders(request.folder_ids)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Convertir resultado a DTO
let response: BatchOperationResponse<FolderDto> = result.into();
// Determinar código de estado basado en los resultados
let status_code = if response.stats.failed > 0 {
if response.stats.successful > 0 {
StatusCode::PARTIAL_CONTENT // Algunas operaciones exitosas, otras fallidas
} else {
StatusCode::BAD_REQUEST // Todas fallaron
}
} else {
StatusCode::OK // Todas exitosas
};
Ok((status_code, Json(response)).into_response())
}
+271 -24
View File
@@ -1,20 +1,55 @@
use std::sync::Arc;
use axum::{
extract::{Path, State, Multipart},
http::{StatusCode, header},
extract::{Path, State, Multipart, Query},
http::{StatusCode, header, HeaderName, HeaderValue, Response},
response::IntoResponse,
Json,
};
use serde::Deserialize;
use std::collections::HashMap;
use futures::Stream;
use std::task::{Context, Poll};
use std::pin::Pin;
use crate::application::services::file_service::FileService;
use crate::domain::repositories::file_repository::FileRepositoryError;
use crate::application::services::file_service::{FileService, FileServiceError};
use crate::infrastructure::services::compression_service::{
CompressionService, GzipCompressionService, CompressionLevel
};
type AppState = Arc<FileService>;
/// Handler for file-related API endpoints
pub struct FileHandler;
// Simpler approach to make streams Unpin - use Pin<Box<dyn Stream>> directly
struct BoxedStream<T> {
inner: Pin<Box<dyn Stream<Item = T> + Send + 'static>>,
}
impl<T> Stream for BoxedStream<T> {
type Item = T;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// Accessing the field directly is safe because BoxedStream is not a structural pinning type
unsafe { self.get_unchecked_mut().inner.as_mut().poll_next(cx) }
}
}
// This is safe because BoxedStream's inner field is already Pin<Box<dyn Stream>>
impl<T> Unpin for BoxedStream<T> {}
impl<T> BoxedStream<T> {
#[allow(dead_code)]
fn new<S>(stream: S) -> Self
where
S: Stream<Item = T> + Send + 'static,
{
BoxedStream {
inner: Box::pin(stream),
}
}
}
impl FileHandler {
/// Uploads a file
pub async fn upload_file(
@@ -49,8 +84,8 @@ impl FileHandler {
Ok(file) => (StatusCode::CREATED, Json(file)).into_response(),
Err(err) => {
let status = match &err {
FileRepositoryError::AlreadyExists(_) => StatusCode::CONFLICT,
FileRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
FileServiceError::Conflict(_) => StatusCode::CONFLICT,
FileServiceError::NotFound(_) => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -66,28 +101,240 @@ impl FileHandler {
}
}
/// Downloads a file
/// Downloads a file with optional compression
pub async fn download_file(
State(service): State<AppState>,
Path(id): Path<String>,
Query(params): Query<HashMap<String, String>>,
) -> impl IntoResponse {
// Get file info and content
let file_result = service.get_file(&id).await;
let content_result = service.get_file_content(&id).await;
// Initialize compression service
let compression_service = GzipCompressionService::new();
match (file_result, content_result) {
(Ok(file), Ok(content)) => {
// Create response with proper headers
let headers = [
(header::CONTENT_TYPE, file.mime_type),
(header::CONTENT_DISPOSITION, format!("attachment; filename=\"{}\"", file.name)),
];
// Check if compression is explicitly requested or rejected
let compression_param = params.get("compress").map(|v| v.as_str());
let force_compress = compression_param == Some("true") || compression_param == Some("1");
let force_no_compress = compression_param == Some("false") || compression_param == Some("0");
(StatusCode::OK, headers, content).into_response()
// Determine compression level from query params
let compression_level = match params.get("compression_level").map(|v| v.as_str()) {
Some("none") => CompressionLevel::None,
Some("fast") => CompressionLevel::Fast,
Some("best") => CompressionLevel::Best,
_ => CompressionLevel::Default, // Default or unrecognized
};
// Get file info first to check it exists and get metadata
match service.get_file(&id).await {
Ok(file) => {
// Determine if we should compress based on file type and size
let should_compress = if force_no_compress {
false
} else if force_compress {
true
} else {
compression_service.should_compress(&file.mime_type, file.size)
};
// Log compression decision for debugging
tracing::debug!(
"Download file: name={}, size={}KB, mime={}, compress={}",
file.name, file.size / 1024, file.mime_type, should_compress
);
// For large files, use streaming response with potential compression
if file.size > 10 * 1024 * 1024 { // 10MB threshold for streaming
match service.get_file_content(&id).await {
Ok(content) => {
// Create base headers
let mut headers = HashMap::new();
headers.insert(
header::CONTENT_DISPOSITION.to_string(),
format!("attachment; filename=\"{}\"", file.name)
);
if should_compress {
// Add content-encoding header for compressed response
headers.insert(header::CONTENT_ENCODING.to_string(), "gzip".to_string());
headers.insert(header::CONTENT_TYPE.to_string(), file.mime_type.clone());
headers.insert(header::VARY.to_string(), "Accept-Encoding".to_string());
// Compress the content
match compression_service.compress_data(&content, compression_level).await {
Ok(compressed_content) => {
tracing::debug!(
"Compressed file: {} from {}KB to {}KB (ratio: {:.2})",
file.name,
content.len() / 1024,
compressed_content.len() / 1024,
content.len() as f64 / compressed_content.len() as f64
);
// Build a custom response with headers and body
let mut response = Response::builder()
.status(StatusCode::OK)
.body(axum::body::Body::from(compressed_content))
.unwrap();
// Add headers to response
for (name, value) in headers {
response.headers_mut().insert(
HeaderName::from_bytes(name.as_bytes()).unwrap(),
HeaderValue::from_str(&value).unwrap()
);
}
response
},
Err(e) => {
tracing::warn!("Compression failed, sending uncompressed: {}", e);
// Fall back to uncompressed
headers.insert(header::CONTENT_TYPE.to_string(), file.mime_type.clone());
// Build a custom response with headers and body
let mut response = Response::builder()
.status(StatusCode::OK)
.body(axum::body::Body::from(content))
.unwrap();
// Add headers to response
for (name, value) in headers {
response.headers_mut().insert(
HeaderName::from_bytes(name.as_bytes()).unwrap(),
HeaderValue::from_str(&value).unwrap()
);
}
response
}
}
} else {
// No compression, return as-is
headers.insert(header::CONTENT_TYPE.to_string(), file.mime_type.clone());
// Build a custom response with headers and body
let mut response = Response::builder()
.status(StatusCode::OK)
.body(axum::body::Body::from(content))
.unwrap();
// Add headers to response
for (name, value) in headers {
response.headers_mut().insert(
HeaderName::from_bytes(name.as_bytes()).unwrap(),
HeaderValue::from_str(&value).unwrap()
);
}
response
}
},
Err(err) => {
tracing::error!("Error getting file content: {}", err);
(StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({
"error": format!("Error reading file: {}", err)
}))).into_response()
}
}
} else {
// For smaller files, load entirely but still potentially compress
match service.get_file_content(&id).await {
Ok(content) => {
// Create base headers
let mut headers = HashMap::new();
headers.insert(
header::CONTENT_DISPOSITION.to_string(),
format!("attachment; filename=\"{}\"", file.name)
);
if should_compress {
// Add content-encoding header for compressed response
headers.insert(header::CONTENT_ENCODING.to_string(), "gzip".to_string());
headers.insert(header::CONTENT_TYPE.to_string(), file.mime_type.clone());
headers.insert(header::VARY.to_string(), "Accept-Encoding".to_string());
// Compress the content
match compression_service.compress_data(&content, compression_level).await {
Ok(compressed_content) => {
tracing::debug!(
"Compressed file: {} from {}KB to {}KB (ratio: {:.2})",
file.name,
content.len() / 1024,
compressed_content.len() / 1024,
content.len() as f64 / compressed_content.len() as f64
);
// Build a custom response with headers and body
let mut response = Response::builder()
.status(StatusCode::OK)
.body(axum::body::Body::from(compressed_content))
.unwrap();
// Add headers to response
for (name, value) in headers {
response.headers_mut().insert(
HeaderName::from_bytes(name.as_bytes()).unwrap(),
HeaderValue::from_str(&value).unwrap()
);
}
response
},
Err(e) => {
tracing::warn!("Compression failed, sending uncompressed: {}", e);
// Fall back to uncompressed
headers.insert(header::CONTENT_TYPE.to_string(), file.mime_type.clone());
// Build a custom response with headers and body
let mut response = Response::builder()
.status(StatusCode::OK)
.body(axum::body::Body::from(content))
.unwrap();
// Add headers to response
for (name, value) in headers {
response.headers_mut().insert(
HeaderName::from_bytes(name.as_bytes()).unwrap(),
HeaderValue::from_str(&value).unwrap()
);
}
response
}
}
} else {
// No compression, return as-is
headers.insert(header::CONTENT_TYPE.to_string(), file.mime_type.clone());
// Build a custom response with headers and body
let mut response = Response::builder()
.status(StatusCode::OK)
.body(axum::body::Body::from(content))
.unwrap();
// Add headers to response
for (name, value) in headers {
response.headers_mut().insert(
HeaderName::from_bytes(name.as_bytes()).unwrap(),
HeaderValue::from_str(&value).unwrap()
);
}
response
}
},
Err(err) => {
tracing::error!("Error getting file content: {}", err);
(StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({
"error": format!("Error reading file: {}", err)
}))).into_response()
}
}
}
},
(Err(err), _) | (_, Err(err)) => {
Err(err) => {
let status = match &err {
FileRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
FileServiceError::NotFound(_) => StatusCode::NOT_FOUND,
FileServiceError::AccessError(_) => StatusCode::SERVICE_UNAVAILABLE,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -110,7 +357,7 @@ impl FileHandler {
},
Err(err) => {
let status = match &err {
FileRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
FileServiceError::NotFound(_) => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -131,7 +378,7 @@ impl FileHandler {
Ok(_) => StatusCode::NO_CONTENT.into_response(),
Err(err) => {
let status = match &err {
FileRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
FileServiceError::NotFound(_) => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -169,11 +416,11 @@ impl FileHandler {
},
Err(err) => {
let status = match &err {
FileRepositoryError::NotFound(_) => {
FileServiceError::NotFound(_) => {
tracing::error!("Error al mover archivo - no encontrado: {}", err);
StatusCode::NOT_FOUND
},
FileRepositoryError::AlreadyExists(_) => {
FileServiceError::Conflict(_) => {
tracing::error!("Error al mover archivo - ya existe: {}", err);
StatusCode::CONFLICT
},
+43 -17
View File
@@ -1,6 +1,6 @@
use std::sync::Arc;
use axum::{
extract::{Path, State},
extract::{Path, State, Query},
http::StatusCode,
response::IntoResponse,
Json,
@@ -8,7 +8,9 @@ use axum::{
use crate::application::services::folder_service::FolderService;
use crate::application::dtos::folder_dto::{CreateFolderDto, RenameFolderDto, MoveFolderDto};
use crate::domain::repositories::folder_repository::FolderRepositoryError;
use crate::application::dtos::pagination::PaginationRequestDto;
use crate::common::errors::ErrorKind;
use crate::application::ports::inbound::FolderUseCase;
type AppState = Arc<FolderService>;
@@ -24,9 +26,9 @@ impl FolderHandler {
match service.create_folder(dto).await {
Ok(folder) => (StatusCode::CREATED, Json(folder)).into_response(),
Err(err) => {
let status = match &err {
FolderRepositoryError::AlreadyExists(_) => StatusCode::CONFLICT,
FolderRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
let status = match err.kind {
ErrorKind::AlreadyExists => StatusCode::CONFLICT,
ErrorKind::NotFound => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -43,8 +45,8 @@ impl FolderHandler {
match service.get_folder(&id).await {
Ok(folder) => (StatusCode::OK, Json(folder)).into_response(),
Err(err) => {
let status = match &err {
FolderRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
let status = match err.kind {
ErrorKind::NotFound => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -66,8 +68,32 @@ impl FolderHandler {
(StatusCode::OK, Json(folders)).into_response()
},
Err(err) => {
let status = match &err {
FolderRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
let status = match err.kind {
ErrorKind::NotFound => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
// Return a JSON error response
(status, Json(serde_json::json!({
"error": err.to_string()
}))).into_response()
}
}
}
/// Lists folders with pagination support
pub async fn list_folders_paginated(
State(service): State<AppState>,
Query(pagination): Query<PaginationRequestDto>,
parent_id: Option<&str>,
) -> impl IntoResponse {
match service.list_folders_paginated(parent_id, &pagination).await {
Ok(paginated_result) => {
(StatusCode::OK, Json(paginated_result)).into_response()
},
Err(err) => {
let status = match err.kind {
ErrorKind::NotFound => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -88,9 +114,9 @@ impl FolderHandler {
match service.rename_folder(&id, dto).await {
Ok(folder) => (StatusCode::OK, Json(folder)).into_response(),
Err(err) => {
let status = match &err {
FolderRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
FolderRepositoryError::AlreadyExists(_) => StatusCode::CONFLICT,
let status = match err.kind {
ErrorKind::NotFound => StatusCode::NOT_FOUND,
ErrorKind::AlreadyExists => StatusCode::CONFLICT,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -111,9 +137,9 @@ impl FolderHandler {
match service.move_folder(&id, dto).await {
Ok(folder) => (StatusCode::OK, Json(folder)).into_response(),
Err(err) => {
let status = match &err {
FolderRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
FolderRepositoryError::AlreadyExists(_) => StatusCode::CONFLICT,
let status = match err.kind {
ErrorKind::NotFound => StatusCode::NOT_FOUND,
ErrorKind::AlreadyExists => StatusCode::CONFLICT,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
@@ -130,8 +156,8 @@ impl FolderHandler {
match service.delete_folder(&id).await {
Ok(_) => StatusCode::NO_CONTENT.into_response(),
Err(err) => {
let status = match &err {
FolderRepositoryError::NotFound(_) => StatusCode::NOT_FOUND,
let status = match err.kind {
ErrorKind::NotFound => StatusCode::NOT_FOUND,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
+4
View File
@@ -1,4 +1,8 @@
pub mod file_handler;
pub mod folder_handler;
pub mod i18n_handler;
pub mod batch_handler;
/// Tipo de resultado para controladores de API
pub type ApiResult<T> = Result<T, (axum::http::StatusCode, String)>;
+75 -3
View File
@@ -2,16 +2,27 @@ use std::sync::Arc;
use axum::{
routing::{get, post, put, delete},
Router,
extract::State,
extract::{State, Query, Path},
};
use tower_http::{compression::CompressionLayer, trace::TraceLayer};
use tower_http::{
compression::CompressionLayer,
trace::TraceLayer,
};
use crate::interfaces::middleware::cache::{HttpCache, start_cache_cleanup_task};
use crate::application::services::folder_service::FolderService;
use crate::application::services::file_service::FileService;
use crate::application::services::i18n_application_service::I18nApplicationService;
use crate::application::services::batch_operations::BatchOperationService;
use crate::interfaces::api::handlers::folder_handler::FolderHandler;
use crate::interfaces::api::handlers::file_handler::FileHandler;
use crate::interfaces::api::handlers::i18n_handler::I18nHandler;
use crate::interfaces::api::handlers::batch_handler::{
self, BatchHandlerState
};
use crate::application::dtos::pagination::PaginationRequestDto;
/// Creates API routes for the application
pub fn create_api_routes(
@@ -19,13 +30,57 @@ pub fn create_api_routes(
file_service: Arc<FileService>,
i18n_service: Option<Arc<I18nApplicationService>>,
) -> Router {
// Inicializar el servicio de operaciones por lotes
let batch_service = Arc::new(BatchOperationService::default(
file_service.clone(),
folder_service.clone()
));
// Crear estado para el manejador de operaciones por lotes
let batch_handler_state = BatchHandlerState {
batch_service: batch_service.clone(),
};
// Implement HTTP Cache
let http_cache = HttpCache::new();
// Define TTL values for different resource types (in seconds)
let _folders_ttl = 300; // 5 minutes
let _files_list_ttl = 300; // 5 minutes
let _i18n_ttl = 3600; // 1 hour
// Start the cleanup task for HTTP cache
start_cache_cleanup_task(http_cache.clone());
let folders_router = Router::new()
.route("/", post(FolderHandler::create_folder))
.route("/", get(|State(service): State<Arc<FolderService>>| async move {
// No parent ID means list root folders
FolderHandler::list_folders(State(service), None).await
}))
.route("/paginated", get(|
State(service): State<Arc<FolderService>>,
pagination: Query<PaginationRequestDto>
| async move {
// Paginación para carpetas raíz (sin parent)
FolderHandler::list_folders_paginated(State(service), pagination, None).await
}))
.route("/{id}", get(FolderHandler::get_folder))
.route("/{id}/contents", get(|
State(service): State<Arc<FolderService>>,
Path(id): Path<String>
| async move {
// Listar contenido de una carpeta por su ID
FolderHandler::list_folders(State(service), Some(&id)).await
}))
.route("/{id}/contents/paginated", get(|
State(service): State<Arc<FolderService>>,
Path(id): Path<String>,
pagination: Query<PaginationRequestDto>
| async move {
// Listar contenido paginado de una carpeta por su ID
FolderHandler::list_folders_paginated(State(service), pagination, Some(&id)).await
}))
.route("/{id}/rename", put(FolderHandler::rename_folder))
.route("/{id}/move", put(FolderHandler::move_folder))
.route("/{id}", delete(FolderHandler::delete_folder))
@@ -47,10 +102,24 @@ pub fn create_api_routes(
.route("/{id}/move", put(FileHandler::move_file))
.with_state(file_service);
// Crear rutas para operaciones por lotes
let batch_router = Router::new()
// Operaciones de archivos
.route("/files/move", post(batch_handler::move_files_batch))
.route("/files/copy", post(batch_handler::copy_files_batch))
.route("/files/delete", post(batch_handler::delete_files_batch))
.route("/files/get", post(batch_handler::get_files_batch))
// Operaciones de carpetas
.route("/folders/delete", post(batch_handler::delete_folders_batch))
.route("/folders/create", post(batch_handler::create_folders_batch))
.route("/folders/get", post(batch_handler::get_folders_batch))
.with_state(batch_handler_state);
// Create a router without the i18n routes
let mut router = Router::new()
.nest("/folders", folders_router)
.nest("/files", files_router);
.nest("/files", files_router)
.nest("/batch", batch_router);
// Add i18n routes if the service is provided
if let Some(i18n_service) = i18n_service {
@@ -68,7 +137,10 @@ pub fn create_api_routes(
router = router.nest("/i18n", i18n_router);
}
// Apply compression and tracing layers
router
.layer(CompressionLayer::new())
.layer(TraceLayer::new_for_http())
// HTTP caching is disabled temporarily due to compatibility issues
// .layer(HttpCacheLayer::new(http_cache.clone()).with_max_age(folders_ttl))
}
+565
View File
@@ -0,0 +1,565 @@
use axum::{
body::Body,
http::{HeaderMap, HeaderValue, Method, Request, Response, StatusCode},
middleware::Next,
};
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};
use std::time::{Duration, SystemTime};
use chrono::{DateTime, Utc};
use serde::Serialize;
use std::sync::{Arc, Mutex};
use std::collections::HashMap;
use tower::{Layer, Service};
use std::task::{Context, Poll};
use std::pin::Pin;
use std::future::Future;
use bytes::Bytes;
use tracing::{debug, info};
const MAX_CACHE_ENTRIES: usize = 1000; // Máximo número de entradas en caché
const DEFAULT_MAX_AGE: u64 = 60; // Tiempo de vida por defecto en segundos
// Definición de tipos para mayor claridad
type CacheKey = String;
type EntityTag = String;
/// Un valor almacenado en caché
#[derive(Clone)]
struct CacheEntry {
/// El ETag calculado para este valor
etag: EntityTag,
/// Los datos serializados en bytes
data: Option<Bytes>,
/// Las cabeceras originales
headers: HeaderMap,
/// Timestamp de cuando fue almacenado
timestamp: SystemTime,
/// Tiempo de vida en segundos
max_age: u64,
}
/// Cache para respuestas HTTP con soporte para ETag
#[derive(Clone)]
pub struct HttpCache {
/// Almacenamiento de entradas en caché
cache: Arc<Mutex<HashMap<CacheKey, CacheEntry>>>,
/// Tiempo de vida por defecto para las entradas
default_max_age: u64,
}
impl HttpCache {
/// Crea una nueva instancia del caché
pub fn new() -> Self {
Self {
cache: Arc::new(Mutex::new(HashMap::with_capacity(100))),
default_max_age: DEFAULT_MAX_AGE,
}
}
/// Crea una nueva instancia con un tiempo de vida especificado
#[allow(dead_code)]
pub fn with_max_age(max_age: u64) -> Self {
Self {
cache: Arc::new(Mutex::new(HashMap::with_capacity(100))),
default_max_age: max_age,
}
}
/// Obtiene estadísticas del caché
pub fn stats(&self) -> (usize, usize) {
let lock = self.cache.lock().unwrap();
let total = lock.len();
// Contar entradas válidas
let _now = SystemTime::now();
let valid = lock.values().filter(|entry| {
match entry.timestamp.elapsed() {
Ok(elapsed) => elapsed.as_secs() < entry.max_age,
Err(_) => false,
}
}).count();
(total, valid)
}
/// Limpia entradas expiradas
pub fn cleanup(&self) -> usize {
let mut lock = self.cache.lock().unwrap();
let initial_count = lock.len();
// Eliminar entradas expiradas
let _now = SystemTime::now();
lock.retain(|_, entry| {
match entry.timestamp.elapsed() {
Ok(elapsed) => elapsed.as_secs() < entry.max_age,
Err(_) => false,
}
});
let removed = initial_count - lock.len();
debug!("HttpCache cleanup: removed {} expired entries", removed);
removed
}
/// Establece una entrada en el caché
fn set(&self, key: &str, etag: EntityTag, data: Option<Bytes>, headers: HeaderMap, max_age: Option<u64>) {
let mut lock = self.cache.lock().unwrap();
// Aplicar política de eviction si el caché está lleno
if lock.len() >= MAX_CACHE_ENTRIES {
debug!("Cache full, removing oldest entries");
// Eliminar el 10% de las entradas más antiguas
self.evict_oldest(&mut lock, MAX_CACHE_ENTRIES / 10);
}
// Almacenar la nueva entrada
lock.insert(key.to_string(), CacheEntry {
etag,
data,
headers,
timestamp: SystemTime::now(),
max_age: max_age.unwrap_or(self.default_max_age),
});
}
/// Elimina las entradas más antiguas del caché
fn evict_oldest(&self, cache: &mut HashMap<CacheKey, CacheEntry>, count: usize) {
// Ordenar por timestamp
let mut entries: Vec<(CacheKey, SystemTime)> = cache
.iter()
.map(|(key, entry)| (key.clone(), entry.timestamp))
.collect();
// Ordenar por timestamp (más antiguo primero)
entries.sort_by(|a, b| a.1.cmp(&b.1));
// Eliminar las entradas más antiguas
for (key, _) in entries.iter().take(count) {
cache.remove(key);
}
}
/// Obtiene una entrada del caché
fn get(&self, key: &str) -> Option<CacheEntry> {
let lock = self.cache.lock().unwrap();
// Buscar la entrada
if let Some(entry) = lock.get(key) {
// Verificar si ha expirado
match entry.timestamp.elapsed() {
Ok(elapsed) if elapsed.as_secs() < entry.max_age => {
// Entry is still valid
return Some(entry.clone());
}
_ => {
// Entry has expired
return None;
}
}
}
None
}
/// Calcula el ETag para una respuesta
#[allow(dead_code)]
fn calculate_etag<T: Serialize>(&self, response: &T) -> EntityTag {
// Serializar la respuesta
let json = serde_json::to_string(response).unwrap_or_default();
// Calcular hash
let mut hasher = DefaultHasher::new();
json.hash(&mut hasher);
let hash = hasher.finish();
format!("\"{}\"", hash)
}
/// Genera un ETag simple para un bloque de bytes
fn calculate_etag_for_bytes(&self, bytes: &[u8]) -> EntityTag {
// Calcular hash
let mut hasher = DefaultHasher::new();
bytes.hash(&mut hasher);
let hash = hasher.finish();
format!("\"{}\"", hash)
}
}
/// Middleware de caché HTTP
#[allow(dead_code)]
pub async fn cache_middleware<T>(
cache: HttpCache,
cache_key: &str,
max_age: Option<u64>,
req: Request<Body>,
next: Next,
) -> Result<Response<Body>, (StatusCode, String)>
where
T: Serialize
{
// Solo aplicar caché para solicitudes GET
if req.method() != Method::GET {
return Ok(next.run(req).await);
}
// Verificar si la respuesta está en caché
let if_none_match = req.headers()
.get("if-none-match")
.and_then(|v| v.to_str().ok());
// Si hay una entrada en caché
if let Some(cache_entry) = cache.get(cache_key) {
// Comprobar si el cliente ya tiene la versión actualizada
if let Some(client_etag) = if_none_match {
if client_etag == cache_entry.etag {
// El cliente tiene la versión más reciente, enviar 304 Not Modified
debug!("Cache hit (304) for key: {}", cache_key);
return Ok(create_not_modified_response(&cache_entry));
}
}
// El cliente necesita la versión actualizada
if let Some(data) = &cache_entry.data {
debug!("Cache hit (200) for key: {}", cache_key);
// Crear respuesta con los datos en caché
let mut response = Response::new(Body::from(data.clone()));
// Copiar cabeceras originales
for (key, value) in &cache_entry.headers {
if !key.as_str().eq_ignore_ascii_case("transfer-encoding") {
response.headers_mut().insert(key.clone(), value.clone());
}
}
// Añadir cabeceras de caché
set_cache_headers(&mut response, &cache_entry.etag, max_age.unwrap_or(cache_entry.max_age));
return Ok(response);
}
}
// No está en caché o ha expirado, continuar con el middleware
debug!("Cache miss for key: {}", cache_key);
let response = next.run(req).await;
// No cachear errores
if !response.status().is_success() {
return Ok(response);
}
// Convertir la respuesta para calcular el ETag
let (parts, _body) = response.into_parts();
let bytes = axum::body::to_bytes(_body, 1024 * 1024 * 10).await.unwrap_or_default();
// Calcular ETag
let etag = cache.calculate_etag_for_bytes(&bytes);
// Guardar en caché
cache.set(
cache_key,
etag.clone(),
Some(bytes.clone()),
parts.headers.clone(),
max_age
);
// Crear la respuesta con ETag
let mut response = Response::from_parts(parts, Body::from(bytes));
set_cache_headers(&mut response, &etag, max_age.unwrap_or(cache.default_max_age));
Ok(response)
}
/// Crea una respuesta 304 Not Modified
fn create_not_modified_response(entry: &CacheEntry) -> Response<Body> {
let mut response = Response::builder()
.status(StatusCode::NOT_MODIFIED)
.body(Body::empty())
.unwrap();
// Copiar cabeceras de caché
if let Some(cache_control) = entry.headers.get("cache-control") {
response.headers_mut().insert("cache-control", cache_control.clone());
}
// Añadir ETag
response.headers_mut().insert(
"etag",
HeaderValue::from_str(&entry.etag).unwrap_or(HeaderValue::from_static(""))
);
response
}
/// Configura las cabeceras de caché para una respuesta
fn set_cache_headers(response: &mut Response<Body>, etag: &str, max_age: u64) {
// Añadir ETag
response.headers_mut().insert(
"etag",
HeaderValue::from_str(etag).unwrap_or(HeaderValue::from_static(""))
);
// Configurar Cache-Control
let cache_control = format!("public, max-age={}", max_age);
response.headers_mut().insert(
"cache-control",
HeaderValue::from_str(&cache_control).unwrap_or(HeaderValue::from_static(""))
);
// Añadir cabecera Last-Modified
let now: DateTime<Utc> = Utc::now();
let last_modified = now.format("%a, %d %b %Y %H:%M:%S GMT").to_string();
response.headers_mut().insert(
"last-modified",
HeaderValue::from_str(&last_modified).unwrap_or(HeaderValue::from_static(""))
);
}
/// Layer para aplicar middleware de caché
#[derive(Clone)]
pub struct HttpCacheLayer {
cache: HttpCache,
max_age: Option<u64>,
}
impl HttpCacheLayer {
/// Crea una nueva capa de caché
#[allow(dead_code)]
pub fn new(cache: HttpCache) -> Self {
Self {
cache,
max_age: None,
}
}
/// Establece el tiempo de vida máximo
#[allow(dead_code)]
pub fn with_max_age(mut self, max_age: u64) -> Self {
self.max_age = Some(max_age);
self
}
}
impl<S> Layer<S> for HttpCacheLayer {
type Service = HttpCacheService<S>;
fn layer(&self, service: S) -> Self::Service {
HttpCacheService {
inner: service,
cache: self.cache.clone(),
max_age: self.max_age,
}
}
}
/// Servicio que implementa la lógica de caché
#[derive(Clone)]
pub struct HttpCacheService<S> {
inner: S,
cache: HttpCache,
max_age: Option<u64>,
}
impl<S, ReqBody, ResBody> Service<Request<ReqBody>> for HttpCacheService<S>
where
S: Service<Request<ReqBody>, Response = Response<ResBody>>,
S::Future: Send + 'static,
S::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
ReqBody: Send + 'static,
ResBody: http_body::Body + Send + 'static,
ResBody::Data: Send + 'static,
ResBody::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
{
type Response = Response<Body>;
type Error = Box<dyn std::error::Error + Send + Sync>;
type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.inner.poll_ready(cx).map_err(|e| e.into())
}
fn call(&mut self, req: Request<ReqBody>) -> Self::Future {
// Generar clave de caché
let cache_key = req.uri().path().to_string();
// Solo aplicar caché para solicitudes GET
if req.method() != Method::GET {
let future = self.inner.call(req);
return Box::pin(async move {
let response = future.await.map_err(|e| e.into())?;
Ok(response_map_body(response))
});
}
// Obtener ETag del cliente
let if_none_match = req.headers()
.get("if-none-match")
.and_then(|v| v.to_str().ok());
// Verificar si hay una entrada en caché
let cache_clone = self.cache.clone();
let max_age = self.max_age;
let entry = cache_clone.get(&cache_key);
match entry {
Some(cache_entry) if if_none_match == Some(&cache_entry.etag) => {
// El cliente tiene la versión correcta, enviar 304
debug!("Cache HIT (304): {}", cache_key);
let response = create_not_modified_response(&cache_entry);
return Box::pin(async move { Ok(response) });
},
Some(cache_entry) if cache_entry.data.is_some() => {
// El cliente necesita la versión actualizada
debug!("Cache HIT (200): {}", cache_key);
let mut response = Response::new(Body::from(cache_entry.data.clone().unwrap()));
// Copiar cabeceras originales
for (key, value) in &cache_entry.headers {
if !key.as_str().eq_ignore_ascii_case("transfer-encoding") {
response.headers_mut().insert(key.clone(), value.clone());
}
}
// Añadir cabeceras de caché
set_cache_headers(&mut response, &cache_entry.etag, max_age.unwrap_or(cache_entry.max_age));
return Box::pin(async move { Ok(response) });
},
_ => {
// No está en caché o ha expirado
debug!("Cache MISS: {}", cache_key);
let future = self.inner.call(req);
let cache_clone = self.cache.clone();
let max_age = self.max_age;
let cache_key = cache_key.clone();
return Box::pin(async move {
let response = future.await.map_err(|e| e.into())?;
let response = response_map_body(response);
// No cachear errores
if !response.status().is_success() {
return Ok(response);
}
// Obtener el cuerpo y calcular ETag
let (parts, body) = response.into_parts();
let bytes = axum::body::to_bytes(body, 1024 * 1024 * 10).await?;
// Calcular ETag
let etag = cache_clone.calculate_etag_for_bytes(&bytes);
// Guardar en caché
cache_clone.set(
&cache_key,
etag.clone(),
Some(bytes.clone()),
parts.headers.clone(),
max_age
);
// Crear la respuesta con ETag
let mut response = Response::from_parts(parts, Body::from(bytes));
set_cache_headers(&mut response, &etag, max_age.unwrap_or(cache_clone.default_max_age));
Ok(response)
});
}
}
}
}
// Función auxiliar para convertir cualquier cuerpo en Body
fn response_map_body<B>(response: Response<B>) -> Response<Body>
where
B: http_body::Body + Send + 'static,
B::Data: Send + 'static,
B::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
{
let (parts, _body) = response.into_parts();
// Create a simple empty body as a fallback - in production you would handle this better
let mapped_body = Body::empty();
Response::from_parts(parts, mapped_body)
}
/// Inicia una tarea de limpieza periódica para el caché
pub fn start_cache_cleanup_task(cache: HttpCache) {
tokio::spawn(async move {
let mut interval = tokio::time::interval(Duration::from_secs(300)); // Cada 5 minutos
loop {
interval.tick().await;
let removed = cache.cleanup();
let (total, valid) = cache.stats();
info!("HTTP Cache cleanup: removed {}, current: {}/{}", removed, valid, total);
}
});
}
#[cfg(test)]
mod tests {
use super::*;
use hyper::{Request, Body, Response};
use axum::routing::get;
use axum::{Extension, Json, Router};
use tower::ServiceExt;
use http::StatusCode;
use serde::{Deserialize, Serialize};
#[derive(Debug, Serialize, Deserialize)]
struct TestData {
id: u32,
name: String,
}
#[tokio::test]
async fn test_etag_generation() {
let cache = HttpCache::new();
let data1 = TestData { id: 1, name: "Test".to_string() };
let data2 = TestData { id: 1, name: "Test".to_string() };
let data3 = TestData { id: 2, name: "Test".to_string() };
let etag1 = cache.calculate_etag(&data1);
let etag2 = cache.calculate_etag(&data2);
let etag3 = cache.calculate_etag(&data3);
// Mismos datos deben generar mismo ETag
assert_eq!(etag1, etag2);
// Datos diferentes deben generar ETags diferentes
assert_ne!(etag1, etag3);
}
#[tokio::test]
async fn test_cache_hit_miss() {
let cache = HttpCache::new();
// Primera petición (cache miss)
let response1 = Response::builder()
.status(StatusCode::OK)
.body(Body::from(r#"{"id":1,"name":"Test"}"#))
.unwrap();
let (parts1, body1) = response1.into_parts();
let bytes1 = hyper::body::to_bytes(body1).await.unwrap();
let etag1 = cache.calculate_etag_for_bytes(&bytes1);
cache.set("test", etag1.clone(), Some(bytes1.clone()), parts1.headers.clone(), None);
// Verificar cache hit
let entry = cache.get("test").unwrap();
assert_eq!(entry.etag, etag1);
assert_eq!(entry.data.unwrap(), bytes1);
// Verificar cache miss
assert!(cache.get("nonexistent").is_none());
}
}
+1
View File
@@ -0,0 +1 @@
pub mod cache;
+62
View File
@@ -0,0 +1,62 @@
use super::cache::{HttpCache, HttpCacheLayer, start_cache_cleanup_task};
use axum::{
routing::get,
Router,
response::IntoResponse,
Json,
extract::State,
};
use serde::{Serialize, Deserialize};
use std::sync::Arc;
use std::time::Duration;
use std::net::SocketAddr;
#[derive(Clone, Debug, Serialize, Deserialize)]
struct TestResponse {
message: &'static str,
timestamp: u64,
}
// Test handler for a simple GET endpoint
async fn test_handler() -> impl IntoResponse {
// Create a simple response with a timestamp
let timestamp = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
// Simulate some processing time
tokio::time::sleep(Duration::from_millis(50)).await;
let response = TestResponse {
message: "Hello, this response is cacheable!",
timestamp,
};
// Log the response generation
tracing::info!("Generated fresh response with timestamp: {}", timestamp);
Json(response)
}
// Run a test server with HTTP caching enabled
pub async fn run_test_server() {
// Initialize HTTP cache with 10 seconds TTL
let http_cache = HttpCache::with_max_age(10);
// Start the cleanup task
start_cache_cleanup_task(http_cache.clone());
// Create a test router with the cache middleware
let app = Router::new()
.route("/test", get(test_handler))
.layer(HttpCacheLayer::new(http_cache));
// Bind to a test port
let addr = SocketAddr::from(([127, 0, 0, 1], 8086));
tracing::info!("HTTP Cache test server listening on {}", addr);
// Start the server
let listener = tokio::net::TcpListener::bind(addr).await.unwrap();
axum::serve(listener, app).await.unwrap();
}
+1
View File
@@ -1,4 +1,5 @@
pub mod api;
pub mod web;
pub mod middleware;
pub use api::create_api_routes;
+18
View File
@@ -0,0 +1,18 @@
// Exportar los módulos principales del proyecto
pub mod common;
pub mod domain;
pub mod application;
pub mod infrastructure;
pub mod interfaces;
// Re-exportaciones públicas comunes
pub use application::services::folder_service::FolderService;
pub use application::services::file_service::FileService;
pub use application::services::i18n_application_service::I18nApplicationService;
pub use application::services::storage_mediator::{StorageMediator, FileSystemStorageMediator};
pub use domain::services::path_service::PathService;
pub use infrastructure::repositories::folder_fs_repository::FolderFsRepository;
pub use infrastructure::repositories::file_fs_repository::FileFsRepository;
pub use infrastructure::repositories::parallel_file_processor::ParallelFileProcessor;
pub use infrastructure::services::buffer_pool::BufferPool;
pub use infrastructure::services::compression_service::GzipCompressionService;
+101 -4
View File
@@ -7,6 +7,7 @@ use axum::serve;
use tower_http::trace::TraceLayer;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};
mod common;
mod domain;
mod application;
mod infrastructure;
@@ -15,9 +16,17 @@ mod interfaces;
use application::services::folder_service::FolderService;
use application::services::file_service::FileService;
use application::services::i18n_application_service::I18nApplicationService;
use application::services::storage_mediator::FileSystemStorageMediator;
use domain::services::path_service::PathService;
use infrastructure::repositories::folder_fs_repository::FolderFsRepository;
use infrastructure::repositories::file_fs_repository::FileFsRepository;
use infrastructure::repositories::parallel_file_processor::ParallelFileProcessor;
use infrastructure::services::file_system_i18n_service::FileSystemI18nService;
use infrastructure::services::id_mapping_service::IdMappingService;
use infrastructure::services::id_mapping_optimizer::IdMappingOptimizer;
use infrastructure::services::file_metadata_cache::FileMetadataCache;
use infrastructure::services::buffer_pool::BufferPool;
use infrastructure::services::compression_service::GzipCompressionService;
use interfaces::{create_api_routes, web::create_web_routes};
#[tokio::main]
@@ -42,11 +51,91 @@ async fn main() {
std::fs::create_dir_all(&locales_path).expect("Failed to create locales directory");
}
// Initialize repositories
let folder_repository = Arc::new(FolderFsRepository::new(storage_path.clone()));
let file_repository = Arc::new(FileFsRepository::new(storage_path.clone(), folder_repository.clone()));
// Initialize path service
let path_service = Arc::new(PathService::new(storage_path.clone()));
// Initialize services
// Initialize ID mapping service with optimizer
let id_mapping_path = storage_path.join("folder_ids.json");
let base_id_mapping_service = Arc::new(
IdMappingService::new(id_mapping_path).await
.expect("Failed to initialize ID mapping service")
);
// Create optimized ID mapping service with batch processing and caching
let id_mapping_optimizer = Arc::new(
IdMappingOptimizer::new(base_id_mapping_service.clone())
);
// Initialize folder repository with all required components
let folder_repository = Arc::new(FolderFsRepository::new(
storage_path.clone(),
Arc::new(FileSystemStorageMediator::new_stub()), // Temporary stub (will be replaced)
base_id_mapping_service.clone(),
path_service.clone()
));
// Initialize storage mediator
let storage_mediator = Arc::new(FileSystemStorageMediator::new(
folder_repository.clone(),
path_service.clone(),
id_mapping_optimizer.clone()
));
// Update folder repository with proper storage mediator
// This replaces the stub we initialized it with
let folder_repository = Arc::new(FolderFsRepository::new(
storage_path.clone(),
storage_mediator.clone(),
base_id_mapping_service.clone(),
path_service.clone()
));
// Start cleanup task for ID mapping optimizer
IdMappingOptimizer::start_cleanup_task(id_mapping_optimizer.clone());
tracing::info!("ID mapping optimizer initialized with batch processing and caching");
// Initialize the metadata cache
let config = common::config::AppConfig::default();
let metadata_cache = Arc::new(FileMetadataCache::default_with_config(config.clone()));
// Start the periodic cleanup task for cache maintenance
let cache_clone = metadata_cache.clone();
tokio::spawn(async move {
FileMetadataCache::start_cleanup_task(cache_clone).await;
});
// Initialize the buffer pool for memory optimization
// Use larger buffer size for better performance with large files
let buffer_pool = BufferPool::new(256 * 1024, 50, 120); // 256KB buffers, 50 max, 2 min TTL
// Start the buffer pool cleanup task
BufferPool::start_cleaner(buffer_pool.clone());
tracing::info!("Buffer pool initialized with 50 buffers of 256KB each");
// Initialize parallel file processor with buffer pool
let parallel_processor = Arc::new(ParallelFileProcessor::new_with_buffer_pool(
config.clone(),
buffer_pool.clone()
));
// Initialize compression service with buffer pool
let _compression_service = Arc::new(GzipCompressionService::new_with_buffer_pool(
buffer_pool.clone()
));
// Initialize file repository with mediator, ID mapping service, metadata cache, and parallel processor
let file_repository = Arc::new(FileFsRepository::new_with_processor(
storage_path.clone(),
storage_mediator,
base_id_mapping_service.clone(), // Use the base service, not the optimizer
path_service.clone(),
metadata_cache.clone(), // Clone to keep a reference for later use
parallel_processor
));
// Initialize application services
let folder_service = Arc::new(FolderService::new(folder_repository));
let file_service = Arc::new(FileService::new(file_repository));
@@ -62,6 +151,8 @@ async fn main() {
tracing::warn!("Failed to load Spanish translations: {}", e);
}
tracing::info!("Compression service initialized with buffer pool support");
// Build application router
let api_routes = create_api_routes(folder_service, file_service, Some(i18n_service));
let web_routes = create_web_routes();
@@ -71,6 +162,12 @@ async fn main() {
.merge(web_routes)
.layer(TraceLayer::new_for_http());
// Preload common directories to warm the cache
tracing::info!("Preloading common directories to warm up cache...");
if let Ok(count) = metadata_cache.preload_directory(&storage_path, true, 1).await {
tracing::info!("Preloaded {} directory entries into cache", count);
}
// Start server
let addr = SocketAddr::from(([127, 0, 0, 1], 8085));
tracing::info!("listening on {}", addr);
+5
View File
@@ -0,0 +1,5 @@
{
"path_to_id": {},
"id_to_path": {},
"version": 0
}
View File
Regular → Executable
View File
Regular → Executable
View File
View File

Before

Width:  |  Height:  |  Size: 657 B

After

Width:  |  Height:  |  Size: 657 B

View File
Regular → Executable
View File
Regular → Executable
View File
View File
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB