Commit Graph

90 Commits

Author SHA1 Message Date
Dionisio 641b6853ad perf: add socket2 TCP_NODELAY + socket tuning for low-latency responses
- Replace basic TcpListener::bind with socket2 tuned socket
- TCP_NODELAY: disable Nagle's algorithm (-5 to 40ms latency on small responses)
- SO_REUSEADDR: port available immediately after server restart
- SO_REUSEPORT: ready for multi-worker scaling (Linux)
- TCP_KEEPALIVE: detect dead connections within 60s/10s interval
- listen(2048): high backlog for WebDAV connection bursts
- Eliminate redundant create_dir_all calls from upload hot path
2026-03-02 00:12:33 +01:00
Dionisio e2fb29ea60 perf: replace SHA-256 with BLAKE3 + add mimalloc global allocator
- Replace SHA-256 with BLAKE3 (~5x faster) for content-addressable hashing
  in dedup_service, file_handler, file_upload_service, chunked_upload_service
- Add mimalloc as global allocator for 10-30% throughput improvement
- sha2 crate retained only for PKCE (OAuth2 standard requirement)
- BLAKE3 produces 64-char hex hashes (same format), no DB schema changes needed
2026-03-01 21:47:39 +01:00
Dionisio 81987e9321 fix: URL-decode DAV paths with spaces + feat: app passwords for Basic Auth
Bug fix:
- URL-decode paths in extract_webdav_path(), extract_caldav_path(),
  extract_carddav_path() so folders with spaces (e.g. 'My Folder') no
  longer return 404 when accessed via encoded URIs (%20)
- Properly encode href values in PROPFIND/PROPPATCH/LOCK XML responses
- Decode Destination header in MOVE/COPY operations

New feature - App Passwords (API keys for DAV clients):
- POST /api/auth/app-passwords  → create (shows token once)
- GET  /api/auth/app-passwords  → list (prefix only)
- DELETE /api/auth/app-passwords/:id → revoke
- Auth middleware now accepts both Bearer JWT and Basic Auth
- Argon2 hashed, scoped (webdav/caldav/carddav), optional expiry
- Compatible with DAVx5, Thunderbird, rclone, curl

Tested: 12/12 E2E tests pass (create, list, WebDAV/CalDAV/CardDAV
Basic Auth, URL-decode with spaces, wrong password 401, revoke, post-
revoke 401).
2026-03-01 20:34:12 +01:00
Dionisio 2421724b80 style: cargo fmt --all 2026-02-26 01:02:25 +01:00
Dionisio f9dde6ffff perf: replace RwLock<HashMap> with DashMap in ChunkedUploadService + decouple disk I/O from lock
Issue #3 (CRITICAL): The global RwLock<HashMap> serialised ALL chunk uploads
across all users. finalize/cancel/cleanup held a write lock during
fs::remove_dir_all (~100-500ms), blocking every concurrent upload.

Changes:
- Replace tokio::sync::RwLock<HashMap<String, UploadSession>> with
  dashmap::DashMap (sharded concurrent map, ~64 shards)
- Operations on independent sessions never contend
- finalize_upload_inner: remove from map (µs), THEN delete temp dir
- cancel_upload_inner: same pattern — disk I/O outside lock
- cleanup_loop: collect expired IDs via lock-free iteration, remove
  from map, THEN delete dirs sequentially with no lock held
- upload_chunk_inner: DashMap::get_mut replaces global write lock
- get_status_inner / complete_upload_inner: DashMap::get replaces read lock
- Remove tokio::sync::RwLock import (dead)

Also includes Issue #2 (dedup_service.rs write-first + upsert) from
previous session.

Impact: p99 latency under 50 concurrent uploads drops from ~500ms to <1ms
for cross-session contention. Cleanup loop no longer blocks uploads.
2026-02-25 23:31:51 +01:00
Dionisio 5f883aa0f8 perf: rewrite batch download_zip to stream via temp file instead of RAM
- Replace in-memory Cursor<Vec<u8>> with async_zip + NamedTempFile (O(256KB) RAM)
- Stream ZIP to client via ReaderStream instead of materializing entire archive
- Replace N+1 BFS folder traversal with 2 bulk ltree queries
- Remove dead zip_service field and with_zip_service() from BatchOperationService
- Remove synchronous zip crate dependency (only async_zip remains)
- Eliminates ~2.5GB RAM spike per 2GB batch download
2026-02-25 23:12:19 +01:00
Dionisio 7786871d6c feat: P1 — audio/video modal player + MIME detection via magic bytes (infer)
Backend:
- Add infer crate for magic-byte MIME detection (<1μs per file)
- New src/common/mime_detect.rs: refine_content_type() with priority
  magic bytes > extension > client Content-Type
- Inject MIME refinement in file upload handler (after spool to temp)
- Inject MIME refinement in chunked upload handler (after assembly)

Frontend:
- Extend isViewableFile() to include audio/* and video/*
- Add createMediaViewer() to InlineViewer with <audio>/<video> controls
- Blob URL pattern for authenticated streaming playback
- Graceful fallback for unsupported codecs (error message + download)
- CSS: video player, audio wrapper with animated icon, responsive
2026-02-24 23:15:10 +01:00
Dionisio 4fc0ab8831 perf: replace buffered decompress_stream with async-compression streaming
- Add async-compression dependency with tokio+gzip features
- Rewrite decompress_stream to use true streaming pipeline:
  Stream<Bytes> → StreamReader → BufReader(64KB) → GzipDecoder → ReaderStream(64KB)
- Memory usage drops from ~4GB (1GB compressed file) to constant ~128KB
- Eliminates OOM risk on large compressed file decompression
2026-02-24 11:47:21 +01:00
Dionisio a162aafd43 perf: replace sync zip crate with async_zip in ZipService
ZipWriter<std::fs::File> performed every write_all() as a blocking
write(2) syscall on the Tokio worker thread, sequestering it for
10-100ms per file (0.8-12s for a 100-file ZIP).

Replace with async_zip::ZipFileWriter backed by a buffered
tokio::fs::File:
- All I/O (headers, deflate chunks, central directory) is fully async
- 256 KB BufWriter minimises syscall count
- Zero Tokio worker blocking during ZIP creation
- Streaming per-chunk writes keep RAM O(1) regardless of archive size
- Removed dead From<zip::result::ZipError> for DomainError impl
- zip crate retained for batch_operations.rs (separate concern)
2026-02-23 23:11:55 +01:00
George Wu 7e530a6b8c Add missing version update from 0.4.1 to 0.4.2 2026-02-21 12:03:09 -08:00
George Wu b28aa341b3 Enable brotli compression for API and static files
- Add compression-br feature to tower-http
- Apply CompressionLayer to API routes (JSON responses)
- Apply CompressionLayer to static files (CSS, JS, locales)
- File downloads remain uncompressed (avoid double compression)
2026-02-17 23:00:45 -08:00
Dionisio a4709426d9 fix(upload): sanitize multipart filename for folder uploads (#121)
Browsers send the full relative path (e.g. 'Screenshots/file.png') as
the multipart filename when uploading folders via webkitRelativePath.
The File entity rejects names containing '/' or '\', causing all files
in a folder upload to fail with 'Invalid file name'.

Three fixes:
- Backend: strip path components from multipart filename in file_handler,
  keeping only the basename. Also prevents path-traversal attacks.
- Frontend (fileOperations.js): explicitly pass file.name as the third
  argument to FormData.append() in uploadFolderFiles() to override the
  browser's relative path.
- Frontend (ui.js): detect folder drops in drag-and-drop handlers by
  checking webkitRelativePath, and route them to uploadFolderFiles()
  instead of uploadFiles() so subfolders are created first.

Closes #121
2026-02-16 17:58:50 +01:00
Dionisio b7bd656a43 perf: fix issues #1,#2,#3,#13 from performance audit
- #1  image_transcode: dedicated rayon pool + moka cache (no tokio blocking)
- #2  ltree materialized paths: eliminate N+1 folder/file queries
- #3  compute_content_hash: read blob_hash column instead of loading file into RAM
- #13 event delegation + DocumentFragment: ~15 delegated listeners replace ~19k per-item listeners

Backend: 82/82 tests pass, cargo check clean.
Frontend: ui.js and app.js syntax-validated via Node.js.
2026-02-16 13:10:44 +01:00
Dionisio 6e1b77f244 chore: remove orphan test_cache.rs and unused memmap2 dependency 2026-02-15 21:20:20 +01:00
Dionisio 1ed20f425f perf: Phase 4+5 optimizations — uploads 10x, downloads 2x, concurrent 2x. moka cache, 512KB buffers, remove sync_all, hash-on-write, preloaded queries, bench.sh v3, gitignore storage/. 500MB upload 12.6s->1.3s (392MB/s). RSS 69-113MB, 0 swap. 2026-02-15 17:56:47 +01:00
Dionisio 1c5cf97cc5 security: fix audit vulnerabilities (RUSTSEC-2026-0007, RUSTSEC-2021-0141)
- Update bytes 1.11.0 -> 1.11.1 (fixes integer overflow in BytesMut::reserve, CVE-2026-25541)
- Replace unmaintained dotenv 0.15.0 with dotenvy 0.15.7 (RUSTSEC-2021-0141)
- Note: rsa 0.9.10 (RUSTSEC-2023-0071) has no patch yet, pulled transitively via jsonwebtoken
2026-02-14 01:37:03 +01:00
Dionisio 28a353e17e chore: bump version to 0.3.5 2026-02-13 22:45:59 +01:00
Dionisio 40bf43b292 fix: cap default storage quota to available disk space (#92)
Previously the admin quota was hardcoded to 100 GB and the regular
user quota to 1 GB, regardless of actual disk capacity. On systems
with less than 100 GB free this produced misleading quota values.

Added the fs2 crate to query available disk space on the storage
filesystem. A new capped_quota() helper now returns
min(default_quota, available_disk_space) when assigning quotas
during user registration, admin setup, and OIDC provisioning.
2026-02-13 21:58:54 +01:00
Dionisio 1be3e4230a feat: admin can create users manually + disable registration (#85)
Backend:
- POST /api/admin/users — admin-only user creation endpoint
  - username & password required, email optional (auto-generated placeholder)
  - role, quota_bytes, active all configurable
  - creates personal folder automatically
- PUT /api/admin/users/{id}/password — admin password reset
- GET/PUT /api/admin/settings/registration — toggle public registration
  - Supports env var OXICLOUD_DISABLE_REGISTRATION override
  - Blocks POST /api/auth/register when disabled
- AdminCreateUserDto, AdminResetPasswordDto added to settings DTOs
- registration_enabled field added to DashboardStatsDto

Frontend (admin.html):
- 'Create User' button in Users tab with full modal form
  (username, password, email, role, quota)
- 'Reset Password' button per user in actions column
- 'Allow public self-registration' toggle in Dashboard > System
  with warning banner when disabled

Closes #85
2026-02-13 16:46:59 +01:00
Dionisio ea234bc6a1 fix: share dialog not opening + connect to backend API
- Fix showShareDialog: add try-catch, null checks, prevent textContent
  from destroying header icon (use span child instead)
- Capture file/folder target before closeContextMenu to prevent race
- createSharedLink now calls real backend POST /api/shares instead of
  localStorage-only mock (still caches locally for offline compat)
- Fix share_handler.rs: use OptionalAuthUser instead of AuthUser to
  prevent 401 when auth is disabled (same pattern as delete/trash)
- Add null-safety to closeShareDialog
- Reset new-share-section on dialog open
2026-02-13 12:29:21 +01:00
Dionisio 915d9a4353 chore: bump version to 0.3.2 2026-02-12 23:27:20 +01:00
Dionisio ad07a5abda fix: auto-apply DB schema on fresh install and fix admin registration (#81)
- Auto-apply schema.sql when database tables don't exist (embedded in binary)
- Handle missing tables gracefully during admin registration (treat as fresh install)
- Fix docker-compose depends_on to wait for postgres healthcheck
- Rename personal folder from 'Mi Carpeta' to 'My Folder' with backward compat
- Translate remaining Spanish messages to English
2026-02-12 14:45:52 +01:00
Dionisio d448e632ed fix: downgrade rand_core to 0.6 for argon2 compatibility
rand_core 0.9.x changed OsRng API (no longer implements RngCore directly)
and is incompatible with argon2 0.5.x which depends on rand_core 0.6.x.
This caused compilation failures in CI.
2026-02-11 17:59:18 +01:00
Dionisio 8ef62109a3 feat(auth): add OpenID Connect (OIDC) authentication support
Implements OIDC Authorization Code Flow for external identity providers
(Authentik, Keycloak, etc.) with JIT user provisioning.

New features:
- OidcService with OpenID Discovery, JWKS caching, RS256 ID token validation
- Authorization Code Flow: /api/auth/oidc/authorize -> IdP -> /api/auth/oidc/callback
- JIT user provisioning from OIDC claims (sub, email, name, groups)
- OIDC group-to-role mapping (admin_groups config)
- Provider info endpoint: GET /api/auth/oidc/providers
- Option to disable password login entirely (OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN)
- Auto-provision toggle (OXICLOUD_OIDC_AUTO_PROVISION)
- Email collision detection (security: prevents account takeover)

Configuration (env vars):
- OXICLOUD_OIDC_ENABLED, OXICLOUD_OIDC_ISSUER_URL
- OXICLOUD_OIDC_CLIENT_ID, OXICLOUD_OIDC_CLIENT_SECRET
- OXICLOUD_OIDC_REDIRECT_URI, OXICLOUD_OIDC_SCOPES
- OXICLOUD_OIDC_FRONTEND_URL, OXICLOUD_OIDC_PROVIDER_NAME
- OXICLOUD_OIDC_AUTO_PROVISION, OXICLOUD_OIDC_ADMIN_GROUPS
- OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN

DB migration:
- ALTER TABLE auth.users ADD oidc_provider, oidc_subject columns
- UNIQUE index on (oidc_provider, oidc_subject)

Files changed: 14 files, ~1400 lines added
Dependencies: reqwest 0.12 (rustls-tls-webpki-roots), base64 0.22
2026-02-10 20:32:32 +01:00
Dionisio ef9ed2cc31 feat: complete CalDAV (RFC 4791) and CardDAV (RFC 6352) implementation
- CalDAV: MKCALENDAR, PROPFIND, PUT/GET/DELETE events, REPORT calendar-query
- CardDAV: MKCOL, PROPFIND, PUT/GET/DELETE vCards, REPORT addressbook-query
- Fix routing: move CalDAV/CardDAV to top-level merge() with explicit routes
- Fix DB schema: VARCHAR(36) -> UUID for entity IDs, vcard_data -> vcard
- Fix 15 repository stub methods that returned empty results
- Fix vCard parser in ContactStorageAdapter (was hardcoded stub)
- All operations tested end-to-end in Docker (201/207/200/204 as expected)
2026-02-10 18:46:59 +01:00
Diocrafts a82faa5eaf refactoring hexagonal and clean architecture 2026-02-08 13:40:23 +01:00
Dionisio 8f2b0a354c big refactoring 2026-02-03 17:59:04 +01:00
Dionisio 52840e57df refactor: remove serde from domain entities for Clean Architecture compliance
- Remove Serialize/Deserialize from File, Folder, Session, User, Contact entities
- Create contact_persistence_dto.rs for JSONB persistence in infrastructure layer
- Update contact_pg_repository to use persistence DTOs
- Fix dependency on zip crate (downgrade from 7.2.0 to 2.1.0)
- Fix unused variable warnings in main.rs
- Move PathService import from domain to infrastructure
- Add missing fields to CoreServices and RepositoryServices
- Create proper service initialization in main.rs

Clean Architecture improvements:
- Domain layer no longer depends on serde framework
- Persistence concerns isolated to infrastructure layer
- TokenClaims in auth_service.rs is only exception (required for JWT)
2026-02-02 23:56:40 +01:00
Shubham Gupta 727aee3ad0 Add dotenv to dependencies 2025-09-10 00:28:18 +08:00
DioCrafts 52d8250d51 adding card dav and cald dav 2025-04-13 01:04:04 +02:00
DioCrafts 5d67bc4d84 fix several bugs 2025-04-10 01:43:25 +02:00
DioCrafts ba3ef2a530 adding webdav features 2025-04-04 21:31:41 +02:00
DioCrafts 705cb5b069 adding pdf and image viewer 2025-04-02 01:22:05 +02:00
dionidev e79ca8304b adding comments, moving technical documentation, improve Dockerfile, delete unuseful files 2025-03-30 14:17:09 +00:00
DioCrafts 3a4cb75ac7 configuring backend topology 2025-03-28 12:38:48 +01:00
DioCrafts bfbef18117 fix warnings 2025-03-27 23:41:56 +01:00
DioCrafts 9a9fd72f61 fixing bugs 2025-03-23 22:44:18 +01:00
DioCrafts cafad0fbfd adding user authentication 2025-03-20 09:22:31 +01:00
DioCrafts d9bbd575d2 adding several features 2025-03-19 00:44:27 +01:00
root fe19bc8505 Initial commit 2025-03-17 21:28:08 +01:00