Commit Graph

20 Commits

Author SHA1 Message Date
DioCrafts 72715c66a0 Merge remote-tracking branch 'origin/main' into perf/auth-me-quota
# Conflicts:
#	example.env
#	src/common/config.rs
2026-06-07 01:25:14 +02:00
DioCrafts d7c6894c80 perf(quota): stop recomputing storage usage on every GET /api/auth/me
GET /api/auth/me ran a synchronous O(N) SUM(size) over all the user's
files plus an unconditional UPDATE of auth.users on every call — one of
the most frequently hit endpoints — adding per-request latency, DB write
load, dead tuples and WAL even when nothing changed.

- /api/auth/me now serves the cached storage_used_bytes column instead of
  recomputing it inline.
- New StorageUsageService::start_reconciliation_job runs a periodic sweep
  on the maintenance pool that keeps the cached value current for every
  mutation (uploads, deletes, trash), so freshness no longer depends on
  hitting /me. Interval via OXICLOUD_STORAGE_USAGE_RECONCILE_SECS (default
  600s, floored at 30s; first sweep deferred one interval to avoid boot load).
- update_storage_usage only writes when the value actually changes
  (IS DISTINCT FROM), so the sweep produces no dead tuple / WAL on no-ops.
- New covering partial index idx_files_user_size_active makes the usage
  SUM an index-only scan instead of a heap scan over all the user's files.

Also collapse the same pre-existing clippy collapsible_else_if in
carddav_handler that blocks the -D warnings gate on this base.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 01:13:58 +02:00
DioCrafts 061306cc84 fix(upload): stream WebDAV/NextCloud PUT to disk to prevent OOM on large files
Large uploads (e.g. ~800 MB ISOs) could OOMKill the process, even on
dedup hits, due to three separate full-file-in-memory paths:

- NextCloud PUT (/remote.php/dav) buffered the entire body in RAM via
  body::to_bytes before any dedup logic, then re-wrote and re-hashed it.
  Now streams the body to a temp file with incremental BLAKE3 and goes
  through update_file_streaming (shared spool helper with the native
  WebDAV PUT handler); peak heap is ~one HTTP frame regardless of size.

- DedupService::store_chunks materialized every new chunk's data in a Vec
  before uploading. Now reads each new chunk by positioned I/O
  (read_exact_at, off the runtime via spawn_blocking) just before its
  upload; peak heap bounded to ~CHUNK_UPLOAD_CONCURRENCY x CDC_MAX_CHUNK.

- The upload spool used the OS temp dir, often tmpfs/RAM in containers
  where its page-cache counts against the cgroup memory limit. Add
  OXICLOUD_UPLOAD_TMPDIR to point the spool at real disk.

Also collapse a pre-existing clippy collapsible_else_if in carddav_handler.

Refs #404

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 01:03:52 +02:00
Edouard Vanbelle 8cc21f17c5 feat(notify): add notif to internal users when granted
- add coalesced protection to avoid mail bombing if an invited goes many grant in a short period
    - add resentd method in share menu item (work for both internal and external users)
    - user can disable email notification via his properties
    - add env variable from admin to disable notifications
2026-06-05 11:25:06 +02:00
Edouard Vanbelle 044bd76738 feat(i18n): add i18n on server side
- remove the hardcoded list of locales in favor of a discovry on start time
    - server will stop on badly formatted locale .json
    - add server.* entries for serer side translation

    server side translation will be used for templating and email
    note: no json in some embded html (like in /magic), amount of work was similar
2026-06-03 13:27:12 +02:00
Edouard Vanbelle 130ff363dc feat(passwordless): pass4: add env variable to enable mgaiclink on account with password
OXICLOUD_MAGIC_LINK_OPEN_TO_PASSWORD_USERS (default false)
    For security I recommand to keep it false
    OIDC cannot be bypassed because OIDC may have MFA in place
2026-06-03 00:35:25 +02:00
Edouard Vanbelle 21c06da700 feat(magic-links): add rate limiting + archirecture documentation 2026-06-03 00:32:00 +02:00
Edouard Vanbelle 408b084ee6 chore: SMTP is a simple implementation, recommand local server to have a spool 2026-06-03 00:31:59 +02:00
Edouard Vanbelle 8a05ee00cb doc: add OXICLOUD_SMTP_* variables + external variables user options 2026-06-03 00:31:59 +02:00
Edouard Vanbelle d273146138 feat(server): add OXICLOUD_REUSE_PORT env parameter (false by default)
initially server was accepting multiple instances on same port (Linux and MacOS only)
    this can create issues during development (if a dev forget another running instance...)

    Add OXICLOUD_REUSE_PORT variable to activate it, so only admins knowing this feature can activate itt
    (permits multiple instance + let the OS schedduler to decide which process will handle a request)

    if not enabled, other instance will receive exit with a:
    `Error: Os { code: 48, kind: AddrInUse, message: "Address already in use" }`
2026-05-28 12:10:01 +02:00
Edouard Vanbelle 815c9df613 feat(server): add support of IPv6 (no dual stack) 2026-05-14 20:11:12 +02:00
Edouard Vanbelle 8b08ee165d doc: update example.env and docs/config/env.md 2026-05-10 22:12:55 +02:00
Edouard Vanbelle 8e1a738056 feat(audit): show trace with HTTP's client_ip and user if logged in + add support of trusted proxy via CIDR 2026-05-05 09:44:25 +02:00
onthebed 0629177278 docs(wopi): document public and callback base URLs
Clarify how OXICLOUD_WOPI_PUBLIC_BASE_URL and OXICLOUD_WOPI_BASE_URL
should be set when the office editor runs on a different hostname.
2026-04-26 13:16:04 +08:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
Nathan Shepperd 769c95fac9 Update example.env
Note in example.env file about accessing remotely via the server's domain name
2026-03-11 08:46:12 +00:00
Dionisio 9aa35aa0ea quick fix 2026-03-06 13:18:36 +01:00
Dionisio f2d35ca792 feat: auto-persist JWT secret, remove setup token requirement
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
2026-03-05 22:12:53 +01:00
Dionisio 9f692f03c3 Implement dual DB pools (primary + maintenance) and wire services 2026-02-24 19:28:00 +01:00
George Wu 2c428678b6 docs: add example.env and document all environment variables
- Create example.env with all 31 environment variables documented
- Update docker-compose.yml to use env_file directive
- Update doc/deployment.md with Quick Start and missing variables
- Add OXICLOUD_BASE_URL and WOPI configuration docs
- Update README.md Docker section with .env setup
- Remove duplicate port mapping (incorporates 4577e56)
- Fix server port default from 8085 to 8086
2026-02-21 18:44:38 -08:00