cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

docs: migrate legacy docs to official site

Browse Source
This commit is contained in:
Diocrafts
2026-04-22 07:50:41 +02:00
parent e10a908f07
commit c0cb86c273
63 changed files with 1646 additions and 10136 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+135 -238
View File
@@ -1,305 +1,202 @@
<p align="center">
<img src="images/oxicloud-logo.png" alt="OxiCloud" width="375" />
<img src="images/oxicloud-logo.png" alt="OxiCloud logo" width="360" />
</p>
<h3 align="center">Self-hosted cloud storage, calendar &amp; contacts — blazingly fast.</h3>
<h1 align="center">OxiCloud</h1>
<div align="center">
<p align="center">
A fast self-hosted cloud for people who want files, calendars, contacts, and office editing without dragging a heavy stack behind them.
</p>
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge)](https://opensource.org/licenses/MIT)
[![Latest Release](https://img.shields.io/github/release/diocrafts/OxiCloud.svg?style=for-the-badge)](https://github.com/diocrafts/OxiCloud/releases)
[![CI](https://img.shields.io/github/actions/workflow/status/diocrafts/OxiCloud/ci.yml?branch=main&style=for-the-badge&label=CI)](https://github.com/diocrafts/OxiCloud/actions/workflows/ci.yml)
[![Rust](https://img.shields.io/badge/Rust-1.93+-orange?style=for-the-badge&logo=rust)](https://www.rust-lang.org/)
[![Docker Image Size](https://img.shields.io/docker/image-size/diocrafts/oxicloud?style=for-the-badge&logo=docker)](https://hub.docker.com/r/diocrafts/oxicloud)
[![GitHub Stars](https://img.shields.io/github/stars/diocrafts/OxiCloud?style=for-the-badge&logo=github)](https://github.com/diocrafts/OxiCloud/stargazers)
[![GitHub Issues](https://img.shields.io/github/issues/diocrafts/OxiCloud?style=for-the-badge)](https://github.com/diocrafts/OxiCloud/issues)
[![Last Commit](https://img.shields.io/github/last-commit/diocrafts/OxiCloud?style=for-the-badge)](https://github.com/diocrafts/OxiCloud/commits/main)
<p align="center">
<a href="https://diocrafts.github.io/OxiCloud/">Documentation</a>
·
<a href="#quick-start">Quick Start</a>
·
<a href="https://github.com/DioCrafts/OxiCloud/stargazers">Star OxiCloud</a>
·
<a href="https://github.com/DioCrafts/OxiCloud/issues/new?template=feature_request.md">Request a Feature</a>
·
<a href="#supported-clients">Supported Clients</a>
·
<a href="#project-status">Project Status</a>
</p>
[**Documentation**](https://diocrafts.github.io/OxiCloud/)
<p align="center">
<a href="https://github.com/diocrafts/OxiCloud/releases"><img src="https://img.shields.io/github/release/diocrafts/OxiCloud.svg?style=for-the-badge" alt="Latest release" /></a>
<a href="https://github.com/diocrafts/OxiCloud/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/diocrafts/OxiCloud/ci.yml?branch=main&style=for-the-badge&label=CI" alt="CI" /></a>
<a href="https://github.com/diocrafts/OxiCloud/stargazers"><img src="https://img.shields.io/github/stars/diocrafts/OxiCloud?style=for-the-badge&logo=github" alt="GitHub stars" /></a>
<a href="https://hub.docker.com/r/diocrafts/oxicloud"><img src="https://img.shields.io/docker/image-size/diocrafts/oxicloud?style=for-the-badge&logo=docker" alt="Docker image size" /></a>
<a href="https://www.rust-lang.org/"><img src="https://img.shields.io/badge/Rust-1.93%2B-orange?style=for-the-badge&logo=rust" alt="Rust 1.93+" /></a>
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT license" /></a>
</p>
<p align="center">
If OxiCloud saves you setup time, RAM, or complexity, <a href="https://github.com/DioCrafts/OxiCloud/stargazers">give it a star</a>.
If something is missing, <a href="https://github.com/DioCrafts/OxiCloud/issues/new?template=feature_request.md">ask for a feature</a> or <a href="https://github.com/DioCrafts/OxiCloud/issues/new?template=documentation_request.md">request a docs improvement</a>.
</p>
NextCloud was too slow on my home server. So I built OxiCloud — a complete cloud platform written in Rust that runs on minimal hardware and stays out of your way.
![OxiCloud dashboard](<images/Captura de pantalla 2025-03-23 230739.png>)
![OxiCloud Dashboard](doc/images/Captura%20de%20pantalla%202025-03-23%20230739.png)
## Why People Try OxiCloud
<!-- TODO: add animated GIF/demo here -->
OxiCloud is aimed at self-hosters, home labs, and small teams who want the useful parts of a cloud suite without the operational drag of a traditional PHP stack.
What pulls people in:
- Standard protocols first: WebDAV, CalDAV, and CardDAV are built in
- Useful product surface already there: files, previews, sharing, trash, search, favorites, and recent items
- Modern auth and admin basics: OIDC/SSO, quotas, roles, and shared links
- Better interoperability: native desktop and mobile clients work without custom sync tooling for basic access
- Lower deployment friction: Docker Compose, environment-based configuration, Helm chart, and Nix module
## OxiCloud vs NextCloud
| Metric | OxiCloud | NextCloud |
|--------|----------|-----------|
| **Language** | Rust (compiled, zero-cost abstractions) | PHP (interpreted) |
| **Docker image** | ~40 MB (Alpine, static binary) | ~1 GB+ (Apache + PHP + modules) |
| **Idle RAM** | ~30–50 MB | ~250–512 MB |
| **Cold start** | < 1 s | 5–15 s |
| **CPU at idle** | ~0 % | 1–5 % (cron, background jobs) |
| **Min. hardware** | 1 vCPU / 512 MB RAM | 2 vCPU / 2 GB RAM (recommended) |
| **Concurrent uploads** | Parallel chunked (TUS-like), async I/O | Sequential PHP workers |
| **File dedup** | SHA-256 content-addressable, ref-counting | None (each user = full copy) |
| **DB connections** | Dual pool (user + maintenance) | Single pool, background jobs compete |
| **LTO + PGO** | Fat LTO, codegen-units=1, opt-level=3 | N/A (interpreted) |
| **Dependencies** | Single binary + PostgreSQL | PHP, Apache/Nginx, Redis, Cron, … |
| **WebDAV** | Built-in (RFC 4918, PROPFIND streaming) | Built-in |
| **CalDAV / CardDAV** | Built-in | Via apps |
| **WOPI (Office editing)** | Built-in (Collabora / OnlyOffice) | Via apps |
| **OIDC / SSO** | Built-in (Keycloak, Authentik, …) | Via apps |
> **Note:** NextCloud is a mature, feature-rich ecosystem. OxiCloud targets users who prioritise raw performance, simplicity, and low resource usage over plugin breadth.
---
## Features
### Storage & Files
- **Upload / download / organise** — drag-and-drop, multi-file, grid & list views
- **Chunked uploads** — TUS-like protocol, parallel chunks, resumable, MD5 integrity
- **File deduplication** — SHA-256 content-addressable blobs with automatic ref-counting
- **Adaptive compression** — zstd / gzip selected per MIME type
- **Trash bin** — soft-delete, restore, auto-purge by retention policy
- **Favourites & recent files**
- **Full-text search** — by name, type, date range, size, recursive subtree (ltree)
- **MIME magic-byte detection** — `infer` crate, not just extension guessing
- **Inline preview** — images, PDF, text, audio & video player modal
- **Thumbnails & transcoding** — WebP / AVIF on-the-fly via `image` crate
### Protocols
- **WebDAV** — RFC 4918, streaming PROPFIND, locking, compatible with all major clients
- **CalDAV** — calendar sync (Thunderbird, GNOME Calendar, iOS, DAVx⁵, …)
- **CardDAV** — contacts sync with vCard support
- **WOPI** — edit Office docs in Collabora Online or OnlyOffice
- **REST API** — complete JSON API for all operations
### Security & Auth
- **JWT authentication** with refresh tokens
- **Argon2id** password hashing
- **OIDC / SSO** — Keycloak, Authentik, Authelia, Google, Azure AD…
- **Role-based access** — admin / user, per-folder permissions
- **Storage quotas** per user
- **Shared links** with optional password protection
### Infrastructure
- **Single binary** — no runtime, no interpreter, no framework overhead
- **~40 MB Docker image** (Alpine)
- **Dual DB pool** — dedicated maintenance pool so background tasks never starve user queries
- **LTO-optimised release** — fat LTO, 1 codegen-unit, `opt-level = 3`, stripped
- **Write-behind caching** (moka) — sub-millisecond hot reads
- **222+ automated tests** — `cargo test` on every push (CI)
- **14 languages** — EN, ES, DE, FR, IT, PT, NL, ZH, JA, KO, AR, HI, FA, RU
---
## Feature Status
| Feature | Status | Notes |
|---------|--------|-------|
| **File storage & upload** | ✅ Working | Chunked uploads, deduplication, thumbnails |
| **WebDAV** | ✅ Working | Fully compatible with major clients |
| **CalDAV (Thunderbird, iOS)** | ✅ Working | Calendar sync functional |
| **CardDAV (Thunderbird, iOS)** | ✅ Working | Contacts sync functional |
| **DAVx⁵ (Android)** | ⚠️ Partial | File sync works; calendar/contacts in progress |
| **WOPI / Office editing** | ✅ Working | Collabora/OnlyOffice integration |
| **OIDC / SSO** | ✅ Working | Keycloak, Authentik, Google, Azure AD |
| **Trash / recycle bin** | ✅ Working | Soft-delete with restore |
| **Full-text search** | ✅ Working | Recursive subtree search |
| **Shared links** | ✅ Working | Optional password protection |
| **Music library & playlists** | ✅ Working | Streaming player, drag-and-drop reorder |
| **Photo gallery** | ✅ Working | Day/month/year views, multi-select |
| **Desktop sync client** | ❌ Planned | Not yet available |
| **Android / iOS app** | ❌ Planned | Not yet available |
| **End-to-end encryption** | ❌ Planned | Roadmap item |
> **Note:** The DAVx⁵ Android app currently works for file sync via WebDAV, but calendar and contacts sync via CalDAV/CardDAV is still being refined. Use Thunderbird or iOS native clients for full CalDAV/CardDAV support.
---
> OxiCloud is not trying to mirror the full plugin ecosystem of Nextcloud. It is designed for a smaller stack, fast startup, and standards-based interoperability.
## Quick Start
### Docker (recommended)
### Docker Compose
Requires Docker and Docker Compose.
```bash
git clone https://github.com/DioCrafts/oxicloud.git
cd oxicloud
# Copy and optionally edit environment
git clone https://github.com/DioCrafts/OxiCloud.git
cd OxiCloud
cp example.env .env
# If users will access OxiCloud through a domain or reverse proxy,
# set OXICLOUD_BASE_URL in .env before the first login.
docker compose up -d
```
Open **http://localhost:8086**. That's it.
Open `http://localhost:8086`.
If you want to access the server remotely set the OXICLOUD_BASE_URL to your server's domain/port or the authentication will fail after setting up the admin user.
### Run from source
### From source
Requires **Rust 1.93+** and **PostgreSQL 13+**.
Requires Rust 1.93+ and PostgreSQL.
```bash
git clone https://github.com/DioCrafts/oxicloud.git
cd oxicloud
git clone https://github.com/DioCrafts/OxiCloud.git
cd OxiCloud
cp example.env .env
# Configure database
echo "DATABASE_URL=postgres://user:pass@localhost/oxicloud" > .env
# Build optimised binary
cargo build --release
# Start the server (migrations run automatically)
cargo run --release
# If PostgreSQL runs on your host instead of Docker, update both
# OXICLOUD_DB_CONNECTION_STRING and DATABASE_URL to use localhost:5432.
cargo run
```
---
Deployment details: [deployment guide](docs/config/deployment.md) · [example.env](example.env)
## Client Setup
## What You Get
OxiCloud speaks standard protocols — any WebDAV / CalDAV / CardDAV client works:
| Area | Included |
| --- | --- |
| Files | Multi-file upload, folders, inline previews, thumbnails, chunked uploads, deduplication, trash |
| Sync and clients | WebDAV, CalDAV, CardDAV, native OS clients, Thunderbird, DAVx5 |
| Security | JWT auth, Argon2id, OIDC/SSO, shared links, quotas, admin/user roles |
| Integrations | REST API and WOPI for Collabora or OnlyOffice |
| Operations | Docker image, Docker Compose, env-driven config, PostgreSQL backend |
| Project tooling | Architecture docs, Helm chart, Nix module, CI |
| Client | Protocol | URL |
|--------|----------|-----|
| Windows Explorer | WebDAV | `http://host:8086/webdav/` |
| macOS Finder | WebDAV | `http://host:8086/webdav/` |
| Nautilus / Dolphin | WebDAV | `dav://host:8086/webdav/` |
| Thunderbird (calendar) | CalDAV | `http://host:8086/caldav/` |
| Thunderbird (contacts) | CardDAV | `http://host:8086/carddav/` |
| DAVx⁵ (Android) | CalDAV + CardDAV | `http://host:8086/` |
| GNOME Calendar | CalDAV | `http://host:8086/caldav/` |
| GNOME Contacts | CardDAV | `http://host:8086/carddav/` |
| Collabora / OnlyOffice | WOPI | See [WOPI docs](doc/wopi-integration.md) |
## Supported Clients
For detailed setup guides: [WebDAV](doc/webdav-integration-guide.md) · [CalDAV](doc/caldav-technical-spec.md) · [CardDAV](doc/carddav-technical-spec.md) · [OIDC/SSO](doc/oidc-integration.md)
OxiCloud uses standard DAV protocols, so it works with native clients instead of requiring a custom sync stack for basic access.
---
| Use case | URL |
| --- | --- |
| Files via WebDAV | `https://your-host/webdav/` |
| Calendars via CalDAV | `https://your-host/caldav/` |
| Contacts via CardDAV | `https://your-host/carddav/` |
## Architecture
Common clients that work well:
Clean / Hexagonal architecture — each layer depends only on the one below:
- macOS Finder
- Windows Explorer
- GNOME Files and KDE Dolphin
- Thunderbird
- Apple Calendar and Contacts
- DAVx5 on Android
```
┌───────────────────────────────────────────────────────────────┐
│ Interfaces │ REST API, WebDAV, CalDAV, CardDAV, WOPI │
├───────────────────────────────────────────────────────────────┤
│ Application │ Use cases, DTOs, port definitions │
├───────────────────────────────────────────────────────────────┤
│ Domain │ Entities, business rules, repository traits │
├───────────────────────────────────────────────────────────────┤
│ Infrastructure│ PostgreSQL, filesystem, caching, auth │
└───────────────────────────────────────────────────────────────┘
```
Client setup guides: [DAV client setup](docs/guide/dav-client-setup.md) · [WebDAV guide](docs/guide/webdav.md) · [CalDAV & CardDAV guide](docs/guide/caldav-carddav.md)
Swap the database, add a new protocol, or change auth — without touching business logic.
## Project Status
For a deep dive: [Internal Architecture](doc/internal-architecture.md) · [Caching](doc/caching-architecture.md) · [DB Transactions](doc/database-transactions.md)
OxiCloud is actively developed and already covers the core self-hosted cloud workflow.
---
| Capability | Status | Notes |
| --- | --- | --- |
| File storage and web UI | Ready | Uploads, previews, sharing, trash, and search |
| WebDAV | Ready | Standard file access for desktop and mobile clients |
| CalDAV and CardDAV | Ready | Working with Thunderbird, Apple clients, and others |
| OIDC / SSO | Ready | Documentation and config examples included |
| WOPI office editing | Ready | Works with Collabora or OnlyOffice |
| DAVx5 Android support | Partial | File sync works well; calendar and contact behavior is still being refined |
| Desktop sync client | Planned | Not yet available |
| Mobile apps | Planned | Not yet available |
| End-to-end encryption | Planned | Roadmap item |
## Configuration
Roadmap: [TODO-LIST.md](TODO-LIST.md)
All config via environment variables (see [`example.env`](example.env)):
## Help Shape OxiCloud
| Variable | Default | Description |
|----------|---------|-------------|
| `OXICLOUD_STORAGE_PATH` | `./storage` | Root directory for file storage |
| `OXICLOUD_SERVER_PORT` | `8086` | HTTP listen port |
| `OXICLOUD_DB_CONNECTION_STRING` | — | PostgreSQL connection string |
| `OXICLOUD_JWT_SECRET` | random | Token signing key (set in production!) |
| `OXICLOUD_OIDC_ENABLED` | `false` | Enable OpenID Connect SSO |
| `OXICLOUD_WOPI_ENABLED` | `false` | Enable Collabora / OnlyOffice editing |
| `OXICLOUD_ENABLE_AUTH` | `true` | Toggle authentication |
| `OXICLOUD_ENABLE_TRASH` | `true` | Toggle trash / recycle bin |
If you want OxiCloud to get better faster, use the repo like a product feedback loop, not just a code dump.
Full reference: [`example.env`](example.env) · [Deployment guide](https://diocrafts.github.io/OxiCloud/config/deployment) · [OIDC setup](https://diocrafts.github.io/OxiCloud/config/oidc)
- Give it a star if you want more people to discover the project: [Star OxiCloud](https://github.com/DioCrafts/OxiCloud/stargazers)
- Propose missing functionality: [feature request](https://github.com/DioCrafts/OxiCloud/issues/new?template=feature_request.md)
- Point out confusing onboarding or weak docs: [documentation request](https://github.com/DioCrafts/OxiCloud/issues/new?template=documentation_request.md)
- Report breakage or regressions: [bug report](https://github.com/DioCrafts/OxiCloud/issues/new?template=bug_report.md)
- Build it with us: [CONTRIBUTING.md](CONTRIBUTING.md)
---
The best feature ideas usually come from real deployment pain. If you hit friction, open an issue and describe the workflow you want.
## Development
## Architecture and Deployment
### Server side
OxiCloud follows a clean, hexagonal architecture so protocol handlers, business logic, and infrastructure stay separated.
```bash
cargo build # Dev build
cargo run # Run locally
cargo test --workspace # 112 tests
cargo clippy -- -D warnings # Lint (zero warnings policy)
cargo fmt --all --check # Format check
RUST_LOG=debug cargo run # Debug logging
```
- Backend: Rust + Axum
- Database: PostgreSQL
- Configuration: environment variables
- Default deployment: Docker Compose
- Additional packaging: [Helm chart](charts/oxicloud) and [Nix module](tools/nix/module.nix)
### Client side
Architecture docs: [internal architecture](docs/architecture/index.md) · [caching architecture](docs/architecture/caching.md) · [database transactions](docs/architecture/database-transactions.md) · [storage safety](docs/architecture/file-system-safety.md)
Run server in dev profile:
## Configuration and Integrations
```bash
PROFILE=dev cargo run
```
Start with [example.env](example.env). The most important settings are:
CSS & JS linter is `biome` (can be installed via `cargo install biome-cli` or on MacOS: `brew install biome`)
- `OXICLOUD_BASE_URL` for reverse proxies, domains, and external access
- `OXICLOUD_DB_CONNECTION_STRING` for PostgreSQL
- `OXICLOUD_OIDC_ENABLED` and related settings for SSO
- `OXICLOUD_WOPI_ENABLED` and discovery URL for office editing
- `MIMALLOC_PURGE_DELAY=0` for lower idle RSS in constrained environments
### Project stats
| Metric | Value |
|--------|-------|
| Rust source files | 170 |
| Lines of code | ~50 000 |
| Automated tests | 222+ |
| Documentation pages | 35+ |
---
Integration docs: [OIDC setup](docs/config/oidc.md) · [OIDC architecture](docs/config/oidc.md) · [OIDC config examples](docs/config/oidc-config-examples.md) · [WOPI integration](docs/config/wopi.md)
## Documentation
📖 **Full documentation:** [**diocrafts.github.io/OxiCloud**](https://diocrafts.github.io/OxiCloud/)
- Docs site: [diocrafts.github.io/OxiCloud](https://diocrafts.github.io/OxiCloud/)
- Deployment: [docs/config/deployment.md](docs/config/deployment.md)
- Batch operations: [docs/guide/batch-operations.md](docs/guide/batch-operations.md)
- Search: [docs/guide/search.md](docs/guide/search.md)
- Thumbnails and transcoding: [docs/guide/thumbnails-and-transcoding.md](docs/guide/thumbnails-and-transcoding.md)
- Deduplication: [docs/guide/deduplication.md](docs/guide/deduplication.md)
| Topic | Online | Source |
|-------|--------|--------|
| Quick Start | [Guide](https://diocrafts.github.io/OxiCloud/guide/installation) | [doc/deployment.md](doc/deployment.md) |
| WebDAV | [Guide](https://diocrafts.github.io/OxiCloud/guide/webdav) | [doc/webdav-integration-guide.md](doc/webdav-integration-guide.md) |
| CalDAV / CardDAV | [Guide](https://diocrafts.github.io/OxiCloud/guide/caldav-carddav) | [doc/caldav-technical-spec.md](doc/caldav-technical-spec.md) |
| OIDC / SSO | [Guide](https://diocrafts.github.io/OxiCloud/config/oidc) | [doc/oidc-integration.md](doc/oidc-integration.md) |
| WOPI (Office) | [Guide](https://diocrafts.github.io/OxiCloud/config/wopi) | [doc/wopi-integration.md](doc/wopi-integration.md) |
| Architecture | [Guide](https://diocrafts.github.io/OxiCloud/architecture/) | [doc/internal-architecture.md](doc/internal-architecture.md) |
| All env variables | [Reference](https://diocrafts.github.io/OxiCloud/config/env) | [`example.env`](example.env) |
## Development
---
```bash
cargo fmt --all --check
cargo clippy -- -D warnings
cargo test --workspace
```
## Roadmap
Check [TODO-LIST.md](TODO-LIST.md) for the full roadmap. Highlights:
- [ ] File versioning & diff viewer
- [ ] End-to-end encryption
- [ ] Desktop sync client (Rust)
- [ ] Android / iOS app
- [ ] OCR & intelligent tagging
- [ ] Automated workflows
---
Contributor guide: [CONTRIBUTING.md](CONTRIBUTING.md)
## Contributing
The project is actively developed. Contributions welcome!
Contributions are welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md) before opening a pull request and [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) for community expectations.
Read [CONTRIBUTING.md](CONTRIBUTING.md) before submitting a PR. Follow the [Code of Conduct](CODE_OF_CONDUCT.md).
If you are not ready to code yet, starring the project and opening a precise feature request is still a meaningful contribution.
## License
MIT — see [LICENSE](LICENSE).
---
## Star History
<div align="center">
<a href="https://star-history.com/#DioCrafts/OxiCloud&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=DioCrafts/OxiCloud&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=DioCrafts/OxiCloud&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=DioCrafts/OxiCloud&type=Date" style="border-radius: 15px; box-shadow: 0 0 30px rgba(0, 217, 255, 0.3);" />
</picture>
</a>
</div>
---
Questions? [Open an issue](https://github.com/DioCrafts/OxiCloud/issues). Want to help? PRs welcome.
MIT. See [LICENSE](LICENSE).
doc/ADMIN-SETTINGS1.md
-162
View File
@@ -1,162 +0,0 @@
# OxiCloud — Admin Settings
## Overview
OxiCloud provides an admin panel API for managing server settings, OIDC configuration, user management, and dashboard statistics. All admin endpoints require a valid JWT token with `role = "admin"`.
## Architecture
| Layer | Component | File |
|---|---|---|
| Domain Port | `SettingsRepository` trait | `src/domain/repositories/settings_repository.rs` |
| Application Service | `AdminSettingsService` | `src/application/services/admin_settings_service.rs` |
| Application DTOs | Settings and user management DTOs | `src/application/dtos/settings_dto.rs` |
| Infrastructure | `SettingsPgRepository` | `src/infrastructure/repositories/pg/settings_pg_repository.rs` |
| Interfaces | `admin_handler` functions | `src/interfaces/api/handlers/admin_handler.rs` |
## REST API
All routes under `/api/admin`, require admin JWT.
### Settings
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/admin/settings/oidc` | `get_oidc_settings` | Get current OIDC configuration |
| `PUT` | `/api/admin/settings/oidc` | `save_oidc_settings` | Update OIDC configuration |
| `POST` | `/api/admin/settings/oidc/test` | `test_oidc_connection` | Test OIDC provider connectivity |
| `GET` | `/api/admin/settings/general` | `get_general_settings` | Get general server settings |
### Dashboard
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/admin/dashboard` | `get_dashboard_stats` | Server dashboard statistics |
### User Management
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/admin/users` | `list_users` | List all users (paginated) |
| `GET` | `/api/admin/users/{id}` | `get_user` | Get user details |
| `DELETE` | `/api/admin/users/{id}` | `delete_user` | Delete a user |
| `PUT` | `/api/admin/users/{id}/role` | `update_user_role` | Change user role |
| `PUT` | `/api/admin/users/{id}/active` | `update_user_active` | Activate/deactivate user |
| `PUT` | `/api/admin/users/{id}/quota` | `update_user_quota` | Set storage quota |
### Safety Guards
- **Self-deletion blocked**: Admins cannot delete their own account
- **Self-role-change blocked**: Admins cannot change their own role
- **Self-deactivation blocked**: Admins cannot deactivate themselves
## OIDC Settings Management
### Get Settings Response
```json
{
"enabled": true,
"issuer_url": "https://keycloak.example.com/realms/main",
"client_id": "oxicloud",
"client_secret_set": true,
"scopes": "openid profile email",
"auto_provision": true,
"admin_groups": "oxicloud-admins",
"disable_password_login": false,
"provider_name": "KeyCloak",
"callback_url": "https://oxicloud.example.com/api/auth/oidc/callback",
"env_overrides": ["issuer_url", "client_id"]
}
```
The `env_overrides` field lists which settings are overridden by environment variables (env vars take priority over DB settings).
### Save Settings Request
```json
{
"enabled": true,
"issuer_url": "https://keycloak.example.com/realms/main",
"client_id": "oxicloud",
"client_secret": "new-secret",
"scopes": "openid profile email",
"auto_provision": true,
"admin_groups": "oxicloud-admins",
"disable_password_login": false,
"provider_name": "KeyCloak"
}
```
After saving, the service hot-reloads OIDC via `auth_app_service.reload_oidc()` or `disable_oidc()`.
### Test OIDC Connection
```json
// Request
{ "issuer_url": "https://keycloak.example.com/realms/main" }
// Response
{
"success": true,
"message": "Successfully connected to OIDC provider",
"issuer": "https://keycloak.example.com/realms/main",
"authorization_endpoint": "https://keycloak.example.com/realms/main/protocol/openid-connect/auth",
"token_endpoint": "https://keycloak.example.com/realms/main/protocol/openid-connect/token",
"userinfo_endpoint": "https://keycloak.example.com/realms/main/protocol/openid-connect/userinfo",
"provider_name_suggestion": "KeyCloak"
}
```
## Dashboard Statistics
```json
{
"server_version": "0.3.2",
"auth_enabled": true,
"oidc_configured": true,
"quotas_enabled": false,
"total_users": 42,
"active_users": 38,
"admin_users": 2,
"total_quota_bytes": 107374182400,
"total_used_bytes": 53687091200,
"storage_usage_percent": 50.0,
"users_over_80_percent": 5,
"users_over_quota": 1
}
```
## User Management DTOs
```rust
pub struct UpdateUserRoleDto { pub role: String } // "user" | "admin"
pub struct UpdateUserActiveDto { pub active: bool }
pub struct UpdateUserQuotaDto { pub quota_bytes: i64 }
pub struct ListUsersQueryDto { pub limit: Option<i64>, pub offset: Option<i64> }
```
## Config Priority
Settings are resolved with the following priority (highest first):
1. **Environment variables** (`OXICLOUD_OIDC_*`)
2. **Database settings** (`auth.admin_settings` table)
3. **Defaults**
## Database Schema
```sql
CREATE TABLE IF NOT EXISTS auth.admin_settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
category TEXT NOT NULL,
is_secret BOOLEAN DEFAULT FALSE,
updated_by VARCHAR(36),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);
```
## Frontend
The admin panel is served from `static/admin.html`.
doc/admin-settings.md
-160
View File
@@ -1,160 +0,0 @@
# 17 - Admin Settings
Admin panel API for managing server settings, OIDC configuration, user management, and dashboard stats. All endpoints under `/api/admin` require a valid JWT with `role = "admin"`.
## Architecture
| Layer | Component | File |
|---|---|---|
| Domain Port | `SettingsRepository` trait | `src/domain/repositories/settings_repository.rs` |
| Application Service | `AdminSettingsService` | `src/application/services/admin_settings_service.rs` |
| Application DTOs | Settings and user management DTOs | `src/application/dtos/settings_dto.rs` |
| Infrastructure | `SettingsPgRepository` | `src/infrastructure/repositories/pg/settings_pg_repository.rs` |
| Interfaces | `admin_handler` functions | `src/interfaces/api/handlers/admin_handler.rs` |
## REST API
All routes under `/api/admin`, require admin JWT.
### Settings
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/admin/settings/oidc` | `get_oidc_settings` | Get current OIDC configuration |
| `PUT` | `/api/admin/settings/oidc` | `save_oidc_settings` | Update OIDC configuration |
| `POST` | `/api/admin/settings/oidc/test` | `test_oidc_connection` | Test OIDC provider connectivity |
| `GET` | `/api/admin/settings/general` | `get_general_settings` | Get general server settings |
### Dashboard
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/admin/dashboard` | `get_dashboard_stats` | Server dashboard statistics |
### User Management
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/admin/users` | `list_users` | List all users (paginated) |
| `GET` | `/api/admin/users/{id}` | `get_user` | Get user details |
| `DELETE` | `/api/admin/users/{id}` | `delete_user` | Delete a user |
| `PUT` | `/api/admin/users/{id}/role` | `update_user_role` | Change user role |
| `PUT` | `/api/admin/users/{id}/active` | `update_user_active` | Activate/deactivate user |
| `PUT` | `/api/admin/users/{id}/quota` | `update_user_quota` | Set storage quota |
### Safety Guards
- **Self-deletion blocked** -- admins cannot delete their own account
- **Self-role-change blocked** -- admins cannot change their own role
- **Self-deactivation blocked** -- admins cannot deactivate themselves
## OIDC Settings Management
### Get Settings Response
```json
{
"enabled": true,
"issuer_url": "https://keycloak.example.com/realms/main",
"client_id": "oxicloud",
"client_secret_set": true,
"scopes": "openid profile email",
"auto_provision": true,
"admin_groups": "oxicloud-admins",
"disable_password_login": false,
"provider_name": "KeyCloak",
"callback_url": "https://oxicloud.example.com/api/auth/oidc/callback",
"env_overrides": ["issuer_url", "client_id"]
}
```
The **env_overrides** field lists which settings are overridden by environment variables. Env vars always take priority over DB settings.
### Save Settings Request
```json
{
"enabled": true,
"issuer_url": "https://keycloak.example.com/realms/main",
"client_id": "oxicloud",
"client_secret": "new-secret",
"scopes": "openid profile email",
"auto_provision": true,
"admin_groups": "oxicloud-admins",
"disable_password_login": false,
"provider_name": "KeyCloak"
}
```
After saving, the service hot-reloads OIDC via **auth_app_service.reload_oidc()** or **disable_oidc()**.
### Test OIDC Connection
```json
// Request
{ "issuer_url": "https://keycloak.example.com/realms/main" }
// Response
{
"success": true,
"message": "Successfully connected to OIDC provider",
"issuer": "https://keycloak.example.com/realms/main",
"authorization_endpoint": "https://keycloak.example.com/realms/main/protocol/openid-connect/auth",
"token_endpoint": "https://keycloak.example.com/realms/main/protocol/openid-connect/token",
"userinfo_endpoint": "https://keycloak.example.com/realms/main/protocol/openid-connect/userinfo",
"provider_name_suggestion": "KeyCloak"
}
```
## Dashboard Statistics
```json
{
"server_version": "0.3.1",
"auth_enabled": true,
"oidc_configured": true,
"quotas_enabled": false,
"total_users": 42,
"active_users": 38,
"admin_users": 2,
"total_quota_bytes": 107374182400,
"total_used_bytes": 53687091200,
"storage_usage_percent": 50.0,
"users_over_80_percent": 5,
"users_over_quota": 1
}
```
## User Management DTOs
```rust
pub struct UpdateUserRoleDto { pub role: String } // "user" | "admin"
pub struct UpdateUserActiveDto { pub active: bool }
pub struct UpdateUserQuotaDto { pub quota_bytes: i64 }
pub struct ListUsersQueryDto { pub limit: Option<i64>, pub offset: Option<i64> }
```
## Config Priority
Settings resolve in this order (highest first):
1. **Environment variables** (`OXICLOUD_OIDC_*`)
2. **Database settings** (`auth.admin_settings` table)
3. **Defaults**
## Database Schema
```sql
CREATE TABLE IF NOT EXISTS auth.admin_settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
category TEXT NOT NULL,
is_secret BOOLEAN DEFAULT FALSE,
updated_by VARCHAR(36),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);
```
## Frontend
The admin panel is served from `static/admin.html`.
doc/batch-operations.md
-140
View File
@@ -1,140 +0,0 @@
# 09 - Batch Operations
Batch endpoints let you perform bulk file and folder operations (move, copy, delete, get, create) in a single request. Operations run concurrently behind a configurable semaphore. Each response includes detailed per-item success/failure info.
## Architecture
| Layer | Component | File |
|---|---|---|
| Application Service | **BatchOperationService** | `src/application/services/batch_operations.rs` |
| Interfaces | `batch_handler` functions | `src/interfaces/api/handlers/batch_handler.rs` |
## REST API
All routes live under `/api/batch` and require authentication.
### File Operations
| Method | Path | Handler | Description |
|---|---|---|---|
| `POST` | `/api/batch/files/move` | `move_files_batch` | Move files to target folder |
| `POST` | `/api/batch/files/copy` | `copy_files_batch` | Copy files to target folder |
| `POST` | `/api/batch/files/delete` | `delete_files_batch` | Delete multiple files |
| `POST` | `/api/batch/files/get` | `get_files_batch` | Get metadata for multiple files |
### Folder Operations
| Method | Path | Handler | Description |
|---|---|---|---|
| `POST` | `/api/batch/folders/delete` | `delete_folders_batch` | Delete multiple folders |
| `POST` | `/api/batch/folders/create` | `create_folders_batch` | Create multiple folders |
| `POST` | `/api/batch/folders/get` | `get_folders_batch` | Get metadata for multiple folders |
## Request DTOs
### File Operations
```json
{
"file_ids": ["id-1", "id-2", "id-3"],
"target_folder_id": "folder-abc" // required for move/copy
}
```
### Folder Delete
```json
{
"folder_ids": ["folder-1", "folder-2"],
"recursive": true,
"target_folder_id": null
}
```
### Folder Create
```json
{
"folders": [
{ "name": "Documents", "parent_id": null },
{ "name": "Photos", "parent_id": "folder-abc" }
]
}
```
## Response Format
Every batch endpoint returns the same structure:
```json
{
"successful": [ ... ],
"failed": [
{ "id": "bad-id", "error": "File not found" }
],
"stats": {
"total": 5,
"successful": 4,
"failed": 1,
"execution_time_ms": 245
}
}
```
### Status Codes
| Code | Meaning |
|---|---|
| `200 OK` / `201 Created` | All operations succeeded |
| `206 Partial Content` | Some succeeded, some failed |
| `400 Bad Request` | All operations failed |
## Concurrency
Operations run concurrently via `tokio::sync::Semaphore`, capped by **max_concurrent_files** (default: 10).
```rust
pub struct BatchOperationService {
file_retrieval: Arc<dyn FileRetrievalUseCase>,
file_management: Arc<dyn FileManagementUseCase>,
folder_service: Arc<FolderService>,
config: AppConfig,
semaphore: Arc<Semaphore>,
}
```
## Error Handling
```rust
pub enum BatchOperationError {
Domain(DomainError), // individual operation error
Cancelled(String), // operation was cancelled
ConcurrencyLimit(String), // semaphore exhausted
PartialFailure(String, usize, usize), // message, success_count, fail_count
Internal(String),
}
```
Individual failures do not abort the batch. They get collected in the `failed` array and the response returns `206 Partial Content`.
## Example
```bash
# Move 3 files to a folder
curl -X POST -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"file_ids":["id-1","id-2","id-3"],"target_folder_id":"folder-abc"}' \
"https://oxicloud.example.com/api/batch/files/move"
# Delete folders recursively
curl -X POST -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"folder_ids":["old-1","old-2"],"recursive":true}' \
"https://oxicloud.example.com/api/batch/folders/delete"
# Create multiple folders
curl -X POST -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"folders":[{"name":"Docs","parent_id":null},{"name":"Photos","parent_id":"root-id"}]}' \
"https://oxicloud.example.com/api/batch/folders/create"
```
doc/caching-architecture.md
-274
View File
@@ -1,274 +0,0 @@
# 04 - Caching Architecture
OxiCloud uses a multi-layer caching system spanning HTTP-level caching down to kernel-level memory mapping. Covers both uploads and downloads.
## Cache Layers Summary
```
┌─────────────────────────────────────────────────────┐
│ Layer 0: HTTP Cache Middleware (ETag + 304) │ All endpoints
├─────────────────────────────────────────────────────┤
│ Layer 1: File Content Cache (LRU, <10MB files) │ Downloads
├─────────────────────────────────────────────────────┤
│ Layer 2: MMAP (memmap2, 10-100MB blobs) │ Downloads
├─────────────────────────────────────────────────────┤
│ Layer 3: Streaming (FramedRead, ≥100MB blobs) │ Downloads
├─────────────────────────────────────────────────────┤
│ Layer 4: Buffer Pool (reusable I/O buffers) │ Compression
└─────────────────────────────────────────────────────┘
```
> **Note:** File metadata (name, size, MIME type, folder) is served from PostgreSQL — no separate filesystem metadata cache is needed.
---
## Layer 0: HTTP Cache Middleware
**File**: `src/interfaces/middleware/cache.rs`
Generic HTTP caching layer applied to API endpoints.
| Parameter | Value |
|---|---|
| Max entries | 1,000 |
| Default max-age | 60 seconds |
| Eviction | LRU (oldest 10% when full) |
| Cleanup | Background task every 5 minutes |
Features:
- ETag-based conditional requests (`If-None-Match` → `304 Not Modified`)
- `Cache-Control` header injection
- Implements Tower `Layer` + `Service` traits for Axum integration
- Per-request key: method + URI
---
## Layer 1: File Content Cache (Download Tier 1)
**File**: `src/infrastructure/services/file_content_cache.rs`
In-memory LRU cache for small files, served directly from RAM.
| Parameter | Value |
|---|---|
| Max file size | 10 MB per file |
| Max total cache size | 512 MB |
| Max entries | 10,000 |
| Structure | `lru::LruCache<String, CacheEntry>` |
| Latency | ~0.1ms |
**CacheEntry**: `{ data: Bytes, etag: String, content_type: String, size: usize }`
Methods:
- `should_cache(size)` — checks if file fits in cache
- `get(file_id)` → `Option<(Bytes, String, String)>` — returns (data, etag, content_type)
- `put(file_id, content, etag, content_type)` — inserts with LRU eviction
- `invalidate(file_id)`, `clear()`
- `stats()` → `CacheStats { current_size_bytes, max_size_bytes, hits, misses, hit_rate_percent }`
Port: implements **ContentCachePort** trait.
---
## Layer 2: MMAP (Download Tier 2)
**File**: `src/infrastructure/repositories/pg/file_blob_read_repository.rs`
Memory-mapped I/O for medium blobs using `memmap2`.
| Parameter | Value |
|---|---|
| File range | 10 MB - 100 MB |
| Implementation | `memmap2::Mmap` via `spawn_blocking` |
| Latency | ~1-5ms |
The blob file (`.blobs/{prefix}/{hash}.blob`) is memory-mapped and its contents copied to `Bytes`. Benefits from kernel page cache for frequently accessed blobs.
---
## Layer 3: Streaming (Download Tier 3)
**File**: `src/infrastructure/repositories/pg/file_blob_read_repository.rs`
Chunked streaming for large blobs using tokio-util codecs.
| Parameter | Value |
|---|---|
| File range | ≥100 MB |
| Chunk size | 1 MB (configurable via **ResourceConfig.chunk_size_bytes**) |
| Implementation | `FramedRead` + `BytesCodec` |
| RAM usage | Near zero (one chunk at a time) |
---
## Layer 4: Buffer Pool
**File**: `src/infrastructure/services/buffer_pool.rs`
Reusable byte buffer pool to reduce allocation pressure during compression operations.
| Parameter | Value |
|---|---|
| Buffer size | 64 KB |
| Max buffers | 100 |
| Buffer TTL | 60 seconds |
| Concurrency control | `tokio::sync::Semaphore` |
Features:
- `get_buffer()` — borrows a buffer (blocks if pool exhausted)
- **BorrowedBuffer** auto-returns to pool on `Drop` via `tokio::spawn`
- Expired buffers are cleaned periodically via `start_cleaner()`
- Tracks stats: gets, hits, misses, returns, evictions, waits
---
## Configuration
All cache-related config in `src/common/config.rs`:
```rust
pub struct ResourceConfig {
pub large_file_threshold_mb: u64, // 100 MB (mmap→streaming boundary)
pub chunk_size_bytes: usize, // 1 MB (streaming chunk size)
pub max_in_memory_file_size_mb: u64, // 50 MB
}
```
## Download Flow
```
Request → ETag check (304?) → Range request (206?)
→ file size < 10MB? → Tier 1: LRU cache (RAM)
→ file size < 100MB? → Tier 2: MMAP (kernel page cache on blob file)
→ file size ≥ 100MB → Tier 3: Streaming (chunked from blob file)
```
In all tiers, metadata (file name, size, MIME type) comes from a PostgreSQL `SELECT` on `storage.files`. Content is read from the DedupService blob at `.blobs/{prefix}/{hash}.blob`.
---
## Range Requests (HTTP 206 Partial Content)
**Files**: `src/interfaces/api/handlers/file_handler.rs`, `src/infrastructure/repositories/pg/file_blob_read_repository.rs`
**Crate**: `http-range-header = "0.4"` for parsing.
### Request Processing Flow
```
Range header present?
├─ parse_range_header(range_str)
│ ├─ Parse OK → ranges.validate(file_size)
│ │ ├─ Valid → take first range → get_file_range_stream(start, end+1)
│ │ │ ├─ Stream OK → 206 Partial Content
│ │ │ └─ Stream Err → fall through to normal download (200)
│ │ └─ Invalid → 416 Range Not Satisfiable
│ └─ Parse Err → fall through to normal download (200)
└─ No Range header → normal 3-tier download
```
### Response Headers (206)
| Header | Value |
|---|---|
| `Content-Type` | File MIME type |
| `Content-Range` | `bytes {start}-{end}/{total_size}` |
| `Content-Length` | Range length (end - start + 1) |
| `Accept-Ranges` | `bytes` |
| `ETag` | `"{file_id}-{modified_at}"` |
| `Cache-Control` | `private, max-age=3600, must-revalidate` |
### 416 Range Not Satisfiable
Returned when `ranges.validate(file_size)` fails:
```
HTTP/1.1 416 Range Not Satisfiable
Content-Range: bytes */12345
```
### Blob File Seek Implementation
`get_file_range_stream()` at the repository level:
1. Resolves blob path from `blob_hash` via DedupService
2. Opens the blob file with `TokioFile::open()`
3. Seeks to `start` via `fh.seek(SeekFrom::Start(start))`
4. Limits read to `range_length` via `fh.take(range_length)`
5. Wraps in `FramedRead` + `BytesCodec`
Adaptive chunk size:
| Range size | Chunk size |
|---|---|
| ≤ 1 MB | 8 KB |
| > 1 MB | 1 MB (from **ResourceConfig.chunk_size_bytes**) |
### Tier Interaction
Range requests **bypass all download tiers** (LRU, MMAP). They always use direct blob file seek + streaming. On stream creation error, the handler falls through to the normal `get_file_optimized()` 3-tier path.
### Limitations
- **Multipart ranges not supported**: only the first range in a multi-range request is served.
- **`If-Range` not handled**: no conditional range support.
- **`If-Modified-Since` not handled**: only `If-None-Match` (ETag) is checked.
---
## Upload Flow
```
Request → file size < 1MB? → Buffered write (sync to blob store)
→ file size ≥ 1MB → Streaming write (chunk-by-chunk to blob store)
```
### Upload Strategy Selection
**File**: `src/application/services/file_upload_service.rs`
```rust
pub enum UploadStrategy {
Buffered, // < 1 MB — collect bytes, write to blob store
Streaming, // ≥ 1 MB — chunk-by-chunk write via save_file_from_stream
}
```
| Constant | Value |
|---|---|
| `STREAMING_UPLOAD_THRESHOLD` | 1 MB |
### Handler-Level Buffering
Upload handlers buffer the multipart body in RAM as `Vec<Bytes>` before calling the service layer:
```rust
let mut chunks: Vec<Bytes> = Vec::new();
while let Some(chunk) = field.chunk().await {
chunks.push(chunk);
}
upload_service.smart_upload(..., chunks, total_size).await
```
### Buffered Path (< 1 MB)
Uses `save_file()` — all bytes are passed to `FileBlobWriteRepository`, which calls `DedupService.store_bytes()` to compute hash and store the blob, then INSERTs metadata into `storage.files`.
### Streaming Path (≥ 1 MB)
`smart_upload()` converts the in-memory `Vec<Bytes>` into a `futures::stream::iter()` and passes it to `save_file_from_stream()`:
```rust
let chunk_stream = stream::iter(chunks.into_iter().map(|c| Ok(c)));
self.file_write.save_file_from_stream(name, folder_id, content_type, chunk_stream).await
```
`FileBlobWriteRepository.save_file_from_stream()` collects the stream, stores via DedupService, and INSERTs metadata.
### Dedup Integration
Deduplication is handled at the **repository layer** (not the service layer) for all upload strategies. `FileBlobWriteRepository` always calls `DedupService.store_bytes()` which:
1. Computes SHA-256 hash of content
2. Checks if blob already exists (dedup hit → increment ref count, skip write)
3. If new → atomic write to `.blobs/{prefix}/{hash}.blob`
4. Returns the hash for storage in `storage.files.blob_hash`
doc/caldav-technical-spec.md
-239
View File
@@ -1,239 +0,0 @@
# 26 - CalDAV Technical Spec
CalDAV (RFC 4791) provides calendar synchronization. Clients like Thunderbird, Apple Calendar, GNOME Calendar, and DAVx5 (Android) connect to manage calendars and events via standard CalDAV.
## Protocol Compliance
- **DAV compliance**: `1, 2, calendar-access`
- **RFC 4791**: Calendar Access (CalDAV)
- **iCalendar**: RFC 5545 (VEVENT parsing and generation)
## Endpoint Structure
All CalDAV endpoints are mounted at the top level (not under `/api`):
```
ANY /caldav → handle_caldav_methods_root
ANY /caldav/ → handle_caldav_methods_root
ANY /caldav/{*path} → handle_caldav_methods
```
### Path Hierarchy
| Path | Resource | Description |
|---|---|---|
| `/caldav/` | Calendar home | User's calendar collection |
| `/caldav/{calendar_id}/` | Calendar | Individual calendar |
| `/caldav/{calendar_id}/{ical_uid}.ics` | Event | Individual calendar event |
### Supported HTTP Methods
| Method | Description |
|---|---|
| `OPTIONS` | Returns DAV capabilities and allowed methods |
| `PROPFIND` | List calendars, calendar properties, events |
| `REPORT` | CalendarQuery, CalendarMultiget, SyncCollection |
| `MKCALENDAR` | Create a new calendar |
| `PUT` | Create/update events (iCalendar format) |
| `GET` | Retrieve individual event as iCalendar |
| `DELETE` | Delete calendars or events |
| `PROPPATCH` | Update calendar properties |
## Architecture
| Layer | Component | File |
|---|---|---|
| Domain Entity | **Calendar**, **CalendarEvent** | `src/domain/entities/calendar.rs`, `calendar_event.rs` |
| Domain Repository | **CalendarRepository**, **CalendarEventRepository** | `src/domain/repositories/calendar_repository.rs`, `calendar_event_repository.rs` |
| Application Port | **CalendarUseCase**, **CalendarStoragePort** | `src/application/ports/calendar_ports.rs` |
| Application Service | **CalendarService** | `src/application/services/calendar_service.rs` |
| Application Adapter | **CalDavAdapter** (XML parsing/generation) | `src/application/adapters/caldav_adapter.rs` |
| Infrastructure | **CalendarPgRepository**, **CalendarEventPgRepository** | `src/infrastructure/repositories/pg/` |
| Interfaces | **CalDavHandler** | `src/interfaces/api/handlers/caldav_handler.rs` |
## Domain Entities
### Calendar
```rust
pub struct Calendar {
id: Uuid,
name: String,
owner_id: String,
description: Option<String>,
color: Option<String>, // #RRGGBB format
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
custom_properties: HashMap<String, String>,
}
```
Validation: **name** non-empty, **owner_id** non-empty, **color** must be `#RRGGBB` hex format.
### CalendarEvent
```rust
pub struct CalendarEvent {
id: Uuid,
calendar_id: Uuid,
summary: String,
description: Option<String>,
location: Option<String>,
start_time: DateTime<Utc>,
end_time: DateTime<Utc>,
all_day: bool,
rrule: Option<String>, // iCal RRULE format (e.g., "FREQ=WEEKLY")
ical_uid: String, // unique iCal identifier
ical_data: String, // full VEVENT block
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
```
Validation: **summary** non-empty, `end_time >= start_time`, **rrule** starts with `FREQ=`, **ical_data** contains `BEGIN:VEVENT`/`END:VEVENT`.
The constructor `from_ical(calendar_id, ical_data)` parses iCalendar data to extract summary, dates, location, etc.
## REPORT Types
The CalDAV **REPORT** method supports three report types:
### CalendarQuery
Filters events by time range. Used for initial sync and view rendering.
```xml
<C:calendar-query xmlns:C="urn:ietf:params:xml:ns:caldav">
<D:prop xmlns:D="DAV:">
<D:getetag/>
<C:calendar-data/>
</D:prop>
<C:filter>
<C:comp-filter name="VCALENDAR">
<C:comp-filter name="VEVENT">
<C:time-range start="20240101T000000Z" end="20240201T000000Z"/>
</C:comp-filter>
</C:comp-filter>
</C:filter>
</C:calendar-query>
```
### CalendarMultiget
Fetches specific events by href. Used for selective sync.
```xml
<C:calendar-multiget xmlns:C="urn:ietf:params:xml:ns:caldav">
<D:prop xmlns:D="DAV:">
<D:getetag/>
<C:calendar-data/>
</D:prop>
<D:href>/caldav/cal-1/event-1.ics</D:href>
<D:href>/caldav/cal-1/event-2.ics</D:href>
</C:calendar-multiget>
```
### SyncCollection
Incremental sync using sync tokens. Used for ongoing synchronization.
```xml
<D:sync-collection xmlns:D="DAV:">
<D:sync-token>sync-token-value</D:sync-token>
<D:prop>
<D:getetag/>
</D:prop>
</D:sync-collection>
```
## XML Namespaces
| Prefix | Namespace |
|---|---|
| `D:` | `DAV:` |
| `C:` | `urn:ietf:params:xml:ns:caldav` |
| `CS:` | `http://calendarserver.org/ns/` |
## CalDAV Adapter
**CalDavAdapter** in `src/application/adapters/caldav_adapter.rs` handles all XML parsing and generation:
- `parse_report(reader)` -> **CalDavReportType**
- `parse_mkcalendar(reader)` -> `(name, description, color)`
- `generate_calendars_propfind_response(...)` -- multi-calendar PROPFIND
- `generate_calendar_collection_propfind(...)` -- single calendar with events
- `generate_calendar_events_response(...)` -- REPORT response
## Calendar Sharing
Calendars support sharing with access levels:
| Level | Permissions |
|---|---|
| `read` | View calendar and events |
| `write` | Create, modify, delete events |
| `owner` | Full control including sharing and deletion |
## Database Schema
```sql
CREATE SCHEMA IF NOT EXISTS caldav;
-- Calendars
CREATE TABLE caldav.calendars (
id UUID PRIMARY KEY,
name TEXT NOT NULL,
owner_id VARCHAR(36) REFERENCES auth.users(id) ON DELETE CASCADE,
description TEXT,
color VARCHAR(9),
is_public BOOLEAN DEFAULT FALSE,
ctag VARCHAR(64) DEFAULT '0',
created_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
);
-- Calendar Events
CREATE TABLE caldav.calendar_events (
id UUID PRIMARY KEY,
calendar_id UUID REFERENCES caldav.calendars(id) ON DELETE CASCADE,
summary TEXT NOT NULL,
description TEXT,
location TEXT,
start_time TIMESTAMPTZ NOT NULL,
end_time TIMESTAMPTZ NOT NULL,
all_day BOOLEAN DEFAULT FALSE,
rrule TEXT,
ical_uid VARCHAR(255) NOT NULL,
ical_data TEXT,
etag VARCHAR(64),
created_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
);
-- Calendar Shares
CREATE TABLE caldav.calendar_shares (
id SERIAL PRIMARY KEY,
calendar_id UUID REFERENCES caldav.calendars(id) ON DELETE CASCADE,
user_id VARCHAR(36) REFERENCES auth.users(id) ON DELETE CASCADE,
access_level VARCHAR(10) DEFAULT 'read',
created_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP,
UNIQUE(calendar_id, user_id)
);
-- Calendar Properties
CREATE TABLE caldav.calendar_properties (
id SERIAL PRIMARY KEY,
calendar_id UUID REFERENCES caldav.calendars(id) ON DELETE CASCADE,
property_name TEXT NOT NULL,
property_value TEXT NOT NULL,
UNIQUE(calendar_id, property_name)
);
```
Indexes: `idx_calendars_owner_id`, `idx_calendar_events_calendar_id`, `idx_calendar_events_ical_uid`, `idx_calendar_events_time_range (start_time, end_time)`.
## Client Configuration
See `dav-client-setup.md` for client-specific configuration instructions.
CalDAV URL: `https://oxicloud.example.com/caldav/`
doc/carddav-implementation-plan.md
-215
View File
@@ -1,215 +0,0 @@
# 28 - CardDAV Implementation Plan
CardDAV is an open protocol for syncing address books and contacts across applications and devices. This plan breaks the implementation into five phases over five weeks.
## Implementation Roadmap
### Phase 1: Core Infrastructure (Week 1)
**Database Schema** -- new migration for CardDAV tables:
- **address_books** -- stores address book collections
- **contacts** -- stores contact data
- **address_book_shares** -- sharing between users
- **contact_groups** -- organizing contacts into groups
- **group_memberships** -- associating contacts with groups
**Domain Layer** -- entity models:
- **Contact** -- core contact entity
- **AddressBook** -- collection entity
- **ContactGroup** -- grouping contacts
Repository interfaces:
- **ContactRepository** -- contact CRUD
- **AddressBookRepository** -- address book management
- **ContactGroupRepository** -- group management
**Testing** -- unit tests for entity models and repository interface contract tests.
### Phase 2: Infrastructure Layer (Week 2)
**Repository Implementations** -- PostgreSQL:
- **ContactPgRepository**
- **AddressBookPgRepository**
- **ContactGroupPgRepository**
Also implement vCard parsing/generation utilities and any data migration tools needed.
**Integration** -- update DI system to include new repositories and connect with existing auth system.
**Testing** -- repository implementation tests, vCard parsing/generation tests, integration tests with the database.
### Phase 3: Application Layer (Week 3)
**Services** -- business logic:
- **ContactService** -- contact management
- **AddressBookService** -- address book management
- **ContactGroupService** -- group management
**DTOs and Ports** -- create DTOs for contact operations, define service interface ports, implement request/response mapping.
**CardDAV Adapter** -- protocol translation adapter with vCard conversion logic and XML parsing/generation utilities.
**Testing** -- service unit tests and integration tests for the adapter.
### Phase 4: Interface Layer (Week 4)
**REST API** -- endpoints for address book operations, contact management, contact groups. Document with OpenAPI.
**CardDAV Protocol Endpoints** -- WebDAV method handlers:
- PROPFIND -- discovery and property retrieval
- REPORT -- querying contacts
- MKCOL -- creating address books
- GET/PUT/DELETE -- contact operations
- CardDAV-specific XML handling
**Integration** -- connect all layers, end-to-end testing, test with various CardDAV clients.
**Testing** -- API endpoint tests, CardDAV protocol compliance tests, client compatibility tests.
### Phase 5: Refinement and Optimization (Week 5)
**Performance** -- caching for frequently accessed resources, query optimization, efficient sync mechanisms.
**Security** -- review auth, validate I/O, add rate limiting.
**Final Testing** -- stress testing with large address books, security testing, user acceptance testing.
**Documentation** -- update API docs, user guides, client setup procedures.
## Technical Specifications
### Database Schema
```sql
-- Address books table
CREATE TABLE IF NOT EXISTS carddav.address_books (
id UUID PRIMARY KEY,
name VARCHAR(255) NOT NULL,
owner_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
description TEXT,
color VARCHAR(50),
is_public BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(owner_id, name)
);
-- Address book sharing
CREATE TABLE IF NOT EXISTS carddav.address_book_shares (
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
can_write BOOLEAN NOT NULL DEFAULT FALSE,
PRIMARY KEY(address_book_id, user_id)
);
-- Contacts table
CREATE TABLE IF NOT EXISTS carddav.contacts (
id UUID PRIMARY KEY,
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
uid VARCHAR(255) NOT NULL,
full_name VARCHAR(255),
first_name VARCHAR(255),
last_name VARCHAR(255),
nickname VARCHAR(255),
email JSONB,
phone JSONB,
address JSONB,
organization VARCHAR(255),
title VARCHAR(255),
notes TEXT,
photo_url TEXT,
birthday DATE,
anniversary DATE,
vcard TEXT NOT NULL,
etag VARCHAR(255) NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(address_book_id, uid)
);
-- Contact groups
CREATE TABLE IF NOT EXISTS carddav.contact_groups (
id UUID PRIMARY KEY,
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
name VARCHAR(255) NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);
-- Group memberships
CREATE TABLE IF NOT EXISTS carddav.group_memberships (
group_id UUID NOT NULL REFERENCES carddav.contact_groups(id) ON DELETE CASCADE,
contact_id UUID NOT NULL REFERENCES carddav.contacts(id) ON DELETE CASCADE,
PRIMARY KEY(group_id, contact_id)
);
```
### API Endpoints
#### REST API
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/address-books` | List all address books |
| POST | `/api/address-books` | Create a new address book |
| GET | `/api/address-books/:id` | Get a specific address book |
| PUT | `/api/address-books/:id` | Update an address book |
| DELETE | `/api/address-books/:id` | Delete an address book |
| GET | `/api/address-books/:id/contacts` | List contacts in an address book |
| POST | `/api/address-books/:id/contacts` | Create a new contact |
| GET | `/api/address-books/:id/contacts/:contactId` | Get a specific contact |
| PUT | `/api/address-books/:id/contacts/:contactId` | Update a contact |
| DELETE | `/api/address-books/:id/contacts/:contactId` | Delete a contact |
#### CardDAV Protocol Endpoints
| Method | Endpoint | Description |
|--------|----------|-------------|
| PROPFIND | `/carddav/` | List all address books |
| PROPFIND | `/carddav/:addressBookId/` | Get address book information |
| REPORT | `/carddav/:addressBookId/` | Query contacts in an address book |
| GET | `/carddav/:addressBookId/:contactId.vcf` | Get a specific contact (vCard) |
| PUT | `/carddav/:addressBookId/:contactId.vcf` | Create or update a contact |
| DELETE | `/carddav/:addressBookId/:contactId.vcf` | Delete a contact |
### Dependencies
- vCard parsing/generation library (e.g., `vcard-rs` or similar)
- XML processing (for CardDAV protocol)
- Database access (PostgreSQL)
- WebDAV base functionality
## Resources Required
- Developer time: 1 full-time developer for 5 weeks
- Testing resources: multiple CardDAV clients (Apple Contacts, Thunderbird, Android)
- Server resources: test environment with PostgreSQL
## Success Criteria
1. Users can create, update, and delete address books
2. Contacts can be managed within address books
3. Address books can be shared between users
4. Standard CardDAV clients can sync with the server
5. Acceptable performance with large address books (1000+ contacts)
6. Security measures properly implemented
## Client Setup Guides
After implementation, setup guides will be created for:
- Apple Contacts (macOS/iOS)
- Thunderbird/Evolution
- Android (using DAVx5)
- Other common CardDAV clients
See `dav-client-setup.md` for general DAV client configuration.
## Future Enhancements
1. Advanced contact search
2. Contact merging for duplicate detection
3. Bulk import/export
4. Contact photo management
5. Extended fields for specialized contact info
6. Integration with external systems (e.g., LDAP directories)
doc/carddav-technical-spec.md
-343
View File
@@ -1,343 +0,0 @@
# 27 - CardDAV Technical Spec
CardDAV (RFC 6352) enables contact synchronization across devices and applications. It extends WebDAV (RFC 4918) to manage address books and vCard-formatted contacts.
## Architecture
The CardDAV implementation follows the hexagonal architecture pattern:
```
┌───────────────────┐ ┌────────────────────┐ ┌────────────────────┐
│ │ │ │ │ │
│ Interfaces │ │ Application │ │ Infrastructure │
│ - CardDAV API │────▶│ - Contact Service │────▶│ - Contact Repo │
│ - Contact API │ │ - CardDAV Adapter │ │ - PG Repository │
│ │ │ │ │ │
└───────────────────┘ └────────────────────┘ └────────────────────┘
│
▼
┌────────────────────┐
│ │
│ Domain │
│ - Contact Entity │
│ - Address Book │
│ │
└────────────────────┘
```
### Components
1. **Domain Layer**
- **Contact** entity -- name, email, phone, etc.
- **AddressBook** entity -- a collection of contacts
- Repository interfaces for contact management
2. **Application Layer**
- **ContactService** -- business logic for managing contacts
- **CardDAVAdapter** -- converts between CardDAV protocol requests/responses and domain objects
3. **Infrastructure Layer**
- **ContactPgRepository** -- PostgreSQL implementation of contact repositories
- **AddressBookPgRepository** -- PostgreSQL implementation of address book repositories
4. **Interface Layer**
- REST API endpoints for contact management
- CardDAV protocol endpoints (WebDAV extension)
## Database Schema
```sql
-- Address books table
CREATE TABLE IF NOT EXISTS carddav.address_books (
id UUID PRIMARY KEY,
name VARCHAR(255) NOT NULL,
owner_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
description TEXT,
color VARCHAR(50),
is_public BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(owner_id, name)
);
-- Address book sharing
CREATE TABLE IF NOT EXISTS carddav.address_book_shares (
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
can_write BOOLEAN NOT NULL DEFAULT FALSE,
PRIMARY KEY(address_book_id, user_id)
);
-- Contacts table
CREATE TABLE IF NOT EXISTS carddav.contacts (
id UUID PRIMARY KEY,
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
uid VARCHAR(255) NOT NULL,
full_name VARCHAR(255),
first_name VARCHAR(255),
last_name VARCHAR(255),
nickname VARCHAR(255),
email JSONB,
phone JSONB,
address JSONB,
organization VARCHAR(255),
title VARCHAR(255),
notes TEXT,
photo_url TEXT,
birthday DATE,
anniversary DATE,
vcard TEXT NOT NULL,
etag VARCHAR(255) NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(address_book_id, uid)
);
-- Contact groups
CREATE TABLE IF NOT EXISTS carddav.contact_groups (
id UUID PRIMARY KEY,
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
name VARCHAR(255) NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);
-- Group memberships
CREATE TABLE IF NOT EXISTS carddav.group_memberships (
group_id UUID NOT NULL REFERENCES carddav.contact_groups(id) ON DELETE CASCADE,
contact_id UUID NOT NULL REFERENCES carddav.contacts(id) ON DELETE CASCADE,
PRIMARY KEY(group_id, contact_id)
);
```
## API Endpoints
### REST API
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/address-books` | List all address books |
| POST | `/api/address-books` | Create a new address book |
| GET | `/api/address-books/:id` | Get a specific address book |
| PUT | `/api/address-books/:id` | Update an address book |
| DELETE | `/api/address-books/:id` | Delete an address book |
| GET | `/api/address-books/:id/contacts` | List contacts in an address book |
| POST | `/api/address-books/:id/contacts` | Create a new contact |
| GET | `/api/address-books/:id/contacts/:contactId` | Get a specific contact |
| PUT | `/api/address-books/:id/contacts/:contactId` | Update a contact |
| DELETE | `/api/address-books/:id/contacts/:contactId` | Delete a contact |
| GET | `/api/address-books/:id/groups` | List contact groups |
| POST | `/api/address-books/:id/groups` | Create a new contact group |
### CardDAV Protocol Endpoints
| Method | Endpoint | Description |
|--------|----------|-------------|
| PROPFIND | `/carddav/` | List all address books |
| PROPFIND | `/carddav/:addressBookId/` | Get address book info |
| REPORT | `/carddav/:addressBookId/` | Query contacts in an address book |
| GET | `/carddav/:addressBookId/:contactId.vcf` | Get a specific contact (vCard) |
| PUT | `/carddav/:addressBookId/:contactId.vcf` | Create or update a contact |
| DELETE | `/carddav/:addressBookId/:contactId.vcf` | Delete a contact |
| MKCOL | `/carddav/:addressBookId/` | Create a new address book |
| DELETE | `/carddav/:addressBookId/` | Delete an address book |
## Data Model
### Contact Entity
```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Contact {
pub id: Uuid,
pub address_book_id: Uuid,
pub uid: String,
pub full_name: Option<String>,
pub first_name: Option<String>,
pub last_name: Option<String>,
pub nickname: Option<String>,
pub email: Vec<Email>,
pub phone: Vec<Phone>,
pub address: Vec<Address>,
pub organization: Option<String>,
pub title: Option<String>,
pub notes: Option<String>,
pub photo_url: Option<String>,
pub birthday: Option<NaiveDate>,
pub anniversary: Option<NaiveDate>,
pub vcard: String,
pub etag: String,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Email {
pub email: String,
pub r#type: String, // home, work, other
pub is_primary: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Phone {
pub number: String,
pub r#type: String, // mobile, home, work, fax, other
pub is_primary: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Address {
pub street: Option<String>,
pub city: Option<String>,
pub state: Option<String>,
pub postal_code: Option<String>,
pub country: Option<String>,
pub r#type: String, // home, work, other
pub is_primary: bool,
}
```
### AddressBook Entity
```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AddressBook {
pub id: Uuid,
pub name: String,
pub owner_id: String,
pub description: Option<String>,
pub color: Option<String>,
pub is_public: bool,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
```
## Repositories
### ContactRepository Interface
```rust
#[async_trait]
pub trait ContactRepository: Send + Sync + 'static {
async fn create_contact(&self, contact: Contact) -> ContactRepositoryResult<Contact>;
async fn update_contact(&self, contact: Contact) -> ContactRepositoryResult<Contact>;
async fn delete_contact(&self, id: &Uuid) -> ContactRepositoryResult<()>;
async fn get_contact_by_id(&self, id: &Uuid) -> ContactRepositoryResult<Option<Contact>>;
async fn get_contact_by_uid(&self, address_book_id: &Uuid, uid: &str) -> ContactRepositoryResult<Option<Contact>>;
async fn get_contacts_by_address_book(&self, address_book_id: &Uuid) -> ContactRepositoryResult<Vec<Contact>>;
async fn get_contacts_by_email(&self, email: &str) -> ContactRepositoryResult<Vec<Contact>>;
async fn get_contacts_by_group(&self, group_id: &Uuid) -> ContactRepositoryResult<Vec<Contact>>;
}
```
### AddressBookRepository Interface
```rust
#[async_trait]
pub trait AddressBookRepository: Send + Sync + 'static {
async fn create_address_book(&self, address_book: AddressBook) -> AddressBookRepositoryResult<AddressBook>;
async fn update_address_book(&self, address_book: AddressBook) -> AddressBookRepositoryResult<AddressBook>;
async fn delete_address_book(&self, id: &Uuid) -> AddressBookRepositoryResult<()>;
async fn get_address_book_by_id(&self, id: &Uuid) -> AddressBookRepositoryResult<Option<AddressBook>>;
async fn get_address_books_by_owner(&self, owner_id: &str) -> AddressBookRepositoryResult<Vec<AddressBook>>;
async fn get_shared_address_books(&self, user_id: &str) -> AddressBookRepositoryResult<Vec<AddressBook>>;
async fn get_public_address_books(&self) -> AddressBookRepositoryResult<Vec<AddressBook>>;
async fn share_address_book(&self, address_book_id: &Uuid, user_id: &str, can_write: bool) -> AddressBookRepositoryResult<()>;
async fn unshare_address_book(&self, address_book_id: &Uuid, user_id: &str) -> AddressBookRepositoryResult<()>;
async fn get_address_book_shares(&self, address_book_id: &Uuid) -> AddressBookRepositoryResult<Vec<(String, bool)>>;
}
```
## CardDAV Protocol Features
1. **Address Book Discovery** -- clients discover available address books
2. **Address Book Collection** -- manage contacts within address books
3. **vCard Support** -- store and retrieve contacts in vCard format (3.0 and 4.0)
4. **Query Support** -- filter contacts by properties
5. **Multiget Support** -- retrieve multiple contacts in a single request
6. **Sync-Collection** -- efficient incremental synchronization
### CardDAV Adapter
The **CardDAVAdapter** handles:
1. Parsing CardDAV XML requests
2. Converting between vCard and **Contact** entities
3. Generating CardDAV XML responses
4. Supporting **PROPFIND**, **REPORT**, and other WebDAV methods
5. Implementing proper WebDAV properties for CardDAV
## Integration Points
1. **Authentication** -- reuses existing auth mechanisms
2. **WebDAV Infrastructure** -- extends the existing WebDAV implementation
3. **Database Layer** -- stores contacts in PostgreSQL
4. **User Management** -- connects contacts with user accounts
## Client Compatibility
Target clients:
- Apple Contacts
- Google Contacts
- Thunderbird
- Outlook
- Android DAVx5
- iOS native contacts app
- Evolution
See `dav-client-setup.md` for detailed connection instructions.
## Implementation Phases
### Phase 1: Core Infrastructure
- Database schema creation
- Entity definitions
- Repository interfaces
- Basic DTO and port definitions
### Phase 2: Core Business Logic
- Address book management service
- Contact management service
- vCard parsing and generation
### Phase 3: REST API
- Address book endpoints
- Contact management endpoints
- Contact group endpoints
### Phase 4: CardDAV Protocol
- **CardDAVAdapter** implementation
- WebDAV method handlers
- XML parsing and generation
- Protocol compliance testing
### Phase 5: Testing and Refinement
- Integration testing with client applications
- Performance optimization
- Edge case handling
## Security
1. **Authentication** -- proper authentication for all operations
2. **Authorization** -- verify permissions for each address book operation
3. **Data Validation** -- validate vCard input to prevent injection attacks
4. **Resource Limits** -- limits to prevent abuse
5. **Error Handling** -- appropriate error responses without leaking sensitive data
## Performance
1. **Indexing** -- proper database indexes for contact queries
2. **Caching** -- cache frequently accessed address books and contacts
3. **Pagination** -- support pagination for large address books
4. **Incremental Sync** -- efficient sync with client devices
5. **ETags** -- prevent unnecessary data transfers
## Testing Strategy
1. **Unit Tests** -- test individual components in isolation
2. **Integration Tests** -- test component interactions
3. **Protocol Compliance Tests** -- verify adherence to the CardDAV spec (RFC 6352)
4. **Client Compatibility Tests** -- test with various CardDAV clients
5. **Performance Tests** -- measure performance with large address books
doc/chunked-uploads.md
-175
View File
@@ -1,175 +0,0 @@
# 07 - Chunked Uploads
OxiCloud implements a TUS-like chunked upload protocol for large files (≥10 MB). Files are split into chunks (default 5 MB) that can be uploaded in parallel (up to 6 concurrent), with progress tracking, optional MD5 checksums, and automatic session expiration.
## Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **ChunkedUploadPort** trait + DTOs | `src/application/ports/chunked_upload_ports.rs` |
| Infrastructure | **ChunkedUploadService** | `src/infrastructure/services/chunked_upload_service.rs` |
| Interfaces | **ChunkedUploadHandler** | `src/interfaces/api/handlers/chunked_upload_handler.rs` |
## Constants
| Constant | Value | Description |
|---|---|---|
| `CHUNKED_UPLOAD_THRESHOLD` | 10 MB | Files above this should use chunked upload |
| `DEFAULT_CHUNK_SIZE` | 5 MB | Default chunk size (minimum 1 MB) |
| `MAX_PARALLEL_CHUNKS` | 6 | Maximum concurrent chunk uploads |
| `SESSION_EXPIRATION` | 24 hours | Sessions expire after this duration |
## REST API
All routes under `/api/uploads`, authentication required.
| Method | Path | Handler | Description |
|---|---|---|---|
| `POST` | `/api/uploads` | `create_upload` | Create upload session |
| `PATCH` | `/api/uploads/{upload_id}` | `upload_chunk` | Upload a single chunk |
| `HEAD` | `/api/uploads/{upload_id}` | `get_upload_status` | Query upload progress |
| `POST` | `/api/uploads/{upload_id}/complete` | `complete_upload` | Assemble chunks → create file |
| `DELETE` | `/api/uploads/{upload_id}` | `cancel_upload` | Cancel and cleanup |
## Protocol Flow
```
1. POST /api/uploads
Body: { "filename": "video.mp4", "total_size": 104857600, "content_type": "video/mp4" }
Response: { "upload_id": "abc-123", "chunk_size": 5242880, "total_chunks": 20, "expires_at": 1707868800 }
2. PATCH /api/uploads/abc-123?chunk_index=0 ──┐
PATCH /api/uploads/abc-123?chunk_index=1 ──┼── Up to 6 in parallel
PATCH /api/uploads/abc-123?chunk_index=2 ──┘
Body: raw chunk bytes
Response: { "chunk_index": 0, "bytes_received": 52428800, "progress": 50.0, "is_complete": false }
3. HEAD /api/uploads/abc-123
Response headers: Upload-Offset, Upload-Length, Upload-Progress, Upload-Chunks-Total, Upload-Chunks-Complete
4. POST /api/uploads/abc-123/complete
Response: { "file_id": "def-456", "filename": "video.mp4", "size": 104857600, "path": "/videos" }
Status: 201 Created
```
## Request/Response DTOs
### Create Upload Request
```rust
pub struct CreateUploadRequest {
pub filename: String,
pub folder_id: Option<String>,
pub content_type: Option<String>, // default: "application/octet-stream"
pub total_size: u64,
pub chunk_size: Option<usize>, // default: 5 MB, minimum: 1 MB
}
```
### Chunk Upload Query Parameters
```rust
pub struct ChunkUploadParams {
pub chunk_index: usize,
pub checksum: Option<String>, // MD5 hash (also via Content-MD5 header)
}
```
### Upload Status Response
```rust
pub struct UploadStatusResponseDto {
pub upload_id: String,
pub filename: String,
pub total_size: u64,
pub bytes_received: u64,
pub progress: f64, // 0.0 - 100.0
pub total_chunks: usize,
pub completed_chunks: usize,
pub pending_chunks: Vec<usize>,
pub is_complete: bool,
}
```
### Complete Upload Response
```rust
pub struct CompleteUploadResponse {
pub file_id: String,
pub filename: String,
pub size: u64,
pub path: String,
}
```
## Custom Response Headers
### On upload_chunk (PATCH)
- `Upload-Offset`: total bytes received so far
- `Upload-Progress`: percentage complete (0-100)
- `Upload-Complete: true` (only when all chunks are uploaded)
### On get_upload_status (HEAD)
- `Upload-Offset`: bytes received
- `Upload-Length`: total expected size
- `Upload-Progress`: percentage
- `Upload-Chunks-Total`: total chunk count
- `Upload-Chunks-Complete`: completed chunk count
## Internal Storage
```
<temp_dir>/
<upload_id>/
chunk_000000 ← individual chunk files
chunk_000001
chunk_000002
...
assembled ← final assembled file (after complete)
```
## Completion Flow
1. `complete_upload()` assembles all chunks in order into a single `assembled` file
2. Reads the assembled file content
3. Delegates to **FileUploadService::upload_file()** to create the permanent file record
4. Calls `finalize_upload()` to clean up the session and temp directory
5. Returns `201 Created` with file metadata
## Session Management
- **In-memory sessions**: `HashMap<String, UploadSession>` protected by `RwLock`
- **Chunk status tracking**: each chunk has a status: `Pending` → `Uploading` → `Complete` (or `Failed`)
- **Expiration**: sessions expire after 24 hours of inactivity
- **Cleanup**: background task runs hourly to remove expired sessions and orphaned temp directories
## Client Usage Example
```bash
# 1. Create upload session
RESPONSE=$(curl -s -X POST -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"filename":"large-file.zip","total_size":52428800}' \
"https://oxicloud.example.com/api/uploads")
UPLOAD_ID=$(echo $RESPONSE | jq -r '.upload_id')
CHUNK_SIZE=$(echo $RESPONSE | jq -r '.chunk_size')
TOTAL_CHUNKS=$(echo $RESPONSE | jq -r '.total_chunks')
# 2. Upload chunks in parallel
for i in $(seq 0 $((TOTAL_CHUNKS - 1))); do
dd if=large-file.zip bs=$CHUNK_SIZE skip=$i count=1 2>/dev/null | \
curl -s -X PATCH -H "Authorization: Bearer $TOKEN" \
--data-binary @- \
"https://oxicloud.example.com/api/uploads/$UPLOAD_ID?chunk_index=$i" &
done
wait
# 3. Complete upload
curl -X POST -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/uploads/$UPLOAD_ID/complete"
# 4. Check progress (optional)
curl -I -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/uploads/$UPLOAD_ID"
```
doc/database-transactions.md
-119
View File
@@ -1,119 +0,0 @@
# 19 - Database Transactions
OxiCloud uses explicit transactions on PostgreSQL to guarantee data integrity. All transactional operations follow ACID properties:
- **Atomicity** -- all-or-nothing. If any part fails, the entire transaction fails.
- **Consistency** -- the database moves from one valid state to another.
- **Isolation** -- concurrent transactions behave as if sequential.
- **Durability** -- once committed, the transaction survives system failures.
## Implementation
The **with_transaction** helper wraps the standard pattern: begin, execute, commit on success, rollback on error.
### Transaction Utility
Located in `src/infrastructure/repositories/pg/transaction_utils.rs`:
```rust
/// Helper function to execute database operations in a transaction
pub async fn with_transaction<F, T, E>(
pool: &Arc<PgPool>,
operation_name: &str,
operation: F,
) -> Result<T, E>
where
F: for<'c> FnOnce(&'c mut Transaction<'_, Postgres>) -> futures::future::BoxFuture<'c, Result<T, E>>,
E: From<SqlxError> + std::fmt::Display
{ ... }
```
This function takes a connection pool and a closure with operations, handles begin/commit/rollback automatically, and provides detailed logging of the transaction lifecycle.
### Repository Usage Example
```rust
// Creación de un usuario con transacción explícita
async fn create_user(&self, user: User) -> UserRepositoryResult<User> {
with_transaction(
&self.pool,
"create_user",
|tx| {
Box::pin(async move {
// Operación principal - insertar usuario
sqlx::query("INSERT INTO auth.users ...")
.bind(...)
.execute(&mut **tx)
.await?;
// Operaciones adicionales dentro de la misma transacción
// ...
Ok(user_clone)
})
}
).await
}
```
## Use Cases
### UserPgRepository
1. **User creation** -- guarantees all insert operations are atomic. Allows related operations (like permission setup) to be bundled.
2. **User update** -- ensures modifications apply fully or not at all. Supports combined operations like profile info and preference updates.
### SessionPgRepository
1. **Session creation** -- inserts the session and updates the user's last-access timestamp in a single transaction. Keeps sessions and user data consistent.
2. **Session revocation** -- ensures revoking one or all sessions for a user is atomic. Allows logging security events within the same transaction.
## Isolation Levels
OxiCloud supports different transaction isolation levels via **with_transaction_isolation**:
```rust
// Ejemplo de uso con nivel de aislamiento específico
with_transaction_isolation(
&pool,
"operacion_critica",
sqlx::postgres::PgIsolationLevel::Serializable,
|tx| { ... }
).await
```
Available isolation levels:
1. **Read Committed** (default) -- guarantees reads see only committed data. Does not prevent non-repeatable or phantom reads.
2. **Repeatable Read** -- guarantees consistent reads throughout the transaction. Prevents non-repeatable reads but not phantom reads.
3. **Serializable** -- highest isolation level. Transactions behave as if executed serially. Can cause serialization errors that require retry.
## Best Practices
1. **Transaction duration** -- keep transactions as short as possible. Avoid long-running operations inside them.
2. **Error handling** -- errors inside a transaction trigger automatic rollback. Use proper logging to diagnose failures.
3. **Transaction boundaries** -- define clearly where transactions begin and end. Group related operations into a single transaction.
4. **Appropriate isolation** -- use the lowest isolation level that fits the use case. Consider serializable for critical operations with conflict potential.
## Benefits
- **Data integrity** -- ACID guarantees for complex operations, prevents inconsistent states.
- **Error handling** -- automatic rollback on failure, predictable behavior.
- **Safe concurrency** -- proper handling of simultaneous operations, prevents race conditions.
- **Performance** -- fewer round-trips to the database, batch operations for better efficiency.
## Performance Considerations
Transactions add some overhead. Performance can be affected by:
- Transaction duration
- Isolation level
- Number of affected records
- Lock contention
doc/dav-client-setup.md
-309
View File
@@ -1,309 +0,0 @@
# 25 - DAV Client Setup
Step-by-step instructions for connecting native OS clients via WebDAV, CalDAV, and CardDAV. Covers Windows, macOS, Linux, iOS, and Android.
## Table of Contents
- [WebDAV Setup](#webdav-setup) (File Access)
- [CalDAV Setup](#caldav-setup) (Calendar Sync)
- [CardDAV Setup](#carddav-setup) (Contact Sync)
- [Troubleshooting](#troubleshooting)
---
## WebDAV Setup
### Connection Information
- **Server URL**: `https://[your-oxicloud-server]/webdav/`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
### Windows
#### Windows 10/11 (File Explorer)
1. Open File Explorer
2. Right-click on "This PC" and select "Add a network location"
3. Click "Next"
4. Select "Choose a custom network location" and click "Next"
5. Enter the WebDAV URL: `https://[your-oxicloud-server]/webdav/`
6. When prompted, enter your username and password
7. Give the connection a name (e.g., "OxiCloud") and click "Next"
8. Click "Finish"
Files now appear as a network drive in File Explorer.
#### Alternative: Map Network Drive
1. Open File Explorer
2. Right-click on "This PC" and select "Map network drive"
3. Choose a drive letter
4. Enter the WebDAV URL: `https://[your-oxicloud-server]/webdav/`
5. Check "Connect using different credentials"
6. Click "Finish"
7. Enter your username and password
**Windows Troubleshooting:**
If connections fail on Windows:
1. Open Registry Editor (regedit.exe)
2. Navigate to `HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters`
3. Modify **BasicAuthLevel** to value `2`
4. Restart the WebClient service or reboot
To increase the file size limit:
1. In the same registry location, modify **FileSizeLimitInBytes** to a higher value (e.g., `4294967295` for 4GB)
2. Restart the WebClient service
### macOS
#### Finder
1. Open Finder
2. From the menu bar, click "Go" > "Connect to Server" (or press Cmd+K)
3. Enter the WebDAV URL: `https://[your-oxicloud-server]/webdav/`
4. Click "Connect"
5. Enter your username and password
6. Click "Connect"
Files appear as a mounted drive in Finder.
### Linux
#### GNOME (Nautilus)
1. Open Files (Nautilus)
2. Click the "+" button in the sidebar or press Ctrl+L
3. Enter: `davs://[your-oxicloud-server]/webdav/`
4. Enter credentials when prompted
5. Click "Connect"
#### KDE (Dolphin)
1. Open Dolphin
2. In the address bar, enter: `webdavs://[your-oxicloud-server]/webdav/`
3. Enter credentials when prompted
4. Click "Connect"
#### Command Line (davfs2)
1. Install davfs2: `sudo apt-get install davfs2` (Debian/Ubuntu) or equivalent
2. Create a mount point: `sudo mkdir /mnt/oxicloud`
3. Edit `/etc/davfs2/secrets` and add: `/mnt/oxicloud [username] [password]`
4. Mount: `sudo mount -t davfs https://[your-oxicloud-server]/webdav/ /mnt/oxicloud`
To auto-mount at boot, add to `/etc/fstab`:
```
https://[your-oxicloud-server]/webdav/ /mnt/oxicloud davfs user,rw,auto 0 0
```
---
## CalDAV Setup
### Apple Calendar (macOS/iOS)
#### macOS:
1. Open the Calendar app
2. Go to **Calendar** > **Add Account** > **Other CalDAV Account**
3. Enter:
- **Account Type**: Advanced
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
- **Server Address**: `https://[your-oxicloud-server]/caldav`
4. Click **Sign In**
5. Select the calendars you want to sync and click **Done**
#### iOS:
1. Go to **Settings** > **Calendar** > **Accounts** > **Add Account** > **Other**
2. Tap **Add CalDAV Account**
3. Enter:
- **Server**: `https://[your-oxicloud-server]/caldav`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
- **Description**: OxiCloud Calendar (or any name)
4. Tap **Next**
5. Turn on **Calendars** and tap **Save**
### Thunderbird with Lightning
1. Open Thunderbird and go to the **Calendar** tab
2. Right-click in the left pane and select **New Calendar**
3. Select **On the Network** and click **Next**
4. Choose **CalDAV** as the format
5. Enter the location: `https://[your-oxicloud-server]/caldav/calendars/your-calendar-id`
6. Click **Next**
7. Enter a name for the calendar and choose a color
8. Click **Next** and then **Finish**
9. When prompted, enter your username and password
### Android (DAVx5)
1. Install [DAVx5](https://play.google.com/store/apps/details?id=at.bitfire.davdroid) from Google Play Store
2. Open DAVx5 and tap the **+** button
3. Select **Login with URL and username**
4. Enter:
- **Base URL**: `https://[your-oxicloud-server]/caldav`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
5. Tap **Connect**
6. Select the calendars you want to sync
7. Tap the checkbox to enable syncing
### Windows (Outlook)
1. Download and install [CalDAV Synchronizer](https://caldavsynchronizer.org/)
2. Open Outlook and navigate to the **CalDAV Synchronizer** tab
3. Click **Synchronization Profiles**
4. Click **Add** to create a new profile
5. Enter:
- **Profile Name**: OxiCloud Calendar (or any name)
- **CalDAV URL**: `https://[your-oxicloud-server]/caldav/calendars/your-calendar-id`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
6. Click **Test or discover settings**
7. Select the Outlook calendar to sync with
8. Click **OK** to save the profile
---
## CardDAV Setup
### Apple Contacts (macOS/iOS)
#### macOS:
1. Open the Contacts app
2. Go to **Contacts** > **Add Account** > **Other contacts account**
3. Select **CardDAV account**
4. Enter:
- **Server**: `https://[your-oxicloud-server]/carddav`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
- **Description**: OxiCloud Contacts (or any name)
5. Click **Sign In**
#### iOS:
1. Go to **Settings** > **Contacts** > **Accounts** > **Add Account** > **Other**
2. Tap **Add CardDAV Account**
3. Enter:
- **Server**: `https://[your-oxicloud-server]/carddav`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
- **Description**: OxiCloud Contacts (or any name)
4. Tap **Next**
5. Turn on **Contacts** and tap **Save**
### Thunderbird
1. Open Thunderbird and go to the **Address Book**
2. Click on **Tools** > **Address Book**
3. Go to **File** > **New** > **Remote Address Book**
4. Enter:
- **Name**: OxiCloud Contacts (or any name)
- **URL**: `https://[your-oxicloud-server]/carddav/address-books/your-address-book-id`
5. Click **OK**
6. When prompted, enter your username and password
### Android (DAVx5)
1. Install [DAVx5](https://play.google.com/store/apps/details?id=at.bitfire.davdroid) from Google Play Store
2. Open DAVx5 and tap the **+** button
3. Select **Login with URL and username**
4. Enter:
- **Base URL**: `https://[your-oxicloud-server]/carddav`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
5. Tap **Connect**
6. Select the address books you want to sync
7. Tap the checkbox to enable syncing
### Windows (Outlook)
1. Download and install [CardDAV Synchronizer](https://caldavsynchronizer.org/) (same tool as CalDAV)
2. Open Outlook and navigate to the **CardDAV Synchronizer** tab
3. Click **Synchronization Profiles**
4. Click **Add** to create a new profile
5. Select **CardDAV** as the synchronization resource
6. Enter:
- **Profile Name**: OxiCloud Contacts (or any name)
- **CardDAV URL**: `https://[your-oxicloud-server]/carddav/address-books/your-address-book-id`
- **Username**: Your OxiCloud username
- **Password**: Your OxiCloud password
7. Click **Test or discover settings**
8. Select the Outlook contacts folder to sync with
9. Click **OK** to save the profile
---
## Troubleshooting
### Common Issues
#### WebDAV Connection Issues
- Verify the server URL includes the `/webdav/` path
- Double-check username and password
- Check if your network blocks ports 80/443
- Confirm WebDAV is enabled on the server
#### Calendar/Contact Sync Issues
- Verify the server URL includes the full path (`/caldav` or `/carddav`)
- Confirm the server is reachable from your network
- Verify the calendar or address book ID is correct
- Check you have proper permissions for the resource
#### Calendar Not Showing
- Confirm the calendar is enabled in your client
- Check if the calendar is shared with your account
- Ensure your client supports the CalDAV protocol version in use
#### Contact Photos Not Syncing
- Some clients have photo sync limitations
- Verify the photo is in a supported format (usually JPEG)
- Check photo size limits on the client side
### Client-Specific Issues
#### Windows File Explorer
- Make sure the **WebClient** service is running
- Increase timeout values in the registry
- Try a third-party WebDAV client like Cyberduck
#### iOS Devices
- If connection fails, try **Settings** > **Accounts & Passwords** and add the account manually
- For persistent issues, remove the account and re-add it
#### Android
- DAVx5 requires battery optimization to be disabled for reliable background sync
- Go to **Settings** > **Apps** > **DAVx5** > **Battery** > **Unrestricted**
#### Outlook
- Make sure you have the latest version of CalDAV/CardDAV Synchronizer
- The plugin may need reactivation after Outlook updates
### Performance Tips
1. **Large Files** -- for files over 100MB, download locally before editing
2. **Slow Connections** -- enable offline caching in your client when available
3. **File Locking** -- some clients support WebDAV locking to prevent conflicts
### Getting Help
If issues persist:
1. Check the server logs for error messages
2. Capture screenshots of any errors
3. Contact support with details about your client application, version, steps to reproduce, and error messages
doc/dav-implementation-plan.md
-286
View File
@@ -1,286 +0,0 @@
# 24 - DAV Implementation Plan
A phased plan for adding WebDAV, CalDAV, and CardDAV support. The order is WebDAV first (file access), then CalDAV (calendars), then CardDAV (contacts). Each phase builds on the shared infrastructure from phase 1.
## Executive Summary
DAV protocol support lets the platform interoperate with a wide range of clients and devices. The plan uses an incremental approach -- common infrastructure first, then each protocol in turn.
## Implementation Phases
### Phase 1: Common DAV Infrastructure (Est. 2-3 weeks)
**Goals:**
- Build the shared infrastructure used by all DAV protocols
- Implement XML request/response handling
- Create adapters for basic DAV operations
**Tasks:**
1. **Week 1: Design and Architecture**
- Design DAV component architecture
- Define interfaces for DAV adapters
- Select libraries for XML processing and RFC 4918 support
2. **Week 2: Base Implementation**
- Implement XML serialization/deserialization handlers
- Develop middleware for DAV request processing
- Create shared structures (properties, namespaces)
- Implement DAV request validation
3. **Week 3: Test Framework**
- Set up test environment for DAV protocols
- Implement automated test clients
- Create test cases for basic DAV operations
**Deliverables:**
- XML processing framework for DAV requests/responses
- Base adapters for existing entities
- Test suite for DAV operations
### Phase 2: WebDAV (Est. 3-4 weeks)
**Goals:**
- Implement the full WebDAV protocol (RFC 4918)
- Enable file and folder access via WebDAV
- Ensure compatibility with common WebDAV clients
**Tasks:**
1. **Week 1: Basic Operations**
- Implement **PROPFIND** and **PROPPATCH** methods
- Develop **OPTIONS** endpoint (capability discovery)
- Implement **GET**, **HEAD**, **PUT** (read/write)
2. **Week 2: Advanced Operations**
- Implement **MKCOL** (directory creation)
- Develop **DELETE** for WebDAV resources
- Implement **COPY** and **MOVE** for files and directories
3. **Week 3: Locking and Extended Features**
- Implement **LOCK** and **UNLOCK** for resources
- Add support for custom properties
- Develop extended WebDAV features as needed
4. **Week 4: Testing and Optimization**
- Test with real clients (Windows, macOS, Linux)
- Optimize performance for large transfers
- Document WebDAV API and behavior
**Deliverables:**
- Full WebDAV implementation (RFC 4918)
- Usage documentation
- Compatibility with the most common WebDAV clients
### Phase 3: CalDAV (Est. 4-5 weeks)
**Goals:**
- Implement the CalDAV protocol (RFC 4791)
- Create entities and repositories for calendars and events
- Support calendar operations with common clients
**Tasks:**
1. **Week 1: Data Model**
- Implement **Calendar** and **CalendarEvent** entities
- Develop storage repositories
- Create CalDAV DTOs and adapters
2. **Week 2: Basic Endpoints**
- Implement **PROPFIND** for calendar discovery
- Develop **MKCALENDAR** for calendar creation
- Implement **GET**/**PUT** for individual events
3. **Week 3: Advanced Queries**
- Implement **REPORT** for calendar queries
- Develop date-range search support
- Add recurrence handling (**RRULE** rules)
4. **Week 4: Interoperability**
- Implement efficient sync (**collection-sync**)
- Add timezone support
- Develop alarm and notification handling
5. **Week 5: Testing and Refinement**
- Test with popular CalDAV clients
- Optimize performance for large calendars
- Document CalDAV API and behavior
**Deliverables:**
- Full CalDAV implementation (RFC 4791)
- Calendar creation and management support
- Compatibility with popular CalDAV clients
- CalDAV usage documentation
### Phase 4: CardDAV (Est. 3-4 weeks)
**Goals:**
- Implement the CardDAV protocol (RFC 6352)
- Create entities and repositories for address books and contacts
- Support contact operations with common clients
**Tasks:**
1. **Week 1: Data Model**
- Implement **AddressBook** and **Contact** entities
- Develop storage repositories
- Create CardDAV DTOs and adapters
2. **Week 2: Basic Endpoints**
- Implement **PROPFIND** for address book discovery
- Develop **MKCOL** for address book creation
- Implement **GET**/**PUT** for individual contacts
3. **Week 3: Queries and Search**
- Implement **REPORT** for contact queries
- Develop criteria-based contact search
- Add support for contact groups
4. **Week 4: Testing and Refinement**
- Test with popular CardDAV clients
- Optimize performance for large address books
- Document CardDAV API and behavior
**Deliverables:**
- Full CardDAV implementation (RFC 6352)
- Address book creation and management support
- Compatibility with popular CardDAV clients
- CardDAV usage documentation
### Phase 5: Integration and Release (Est. 2-3 weeks)
**Goals:**
- Integrate all DAV protocols into a cohesive solution
- Ensure cross-protocol compatibility
- Prepare documentation and release materials
**Tasks:**
1. **Week 1: Integration**
- Consolidate shared code across protocols
- Ensure behavioral consistency
- Refine error handling and recovery
2. **Week 2: System Testing**
- Run end-to-end integration tests
- Validate performance under load
- Verify security and permissions
3. **Week 3: Documentation and Release**
- Finalize user guides for DAV clients
- Create developer documentation
- Prepare release package
**Deliverables:**
- Complete, integrated DAV solution
- User and developer documentation
- Release-ready deployment package
## Infrastructure Requirements
### Library Dependencies
```toml
# Add to Cargo.toml
[dependencies]
# XML processing
quick-xml = "0.30.0"
xml-rs = "0.8.14"
# iCalendar support
icalendar = "0.15.0"
# vCard support
vcard = "0.2.0"
# DAV utilities
http-multipart = "0.3.0"
```
### Database Schema
New tables for CalDAV and CardDAV are created during their respective phases. See `dav-integration.md` for the full schema.
## Testing Strategy
### Unit Tests
- XML serialization/deserialization tests
- Input validation tests
- Business logic tests for each DAV operation
### Integration Tests
- End-to-end tests with simulated clients
- Full-flow tests (create, update, delete)
- Concurrency and conflict handling tests
### Compatibility Tests
- Test matrix with real clients (at least 3 per protocol)
- Testing across different operating systems
- RFC conformance verification
## Performance Considerations
1. **Query Optimization**
- Implement pagination for large result sets
- Optimize SQL queries for calendars and contacts
- Use proper indexes for fast lookups
2. **Caching**
- Cache properties for **PROPFIND** responses
- Use ETags for cache validation
- Apply query caching for frequent reports
3. **Efficient Processing**
- Efficient XML processing for large requests
- Data streaming for large files
- Async processing for expensive operations
## Risks and Mitigation
| Risk | Impact | Likelihood | Mitigation Strategy |
|------|--------|------------|---------------------|
| Client compatibility issues | High | Medium | Early testing with a variety of clients, strict adherence to specs |
| Insufficient performance | Medium | Low | Load testing from the start, design for scalability |
| Excessive complexity | Medium | Medium | Modular approach, clear abstractions, frequent code reviews |
| Security vulnerabilities | High | Low | Security reviews, strict input validation, penetration testing |
| Schedule delays | Medium | Medium | Conservative planning, clear milestones, iterative approach |
## Success Criteria
1. **Compatibility**
- All protocols comply with their respective RFCs
- Verified compatibility with at least 3 major clients per protocol
- Works on all major operating systems
2. **Performance**
- Typical operation response time < 500ms
- Supports calendars with 1000+ events without significant degradation
- Supports address books with 1000+ contacts without significant degradation
3. **Usability**
- Simple, documented client setup process
- Clear and specific error messages
- Full documentation for users and developers
## Required Resources
1. **Development Team**
- 1-2 backend developers (Rust)
- 1 frontend developer (for UI integration if needed)
- 1 tester
2. **Infrastructure**
- Multi-OS test environment
- Assorted DAV clients for testing
- CI/CD server for automated tests
3. **Skills**
- Experience with advanced HTTP protocols
- XML processing knowledge
- Familiarity with WebDAV, CalDAV, and CardDAV standards
## Next Steps
1. Assign resources to the project
2. Set up code repository and initial structure
3. Start Phase 1 (Common DAV Infrastructure)
4. Configure CI/CD environment for testing
5. Review and refine the plan as needed during implementation
doc/dav-integration.md
-637
View File
@@ -1,637 +0,0 @@
# 23 - DAV Integration
WebDAV, CalDAV, and CardDAV extend the platform to support clients and devices that speak these standard protocols. The implementation follows the existing hexagonal architecture -- each protocol gets its own adapter, service, and domain layer.
## Table of Contents
1. [Introduction](#introduction)
2. [Implementation Architecture](#implementation-architecture)
3. [WebDAV](#webdav)
4. [CalDAV](#caldav)
5. [CardDAV](#carddav)
6. [Security Considerations](#security-considerations)
7. [Testing and Compatibility](#testing-and-compatibility)
## Introduction
### WebDAV (Web Distributed Authoring and Versioning)
An HTTP extension that lets clients create, modify, move, and delete files and directories on a remote server.
### CalDAV (Calendaring Extensions to WebDAV)
A WebDAV-based protocol for accessing and managing calendar data (events and tasks).
### CardDAV (vCard Extensions to WebDAV)
Extends WebDAV to allow access and management of contact data in vCard format.
## Implementation Architecture
DAV protocols plug into the existing hexagonal architecture:
```
┌────────────────────────────────────────────────────────────────────┐
│ INTERFACES │
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────┐ │
│ │ │ │ │ │ │ │
│ │ REST API │ │ WebDAV API │ │ CalDAV/CardDAV API │ │
│ │ │ │ │ │ │ │
│ └───────┬───────┘ └───────┬───────┘ └───────────┬───────────┘ │
│ │ │ │ │
└──────────┼──────────────────┼──────────────────────┼──────────────┘
│ │ │
▼ ▼ ▼
┌──────────────────────────────────────────────────────────────────┐
│ APPLICATION │
│ │
│ ┌───────────┐ ┌────────────┐ ┌───────────┐ ┌──────────────┐ │
│ │ │ │ │ │ │ │ │ │
│ │FileService│ │FolderService│ │CalendarSvc│ │ContactService│ │
│ │ │ │ │ │ │ │ │ │
│ └─────┬─────┘ └──────┬─────┘ └─────┬─────┘ └──────┬───────┘ │
│ │ │ │ │ │
└────────┼───────────────┼──────────────┼───────────────┼─────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌────────────────────────────────────────────────────────────────┐
│ DOMAIN │
│ │
│ ┌─────────┐ ┌──────────┐ ┌────────────┐ ┌───────────────┐ │
│ │ │ │ │ │ │ │ │ │
│ │ File │ │ Folder │ │ Calendar │ │ Contact │ │
│ │ │ │ │ │ │ │ │ │
│ └─────────┘ └──────────┘ └────────────┘ └───────────────┘ │
│ │
└────────────────────────────────────────────────────────────────┘
```
### Main Components
1. **DAV Adapters** -- convert between DAV specs and internal models
2. **Application Services** -- extended to include DAV-specific functionality
3. **Domain Models** -- new entities for **Calendar** and **Contact**
4. **Repositories** -- storage implementations for calendars and contacts
## WebDAV
### Required Endpoints
| HTTP Method | Endpoint | Description |
|-------------|----------|-------------|
| OPTIONS | /webdav/{path} | Reports supported WebDAV capabilities |
| PROPFIND | /webdav/{path} | Retrieves resource properties |
| PROPPATCH | /webdav/{path} | Modifies resource properties |
| MKCOL | /webdav/{path} | Creates collections (directories) |
| GET | /webdav/{path} | Retrieves resource content |
| HEAD | /webdav/{path} | Retrieves resource metadata |
| PUT | /webdav/{path} | Creates or updates resources |
| DELETE | /webdav/{path} | Deletes resources |
| COPY | /webdav/{path} | Copies resources |
| MOVE | /webdav/{path} | Moves resources |
| LOCK | /webdav/{path} | Locks resources |
| UNLOCK | /webdav/{path} | Unlocks resources |
### Implementation
1. **WebDAV Handler**:
```rust
// src/interfaces/api/handlers/webdav_handler.rs
use std::sync::Arc;
use axum::{
Router,
routing::get,
extract::{Path, State, Request, Extension},
http::StatusCode,
response::Response,
};
use http::{Method, header};
use crate::common::di::AppState;
use crate::interfaces::middleware::auth::CurrentUser;
use crate::application::ports::file_ports::{FileRetrievalUseCase, FileUploadUseCase};
use crate::application::ports::folder_ports::FolderUseCase;
use crate::common::errors::AppError;
pub fn webdav_routes() -> Router<Arc<AppState>> {
Router::new()
.route("/webdav/*path", get(handle_get))
.route_with_tsr("/webdav/*path", axum::routing::on(
Method::OPTIONS, handle_options,
Method::PROPFIND, handle_propfind,
Method::PROPPATCH, handle_proppatch,
Method::MKCOL, handle_mkcol,
Method::PUT, handle_put,
Method::DELETE, handle_delete,
Method::COPY, handle_copy,
Method::MOVE, handle_move,
Method::LOCK, handle_lock,
Method::UNLOCK, handle_unlock,
))
}
// Implement functions for each WebDAV method...
```
2. **WebDAV Adapter**:
```rust
// src/application/adapters/webdav_adapter.rs
use xml::reader::{EventReader, XmlEvent};
use xml::writer::{EventWriter, EmitterConfig, XmlEvent as WriteEvent};
use std::io::{Read, Write};
use crate::application::dtos::file_dto::FileDto;
use crate::application::dtos::folder_dto::FolderDto;
/// Converts between internal objects and WebDAV representations
pub struct WebDavAdapter;
impl WebDavAdapter {
/// Parses a PROPFIND XML property into a request object
pub fn parse_propfind<R: Read>(reader: R) -> Result<PropFindRequest, Error> {
// Implementation...
}
/// Generates PROPFIND XML response from files and folders
pub fn generate_propfind_response<W: Write>(
writer: W,
files: &[FileDto],
folders: &[FolderDto],
base_url: &str,
) -> Result<(), Error> {
// Implementation...
}
// Other methods for different WebDAV operations...
}
```
## CalDAV
### Required Endpoints
| HTTP Method | Endpoint | Description |
|-------------|----------|-------------|
| PROPFIND | /caldav/{calendar} | Retrieves calendar properties |
| REPORT | /caldav/{calendar} | Queries calendar events |
| MKCALENDAR | /caldav/{calendar} | Creates a new calendar |
| PUT | /caldav/{calendar}/{event}.ics | Creates or updates an event |
| GET | /caldav/{calendar}/{event}.ics | Retrieves an event |
| DELETE | /caldav/{calendar}/{event}.ics | Deletes an event |
### Implementation
1. **Domain Entities**:
```rust
// src/domain/entities/calendar.rs
use uuid::Uuid;
use chrono::{DateTime, Utc};
#[derive(Debug, Clone)]
pub struct Calendar {
id: Uuid,
name: String,
owner_id: String,
description: Option<String>,
color: Option<String>,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
// src/domain/entities/calendar_event.rs
use uuid::Uuid;
use chrono::{DateTime, Utc};
#[derive(Debug, Clone)]
pub struct CalendarEvent {
id: Uuid,
calendar_id: Uuid,
summary: String,
description: Option<String>,
location: Option<String>,
start_time: DateTime<Utc>,
end_time: DateTime<Utc>,
all_day: bool,
rrule: Option<String>, // Recurrence rule
ical_data: String, // Full iCalendar data
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
```
2. **Repositories**:
```rust
// src/domain/repositories/calendar_repository.rs
use async_trait::async_trait;
use uuid::Uuid;
use crate::domain::entities::calendar::Calendar;
use crate::common::errors::Result;
#[async_trait]
pub trait CalendarRepository: Send + Sync {
async fn create_calendar(&self, calendar: Calendar) -> Result<Calendar>;
async fn get_calendar_by_id(&self, id: &Uuid) -> Result<Calendar>;
async fn get_calendars_by_owner(&self, owner_id: &str) -> Result<Vec<Calendar>>;
async fn update_calendar(&self, calendar: Calendar) -> Result<Calendar>;
async fn delete_calendar(&self, id: &Uuid) -> Result<()>;
}
// src/domain/repositories/calendar_event_repository.rs
use async_trait::async_trait;
use uuid::Uuid;
use chrono::{DateTime, Utc};
use crate::domain::entities::calendar_event::CalendarEvent;
use crate::common::errors::Result;
#[async_trait]
pub trait CalendarEventRepository: Send + Sync {
async fn create_event(&self, event: CalendarEvent) -> Result<CalendarEvent>;
async fn get_event_by_id(&self, id: &Uuid) -> Result<CalendarEvent>;
async fn get_events_by_calendar(&self, calendar_id: &Uuid) -> Result<Vec<CalendarEvent>>;
async fn get_events_in_timerange(
&self,
calendar_id: &Uuid,
start: &DateTime<Utc>,
end: &DateTime<Utc>
) -> Result<Vec<CalendarEvent>>;
async fn update_event(&self, event: CalendarEvent) -> Result<CalendarEvent>;
async fn delete_event(&self, id: &Uuid) -> Result<()>;
}
```
3. **CalDAV Service**:
```rust
// src/application/services/calendar_service.rs
use std::sync::Arc;
use crate::application::ports::calendar_ports::CalendarStoragePort;
use crate::application::dtos::calendar_dto::*;
pub struct CalendarService {
storage: Arc<dyn CalendarStoragePort>,
}
impl CalendarService {
pub fn new(storage: Arc<dyn CalendarStoragePort>) -> Self {
Self { storage }
}
// Implements CalendarUseCase for calendar and event operations...
}
```
4. **CalDAV Handler**:
```rust
// src/interfaces/api/handlers/caldav_handler.rs
use std::sync::Arc;
use axum::{
Router,
routing::{get, put, delete},
extract::{Path, State, Request, Extension},
http::StatusCode,
response::Response,
};
use http::{Method, header};
use crate::common::di::AppState;
use crate::interfaces::middleware::auth::CurrentUser;
use crate::application::services::caldav_service::CalDavService;
use crate::common::errors::AppError;
pub fn caldav_routes() -> Router<Arc<AppState>> {
Router::new()
.route("/caldav/", get(get_calendars))
.route("/caldav/:calendar", get(get_calendar))
.route_with_tsr("/caldav/:calendar", axum::routing::on(
Method::PROPFIND, handle_calendar_propfind,
Method::REPORT, handle_calendar_report,
Method::MKCALENDAR, handle_mkcalendar,
))
.route("/caldav/:calendar/:event", get(get_event))
.route("/caldav/:calendar/:event", put(put_event))
.route("/caldav/:calendar/:event", delete(delete_event))
}
// Implement functions for each CalDAV method...
```
## CardDAV
### Required Endpoints
| HTTP Method | Endpoint | Description |
|-------------|----------|-------------|
| PROPFIND | /carddav/addressbooks/{addressbook} | Retrieves address book properties |
| REPORT | /carddav/addressbooks/{addressbook} | Queries contacts |
| MKCOL | /carddav/addressbooks/{addressbook} | Creates a new address book |
| PUT | /carddav/addressbooks/{addressbook}/{contact}.vcf | Creates or updates a contact |
| GET | /carddav/addressbooks/{addressbook}/{contact}.vcf | Retrieves a contact |
| DELETE | /carddav/addressbooks/{addressbook}/{contact}.vcf | Deletes a contact |
### Implementation
1. **Domain Entities**:
```rust
// Note: AddressBook and Contact are both defined in src/domain/entities/contact.rs
#[derive(Debug, Clone)]
pub struct AddressBook {
id: Uuid,
name: String,
owner_id: String,
description: Option<String>,
color: Option<String>,
is_public: bool,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
#[derive(Debug, Clone)]
pub struct Contact {
id: Uuid,
address_book_id: Uuid,
uid: String,
full_name: Option<String>,
first_name: Option<String>,
last_name: Option<String>,
nickname: Option<String>,
email: Vec<Email>, // Struct with email, type, is_primary
phone: Vec<Phone>, // Struct with number, type, is_primary
address: Vec<Address>, // Struct with street, city, state, postal_code, country, type, is_primary
organization: Option<String>,
title: Option<String>,
notes: Option<String>,
photo_url: Option<String>,
birthday: Option<NaiveDate>,
anniversary: Option<NaiveDate>,
vcard: String, // Full vCard data
etag: String,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
```
2. **Repositories**:
```rust
// src/domain/repositories/address_book_repository.rs
use async_trait::async_trait;
use uuid::Uuid;
use crate::domain::entities::address_book::AddressBook;
use crate::common::errors::Result;
#[async_trait]
pub trait AddressBookRepository: Send + Sync {
async fn create_address_book(&self, address_book: AddressBook) -> Result<AddressBook>;
async fn get_address_book_by_id(&self, id: &Uuid) -> Result<AddressBook>;
async fn get_address_books_by_owner(&self, owner_id: &str) -> Result<Vec<AddressBook>>;
async fn update_address_book(&self, address_book: AddressBook) -> Result<AddressBook>;
async fn delete_address_book(&self, id: &Uuid) -> Result<()>;
}
// src/domain/repositories/contact_repository.rs
use async_trait::async_trait;
use uuid::Uuid;
use crate::domain::entities::contact::Contact;
use crate::common::errors::Result;
#[async_trait]
pub trait ContactRepository: Send + Sync {
async fn create_contact(&self, contact: Contact) -> Result<Contact>;
async fn get_contact_by_id(&self, id: &Uuid) -> Result<Contact>;
async fn get_contacts_by_address_book(&self, address_book_id: &Uuid) -> Result<Vec<Contact>>;
async fn search_contacts(&self, address_book_id: &Uuid, query: &str) -> Result<Vec<Contact>>;
async fn update_contact(&self, contact: Contact) -> Result<Contact>;
async fn delete_contact(&self, id: &Uuid) -> Result<()>;
}
```
3. **CardDAV Service**:
```rust
// src/application/services/contact_service.rs
use std::sync::Arc;
use crate::application::dtos::contact_dto::*;
use crate::application::dtos::address_book_dto::*;
pub struct ContactService {
// Implements AddressBookUseCase and ContactUseCase
// Uses ContactStorageAdapter as infrastructure
}
impl ContactService {
// Implements methods for CardDAV operations...
}
```
4. **CardDAV Handler**:
```rust
// src/interfaces/api/handlers/carddav_handler.rs
use std::sync::Arc;
use axum::{
Router,
routing::{get, put, delete},
extract::{Path, State, Request, Extension},
http::StatusCode,
response::Response,
};
use http::{Method, header};
use crate::common::di::AppState;
use crate::interfaces::middleware::auth::CurrentUser;
use crate::application::services::carddav_service::CardDavService;
use crate::common::errors::AppError;
pub fn carddav_routes() -> Router<Arc<AppState>> {
Router::new()
.route("/carddav/addressbooks/", get(get_address_books))
.route("/carddav/addressbooks/:addressbook", get(get_address_book))
.route_with_tsr("/carddav/addressbooks/:addressbook", axum::routing::on(
Method::PROPFIND, handle_addressbook_propfind,
Method::REPORT, handle_addressbook_report,
Method::MKCOL, handle_mkaddressbook,
))
.route("/carddav/addressbooks/:addressbook/:contact", get(get_contact))
.route("/carddav/addressbooks/:addressbook/:contact", put(put_contact))
.route("/carddav/addressbooks/:addressbook/:contact", delete(delete_contact))
}
// Implement functions for each CardDAV method...
```
## Database Schema
```sql
-- CalDAV schema
CREATE SCHEMA IF NOT EXISTS caldav;
CREATE TABLE IF NOT EXISTS caldav.calendars (
id UUID PRIMARY KEY,
name VARCHAR(255) NOT NULL,
owner_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
description TEXT,
color VARCHAR(50),
is_public BOOLEAN NOT NULL DEFAULT FALSE,
ctag VARCHAR(255),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS caldav.calendar_events (
id UUID PRIMARY KEY,
calendar_id UUID NOT NULL REFERENCES caldav.calendars(id) ON DELETE CASCADE,
summary VARCHAR(255) NOT NULL,
description TEXT,
location TEXT,
start_time TIMESTAMPTZ NOT NULL,
end_time TIMESTAMPTZ NOT NULL,
all_day BOOLEAN NOT NULL DEFAULT FALSE,
rrule TEXT,
ical_uid VARCHAR(255) NOT NULL,
ical_data TEXT NOT NULL,
etag VARCHAR(255),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS caldav.calendar_shares (
id SERIAL PRIMARY KEY,
calendar_id UUID NOT NULL REFERENCES caldav.calendars(id) ON DELETE CASCADE,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
access_level VARCHAR(20) NOT NULL DEFAULT 'read',
UNIQUE(calendar_id, user_id)
);
CREATE TABLE IF NOT EXISTS caldav.calendar_properties (
id SERIAL PRIMARY KEY,
calendar_id UUID NOT NULL REFERENCES caldav.calendars(id) ON DELETE CASCADE,
property_name VARCHAR(255) NOT NULL,
property_value TEXT,
UNIQUE(calendar_id, property_name)
);
-- CardDAV schema
CREATE SCHEMA IF NOT EXISTS carddav;
CREATE TABLE IF NOT EXISTS carddav.address_books (
id UUID PRIMARY KEY,
name VARCHAR(255) NOT NULL,
owner_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
description TEXT,
color VARCHAR(50),
is_public BOOLEAN NOT NULL DEFAULT FALSE,
ctag VARCHAR(255),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(owner_id, name)
);
CREATE TABLE IF NOT EXISTS carddav.contacts (
id UUID PRIMARY KEY,
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
uid VARCHAR(255) NOT NULL,
full_name VARCHAR(255),
first_name VARCHAR(255),
last_name VARCHAR(255),
nickname VARCHAR(255),
organization VARCHAR(255),
title VARCHAR(255),
notes TEXT,
photo_url TEXT,
birthday DATE,
anniversary DATE,
email JSONB,
phone JSONB,
address JSONB,
vcard TEXT NOT NULL,
etag VARCHAR(255) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(address_book_id, uid)
);
CREATE TABLE IF NOT EXISTS carddav.address_book_shares (
id SERIAL PRIMARY KEY,
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
can_write BOOLEAN NOT NULL DEFAULT FALSE,
UNIQUE(address_book_id, user_id)
);
CREATE TABLE IF NOT EXISTS carddav.contact_groups (
id UUID PRIMARY KEY,
address_book_id UUID NOT NULL REFERENCES carddav.address_books(id) ON DELETE CASCADE,
name VARCHAR(255) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS carddav.group_memberships (
id SERIAL PRIMARY KEY,
group_id UUID NOT NULL REFERENCES carddav.contact_groups(id) ON DELETE CASCADE,
contact_id UUID NOT NULL REFERENCES carddav.contacts(id) ON DELETE CASCADE,
UNIQUE(group_id, contact_id)
);
-- Indexes for efficient lookup
CREATE INDEX IF NOT EXISTS idx_calendars_owner ON caldav.calendars(owner_id);
CREATE INDEX IF NOT EXISTS idx_calendar_events_calendar ON caldav.calendar_events(calendar_id);
CREATE INDEX IF NOT EXISTS idx_address_books_owner ON carddav.address_books(owner_id);
CREATE INDEX IF NOT EXISTS idx_contacts_address_book ON carddav.contacts(address_book_id);
CREATE INDEX IF NOT EXISTS idx_contacts_full_name ON carddav.contacts(full_name);
```
## Security Considerations
1. **Authentication**
- Uses existing authentication
- Supports HTTP Basic Authentication for DAV clients
- Digest authentication can be added if needed
2. **Authorization**
- Verify user permissions before granting resource access
- Owner-based and shared-permission access control
- Users can only access their own calendars and address books
3. **Attack Prevention**
- Validate and sanitize all XML input
- Limit maximum payload size
- Rate limiting on DAV endpoints
## Testing and Compatibility
### Clients to Test
1. **WebDAV**
- Windows Explorer
- macOS Finder
- Cyberduck
- FileZilla (with WebDAV extension)
2. **CalDAV**
- Apple Calendar
- Mozilla Thunderbird (Lightning)
- Microsoft Outlook (with CalDAV add-in)
- Google Calendar (via sync)
3. **CardDAV**
- Apple Contacts
- Mozilla Thunderbird
- Microsoft Outlook (with CardDAV add-in)
- Google Contacts (via sync)
### Compliance Testing
- Use the **CalDAVTester** suite to verify standards conformance
- Validate RFC compliance for each protocol
- Stress tests to evaluate performance under load
### Debugging
- Detailed logging for DAV operations
- Diagnostic tools for debugging complex DAV requests
- Clear error messages for troubleshooting
doc/deduplication.md
-456
View File
@@ -1,456 +0,0 @@
# 06 - Deduplication
OxiCloud uses **content-defined chunking (CDC)** via FastCDC for sub-file deduplication. Files are split into variable-size chunks (64 KB – 1 MB, average 256 KB) using the FastCDC 2020 algorithm. Each chunk is individually BLAKE3-hashed and stored in a pluggable blob backend (local FS, S3, Azure). A PostgreSQL *manifest* maps the whole-file BLAKE3 hash to the ordered list of chunk hashes that compose it. Identical chunks across any files are stored once and reference-counted.
Deduplication is always enabled and non-fatal — if dedup fails, file operations proceed normally with a warning log.
**Backward compatibility**: files uploaded before CDC (legacy whole-file blobs in `storage.blobs`) are served transparently. When no manifest row exists for a hash, the service falls back to direct blob reads.
## Architecture
```
┌─────────────────┐ ┌─────────────────────┐ ┌───────────────┐
│ storage.files │────▶│ chunk_manifests │────▶│ storage.blobs │──▶ Blob Store
│ (references) │ │ (file→[chunk_hashes])│ │ (chunks) │ (Local/S3/Azure)
└─────────────────┘ └─────────────────────┘ └───────────────┘
```
### Database Tables
| Table | Schema | Purpose |
|---|---|---|
| `storage.chunk_manifests` | `storage` | Maps file_hash → ordered chunk_hashes[] + chunk_sizes[] + ref_count |
| `storage.blobs` | `storage` | Per-chunk metadata: hash (PK), size, ref_count, content_type |
Defined in `migrations/20260414000000_chunk_manifests.sql`. Blobs table is part of the initial schema.
### Layer Placement
| Layer | Component | File |
|---|---|---|
| Application Port | **DedupPort** trait + DTOs | `src/application/ports/dedup_ports.rs` |
| Application Port | **BlobStorageBackend** trait | `src/application/ports/blob_storage_ports.rs` |
| Infrastructure | **DedupService** implementation | `src/infrastructure/services/dedup_service.rs` |
| Infrastructure | Blob backends (Local, S3, Azure, Retry, Encrypted, Cached, Migration) | `src/infrastructure/services/*_blob_backend.rs` |
| Interfaces | **DedupHandler** REST endpoints | `src/interfaces/api/handlers/dedup_handler.rs` |
| Integration | **FileBlobWriteRepository** (dedup on upload) | `src/infrastructure/repositories/pg/file_blob_write_repository.rs` |
| Integration | **FileBlobReadRepository** (dedup reads) | `src/infrastructure/repositories/pg/file_blob_read_repository.rs` |
## Constants
| Constant | Value | Description |
|---|---|---|
| `CDC_MIN_CHUNK` | 64 KB (`65_536`) | Minimum CDC chunk size |
| `CDC_AVG_CHUNK` | 256 KB (`262_144`) | Average / target CDC chunk size |
| `CDC_MAX_CHUNK` | 1 MB (`1_048_576`) | Maximum CDC chunk size |
| `CHUNK_UPLOAD_CONCURRENCY` | 8 | Maximum parallel chunk uploads to blob backend |
Hardcoded in `dedup_service.rs`.
## Write Path: `store_from_file`
The core write operation follows a **write-first strategy** that never holds a PG connection during disk I/O:
```
store_from_file(source_path, content_type, pre_computed_hash)
│
├─ Fast path: pre_computed_hash provided?
│ └─ try_dedup_hit() → check manifest + legacy blob
│ └─ Hit? → bump ref_count, delete source, return ExistingBlob
│
├─ CDC analysis (single mmap pass, spawn_blocking):
│ ├─ Memory-map the file (memmap2)
│ ├─ FastCDC 2020 boundary detection → ChunkMeta[]
│ └─ BLAKE3 whole-file hash (concurrent with chunking)
│
├─ Second dedup check with computed hash (if no pre_computed_hash)
│
├─ store_chunks() — 3-phase pipeline:
│ │
│ ├─ Phase 0: Batch-check existing chunks (single PG query)
│ │ SELECT hash FROM storage.blobs WHERE hash = ANY($1)
│ │ → HashSet<String> of already-stored chunk hashes
│ │
│ ├─ Phase 1: Selective disk read (sequential, one pass)
│ │ For each chunk:
│ │ existing? → skip read (None)
│ │ new? → seek + read_exact → Some(Bytes)
│ │
│ └─ Phase 2: Parallel operations (buffer_unordered × 8)
│ new chunk: put_blob_from_bytes + INSERT ON CONFLICT
│ existing chunk: UPDATE ref_count + 1 (no disk I/O)
│
├─ INSERT manifest into storage.chunk_manifests
│ (file_hash, chunk_hashes[], chunk_sizes[], total_size, chunk_count)
│
└─ Delete source file, return NewBlob { hash, size }
```
### Dedup Skip Optimization
The **biggest I/O saving** for versioned files. Before reading any chunk from disk or uploading it to the blob backend, `store_chunks` batch-queries PG to discover which chunk hashes already exist:
```sql
SELECT hash FROM storage.blobs WHERE hash = ANY($1)
```
This single round-trip returns all known chunks. For each existing chunk, the service skips:
- `seek()` + `read_exact()` from the source file (no disk I/O)
- `put_blob_from_bytes()` to the backend (no network I/O for S3/Azure)
Only a lightweight `UPDATE ref_count + 1` is executed in PG (~0.1 ms per chunk).
**Impact**: for a 100 MB versioned file where 95% of chunks are unchanged, only ~5 MB is read from disk and uploaded. The remaining 95% costs only PG ref-count bumps.
### Parallel Chunk Storage
Phase 2 of `store_chunks` uses `futures::stream::buffer_unordered(8)` to execute up to 8 concurrent chunk operations. This is a major win for S3/Azure backends where each PUT has 50-200 ms of network latency.
Chunk order in the returned `(chunk_hashes, chunk_sizes)` is preserved by deriving both from the original `ChunkMeta` slice (CDC order), not from the unordered parallel results.
### Full-File Dedup Hit (Fast Path)
When a file with the exact same BLAKE3 hash already has a manifest, `try_dedup_hit` returns immediately:
- Bumps `chunk_manifests.ref_count`
- Deletes the source file
- Returns `ExistingBlob` — **zero chunk I/O**
Also checks legacy whole-file blobs in `storage.blobs` for backward compatibility.
## Read Path
### Streaming Read (`read_blob_stream`)
CDC-aware with legacy fallback:
1. Query `chunk_manifests` for `chunk_hashes[]`
2. If found: stream chunks in order via `backend.get_blob_stream(chunk_hash)`, concatenated into a single byte stream with `buffered(1) + try_flatten`
3. If not found: fall back to `backend.get_blob_stream(hash)` for legacy blobs
### Range Read (`read_blob_range_stream`)
For HTTP Range requests (and WOPI/WebDAV partial reads):
1. Query manifest for `chunk_hashes[]`, `chunk_sizes[]`, `total_size`
2. Calculate which chunks overlap `[start, end)` using cumulative offsets
3. For each overlapping chunk, compute the sub-range within that chunk
4. Stream only the relevant chunk portions via `backend.get_blob_range_stream()`
### Blob Size (`blob_size`)
Returns `total_size` from the manifest (O(1) PG lookup). Falls back to `backend.blob_size()` for legacy blobs. Used by HEAD requests for Content-Length.
## Reference Counting
### Adding References (`add_reference`)
Manifest-aware with legacy fallback:
1. Try `UPDATE chunk_manifests SET ref_count = ref_count + 1 WHERE file_hash = $1`
2. If no rows affected, try `UPDATE storage.blobs SET ref_count + 1 WHERE hash = $1`
3. If neither exists, return NotFound error
### Removing References (`remove_reference`)
**CDC manifest path** (transactional):
1. Check `chunk_manifests` for the file hash
2. If `ref_count > 1`: decrement manifest ref_count → commit
3. If `ref_count == 1` (last reference):
- `SELECT ... FOR UPDATE` to lock the manifest row
- `DELETE FROM chunk_manifests`
- `UPDATE storage.blobs SET ref_count = ref_count - 1 WHERE hash = ANY(chunk_hashes)`
- `DELETE FROM storage.blobs WHERE hash = ANY(chunk_hashes) AND ref_count <= 0 RETURNING hash`
- Commit TX
- Delete orphaned chunk blob files from backend (after commit)
**Legacy blob path** (transactional):
1. `SELECT ref_count, size FROM storage.blobs WHERE hash = $1 FOR UPDATE`
2. If `ref_count == 1`: `DELETE FROM storage.blobs` + delete blob file
3. If `ref_count > 1`: `UPDATE SET ref_count = ref_count - 1`
## Port: DedupPort Trait
Defined in `src/application/ports/dedup_ports.rs`:
```rust
pub trait DedupPort: Send + Sync + 'static {
/// Store content with CDC deduplication (from file).
async fn store_from_file(
&self,
source_path: &Path,
content_type: Option<String>,
pre_computed_hash: Option<String>,
) -> Result<DedupResultDto, DomainError>;
/// Check if a blob exists by hash (manifest or legacy).
async fn blob_exists(&self, hash: &str) -> bool;
/// Get metadata for a blob.
async fn get_blob_metadata(&self, hash: &str) -> Option<BlobMetadataDto>;
/// Stream blob content — CDC-aware with legacy fallback.
async fn read_blob_stream(&self, hash: &str)
-> Result<Pin<Box<dyn Stream<Item = Result<Bytes, io::Error>> + Send>>, DomainError>;
/// Stream a byte range — CDC-aware with legacy fallback.
async fn read_blob_range_stream(&self, hash: &str, start: u64, end: Option<u64>)
-> Result<Pin<Box<dyn Stream<Item = Result<Bytes, io::Error>> + Send>>, DomainError>;
/// Get blob size without reading content.
async fn blob_size(&self, hash: &str) -> Result<u64, DomainError>;
/// Increment reference count (manifest-aware).
async fn add_reference(&self, hash: &str) -> Result<(), DomainError>;
/// Decrement reference count. Returns true if blob was deleted.
async fn remove_reference(&self, hash: &str) -> Result<bool, DomainError>;
/// Calculate BLAKE3 hash of a file (mmap + rayon).
async fn hash_file(&self, path: &Path) -> Result<String, DomainError>;
/// Get local filesystem path for a blob hash.
fn blob_path(&self, hash: &str) -> PathBuf;
/// Get deduplication statistics (computed from PG).
async fn get_stats(&self) -> DedupStatsDto;
/// Flush index to persistent storage (no-op for PG backend).
async fn flush(&self) -> Result<(), DomainError>;
/// Verify integrity of all stored blobs and manifests.
async fn verify_integrity(&self) -> Result<Vec<String>, DomainError>;
}
```
### Port DTOs
```rust
/// Result of a dedup store operation.
pub enum DedupResultDto {
NewBlob { hash: String, size: u64 },
ExistingBlob { hash: String, size: u64, saved_bytes: u64 },
}
// Methods: hash(), size(), was_deduplicated()
/// Metadata for a stored blob.
pub struct BlobMetadataDto {
pub hash: String, // BLAKE3 hex string
pub size: u64,
pub ref_count: u32,
pub content_type: Option<String>,
}
/// Aggregate dedup statistics (computed from PG).
pub struct DedupStatsDto {
pub total_blobs: u64,
pub total_bytes_stored: u64,
pub total_bytes_referenced: u64,
pub bytes_saved: u64,
pub dedup_hits: u64,
pub dedup_ratio: f64,
}
```
## Infrastructure: DedupService
Implemented in `src/infrastructure/services/dedup_service.rs`.
### Struct
```rust
pub struct DedupService {
backend: Arc<dyn BlobStorageBackend>, // Pluggable blob storage (Local/S3/Azure/...)
pool: Arc<PgPool>, // Primary pool (request-path operations)
maintenance_pool: Arc<PgPool>, // Isolated pool (verify_integrity, GC)
}
```
### Key Methods
| Method | Description |
|---|---|
| `new(backend, pool, maintenance_pool)` | Construct — wires pluggable backend + dual PG pools |
| `initialize()` | Initialize backend + log blob/manifest counts from PG |
| `cdc_hash_and_chunk_file(path)` | Single mmap pass: BLAKE3 whole-file hash + FastCDC chunk boundaries + per-chunk BLAKE3 |
| `cdc_chunk_file(path)` | CDC without whole-file hash (when hash is pre-computed) |
| `hash_file(path)` | BLAKE3 hash via mmap + rayon parallelism |
| `store_from_file(path, ct, hash)` | CDC → store_chunks → manifest INSERT (main write path) |
| `try_dedup_hit(hash, path)` | Check manifest/legacy for full-file dedup hit |
| `store_chunks(path, chunks)` | 3-phase: batch-check → selective read → parallel upload |
| `blob_exists(hash)` | Check manifest + legacy blob existence |
| `user_owns_blob_reference(hash, user_id)` | Authorization: check file ownership |
| `get_blob_metadata(hash)` | Manifest-aware metadata with legacy fallback |
| `add_reference(hash)` | Manifest-aware ref_count increment |
| `remove_reference(hash)` | Manifest-aware ref_count decrement + cascade cleanup |
| `read_blob_stream(hash)` | CDC chunk-streaming with legacy fallback |
| `read_blob_range_stream(hash, start, end)` | CDC range-streaming with legacy fallback |
| `blob_size(hash)` | O(1) from manifest, fallback to backend |
| `get_stats()` | Compute stats from PG (blobs + manifests) |
| `verify_integrity()` | Verify manifests (counts, sizes) + blobs (existence, size, re-hash) |
| `garbage_collect()` | Batch-delete orphaned manifests/blobs (uses maintenance pool) |
### Key Behaviors
- **CDC analysis in `spawn_blocking`**: mmap + FastCDC runs off the async runtime to avoid blocking the event loop
- **Dual PG pools**: request-path operations use the primary pool; `verify_integrity` and `garbage_collect` use the maintenance pool to prevent starvation
- **Pluggable blob backend**: all chunk I/O goes through `Arc<dyn BlobStorageBackend>` — works with local FS, S3, Azure, or any composed backend (retry, encryption, caching)
- **Atomic chunk storage**: `put_blob_from_bytes` is idempotent; `INSERT ON CONFLICT` handles concurrent uploads of the same chunk
- **Delete-after-commit**: blob files are deleted from the backend only after the PG transaction commits, preventing orphaned PG rows
- **Flush is no-op**: PG handles durability via WAL/commit — no explicit index persistence needed
## REST API Endpoints
All routes under `/api/dedup`, authentication required.
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/dedup/check/{hash}` | `DedupHandler::check_hash` | Check if a blob exists by BLAKE3 hash |
| `POST` | `/api/dedup/upload` | `DedupHandler::upload_with_dedup` | Multipart upload with automatic dedup |
| `GET` | `/api/dedup/stats` | `DedupHandler::get_stats` | Get deduplication statistics |
| `GET` | `/api/dedup/blob/{hash}` | `DedupHandler::get_blob` | Retrieve raw blob content by hash |
| `DELETE` | `/api/dedup/blob/{hash}` | `DedupHandler::remove_reference` | Decrement ref-count (deletes blob if 0) |
| `POST` | `/api/dedup/recalculate` | `DedupHandler::recalculate_stats` | Run integrity verification + refresh stats |
### API Response Types
**Hash Check** (`GET /api/dedup/check/{hash}`):
```json
{
"exists": true,
"hash": "a1b2c3d4...",
"existing_size": 1048576,
"ref_count": 3
}
```
**Dedup Upload** (`POST /api/dedup/upload`):
```json
{
"is_new": false,
"hash": "a1b2c3d4...",
"size": 1048576,
"bytes_saved": 1048576,
"ref_count": 2
}
```
**Stats** (`GET /api/dedup/stats`):
```json
{
"unique_blobs": 150,
"total_references": 300,
"bytes_saved": 524288000,
"total_logical_bytes": 1073741824,
"total_physical_bytes": 549453824,
"dedup_ratio": 2.0,
"savings_percentage": 48.8
}
```
## DI Wiring
In `src/common/di.rs`:
```rust
// Blob backend is assembled from layered backends:
// LocalBlobBackend / S3BlobBackend / AzureBlobBackend
// → RetryBlobBackend → EncryptedBlobBackend → CachedBlobBackend
let blob_backend: Arc<dyn BlobStorageBackend> = /* ... */;
// DedupService receives the composed blob backend + dual PG pools
let dedup_service = Arc::new(DedupService::new(
blob_backend,
db_pool.clone(),
maintenance_pool.clone(),
));
dedup_service.initialize().await?;
// Stored in CoreServices as:
pub struct CoreServices {
pub dedup_service: Arc<dyn DedupPort>,
// ...
}
// Injected into blob repositories:
FileBlobReadRepository::new(pool, core.dedup_service.clone(), folder_repo)
FileBlobWriteRepository::new(pool, core.dedup_service.clone(), folder_repo)
```
## Maintenance
### Garbage Collection (`garbage_collect`)
Two-phase batch deletion using the maintenance pool:
1. **Phase 1 — Orphaned manifests**: `DELETE FROM chunk_manifests WHERE ref_count <= 0` (batches of 500). For each deleted manifest, `UPDATE storage.blobs SET ref_count = ref_count - 1` for its chunks.
2. **Phase 2 — Orphaned blobs**: `DELETE FROM storage.blobs WHERE ref_count <= 0` (batches of 500). Deletes blob files from backend + thumbnail cleanup (best-effort).
Uses `tokio::task::yield_now()` between batches to avoid starving other tasks.
### Integrity Verification (`verify_integrity`)
Phase 1 — Verify CDC manifests:
- `chunk_hashes.len() == chunk_sizes.len()`
- `SUM(chunk_sizes) == total_size`
- Every referenced chunk exists in the blob backend with correct size
Phase 2 — Verify blobs (chunks + legacy):
- Blob file exists in backend
- Actual size matches PG record
- (Local backends only) Re-hash file content to verify BLAKE3 integrity
- Processes 16 blobs concurrently via `buffer_unordered`
## Tests
Located at the bottom of `src/infrastructure/services/dedup_service.rs` (12 tests):
| Test | Description |
|---|---|
| `test_cdc_deterministic_same_content` | Same content → same file hash + same chunk hashes/offsets/lengths |
| `test_cdc_empty_file` | Empty file → zero chunks, correct BLAKE3 empty hash |
| `test_cdc_small_file_single_chunk` | File below min chunk → single chunk covering entire file |
| `test_cdc_chunk_sizes_within_bounds` | All non-last chunks are within [64 KB, 1 MB] |
| `test_cdc_file_hash_matches_hash_file` | CDC whole-file hash matches standalone `hash_file()` |
| `test_cdc_chunk_hashes_are_correct` | Each chunk hash == BLAKE3 of that chunk's data |
| `test_cdc_reassembly_matches_original` | Concatenating chunks reproduces original file |
| `test_cdc_chunks_are_contiguous` | Chunks cover entire file with no gaps or overlaps |
| `test_cdc_similar_files_share_chunks` | Editing last 64 KB of 2 MB file → most chunks shared |
| `test_cdc_chunk_file_matches_full` | `cdc_chunk_file` produces same chunks as `cdc_hash_and_chunk_file` |
| `test_cdc_large_file_chunk_count` | 8 MB file produces 8-128 chunks (avg ~256 KB) |
| `test_cdc_insert_at_beginning_preserves_later_chunks` | 128 KB prefix insert → CDC resynchronizes, later chunks shared |
## Performance Characteristics
| Scenario | Behavior |
|---|---|
| **First upload of new file** | Single mmap pass (CDC + hash) → parallel chunk upload → manifest INSERT |
| **Re-upload of identical file** | `try_dedup_hit` → manifest ref_count bump → zero chunk I/O |
| **Upload of edited file (5% changed)** | CDC → batch-check finds 95% existing → reads only 5% → uploads 5% → ref-bumps 95% |
| **Range read (1 MB from 1 GB file)** | Manifest lookup → identify overlapping chunks → stream only those portions |
| **Delete last reference** | TX: delete manifest → batch-decrement chunks → delete zero-ref chunks → commit → delete blob files |
| **Garbage collection** | Maintenance pool, batches of 500, yields between batches |
## Client Usage Example
```bash
# 1. Check if file already exists by hash
HASH=$(b3sum myfile.txt | cut -d' ' -f1)
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/dedup/check/$HASH"
# 2. Upload with dedup (if not exists)
curl -X POST -H "Authorization: Bearer $TOKEN" \
-F "file=@myfile.txt" \
"https://oxicloud.example.com/api/dedup/upload"
# 3. Get dedup statistics
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/dedup/stats"
# 4. Retrieve blob content
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/dedup/blob/$HASH" -o output.bin
# 5. Recalculate stats with integrity check
curl -X POST -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/dedup/recalculate"
```
doc/deployment.md
-247
View File
@@ -1,247 +0,0 @@
# 02 - Deployment
OxiCloud is deployed as a containerized application with PostgreSQL.
---
## Docker Setup
### Quick Start
1. **Copy the example environment file and customize it:**
```bash
cp example.env .env
# Edit .env with your settings
```
2. **Start the services:**
```bash
docker compose up -d
```
3. **Access OxiCloud:**
Open `http://localhost:8086` in your browser.
### Docker Compose
```yaml
# docker-compose.yml
services:
postgres:
image: postgres:17.4-alpine
environment:
POSTGRES_DB: oxicloud
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
volumes:
- pg_data:/var/lib/postgresql/
- ./db/schema.sql:/docker-entrypoint-initdb.d/schema.sql
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 5s
retries: 5
oxicloud:
image: diocrafts/oxicloud:latest
ports:
- "8086:8086"
env_file:
- .env
volumes:
- storage_data:/app/storage
depends_on:
postgres:
condition: service_healthy
volumes:
pg_data:
storage_data:
```
### Dockerfile
3-stage Alpine-based build:
1. **Cacher** -- pre-builds dependency layer
2. **Builder** -- compiles OxiCloud (`rust:1.93.0-alpine3.23`)
3. **Runtime** -- minimal Alpine image (`alpine:3.23.3`) with `libgcc`, `ca-certificates`, `libpq`, `tzdata`, `su-exec`
Non-root user: `oxicloud` (UID/GID 1001). Exposed port: `8086`. Entrypoint: `entrypoint.sh` (chown storage + drop privileges via `su-exec`).
---
## Environment Variables
### Server
| Variable | Default | Description |
|---|---|---|
| `OXICLOUD_STORAGE_PATH` | `./storage` | Root storage directory |
| `OXICLOUD_STATIC_PATH` | `./static` | Static files directory |
| `OXICLOUD_SERVER_PORT` | `8086` | Server port |
| `OXICLOUD_SERVER_HOST` | `127.0.0.1` | Server bind address |
| `OXICLOUD_BASE_URL` | (auto-detected) | Public base URL for share links. If not set, uses `http://{host}:{port}` |
### Database
| Variable | Default | Description |
|---|---|---|
| `OXICLOUD_DB_CONNECTION_STRING` | `postgres://postgres:postgres@localhost:5432/oxicloud` | PostgreSQL connection string |
| `OXICLOUD_DB_MAX_CONNECTIONS` | `20` | Max pool connections |
| `OXICLOUD_DB_MIN_CONNECTIONS` | `5` | Min pool connections |
### Authentication
| Variable | Default | Description |
|---|---|---|
| `OXICLOUD_JWT_SECRET` | (auto-generated) | JWT signing secret. If empty, a random 32-byte hex secret is generated per session |
| `OXICLOUD_ACCESS_TOKEN_EXPIRY_SECS` | `3600` (1h) | Access token lifetime |
| `OXICLOUD_REFRESH_TOKEN_EXPIRY_SECS` | `2592000` (30d) | Refresh token lifetime |
### Feature Flags
| Variable | Default | Description |
|---|---|---|
| `OXICLOUD_ENABLE_AUTH` | `true` | Enable authentication system |
| `OXICLOUD_ENABLE_USER_STORAGE_QUOTAS` | `false` | Enable per-user storage quotas |
| `OXICLOUD_ENABLE_FILE_SHARING` | `true` | Enable file/folder sharing |
| `OXICLOUD_ENABLE_TRASH` | `true` | Enable trash/recycle bin |
| `OXICLOUD_ENABLE_SEARCH` | `true` | Enable search functionality |
### OIDC / SSO
| Variable | Default | Description |
|---|---|---|
| `OXICLOUD_OIDC_ENABLED` | `false` | Enable OIDC authentication |
| `OXICLOUD_OIDC_ISSUER_URL` | (empty) | OIDC provider issuer URL |
| `OXICLOUD_OIDC_CLIENT_ID` | (empty) | OIDC client ID |
| `OXICLOUD_OIDC_CLIENT_SECRET` | (empty) | OIDC client secret |
| `OXICLOUD_OIDC_REDIRECT_URI` | `http://localhost:8086/api/auth/oidc/callback` | Callback URL |
| `OXICLOUD_OIDC_SCOPES` | `openid profile email` | Requested OIDC scopes |
| `OXICLOUD_OIDC_FRONTEND_URL` | `http://localhost:8086` | Frontend URL for redirects |
| `OXICLOUD_OIDC_AUTO_PROVISION` | `true` | Auto-create users on first login |
| `OXICLOUD_OIDC_ADMIN_GROUPS` | (empty) | OIDC groups that grant admin role |
| `OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN` | `false` | Disable password login when OIDC is active |
| `OXICLOUD_OIDC_PROVIDER_NAME` | `SSO` | Display name for the OIDC provider |
### OIDC Validation
If **OXICLOUD_OIDC_ENABLED** is `true` but **issuer_url**, **client_id**, or **client_secret** are empty, OIDC is automatically disabled with an error log.
### WOPI (Office Document Editing)
| Variable | Default | Description |
|---|---|---|
| `OXICLOUD_WOPI_ENABLED` | `false` | Enable WOPI integration for office document editing |
| `OXICLOUD_WOPI_DISCOVERY_URL` | (empty) | WOPI client discovery URL (e.g., Collabora, OnlyOffice) |
| `OXICLOUD_WOPI_SECRET` | (falls back to JWT secret) | Secret key for signing WOPI access tokens |
| `OXICLOUD_WOPI_TOKEN_TTL_SECS` | `86400` (24h) | WOPI access token lifetime |
| `OXICLOUD_WOPI_LOCK_TTL_SECS` | `1800` (30m) | WOPI lock expiration time |
---
## Internal Configuration (Not Environment-Configurable)
Hardcoded defaults in `src/common/config.rs`:
### Cache
| Parameter | Default |
|---|---|
| File cache TTL | 60,000 ms (1 min) |
| Directory cache TTL | 120,000 ms (2 min) |
| Max cache entries | 10,000 |
### Timeouts
| Parameter | Default |
|---|---|
| File operation | 10,000 ms |
| Directory operation | 30,000 ms |
| Lock acquisition | 5,000 ms |
| Network operation | 15,000 ms |
### Resources
| Parameter | Default |
|---|---|
| Large file threshold | 100 MB |
| Large directory threshold | 1,000 entries |
| Streaming chunk size | 1 MB |
| Max in-memory file size | 50 MB |
### Concurrency
| Parameter | Default |
|---|---|
| Max concurrent files | 10 |
| Max concurrent dirs | 5 |
| Max concurrent I/O | 20 |
| Max parallel chunks | 8 |
| Min size for parallel chunks | 200 MB |
| Parallel chunk size | 8 MB |
### Storage
| Parameter | Default |
|---|---|
| Trash retention | 30 days |
### Auth Hashing (Argon2id)
| Parameter | Default |
|---|---|
| Memory cost | 65,536 KB (64 MB) |
| Time cost | 3 iterations |
---
## Feature Dependency Matrix
| Feature | Requires DB | Requires Auth | Feature Flag |
|---|---|---|---|
| File storage | Yes | No | Always on |
| Authentication | Yes | -- | `OXICLOUD_ENABLE_AUTH` |
| OIDC / SSO | Yes | Yes | `OXICLOUD_OIDC_ENABLED` |
| File sharing | Yes | Yes | `OXICLOUD_ENABLE_FILE_SHARING` |
| Trash | Yes | No | `OXICLOUD_ENABLE_TRASH` |
| Search | Yes | No | `OXICLOUD_ENABLE_SEARCH` |
| Favorites | Yes | Yes | Always on (when DB available) |
| Recent items | Yes | Yes | Always on (when DB available) |
| Storage quotas | Yes | Yes | `OXICLOUD_ENABLE_USER_STORAGE_QUOTAS` |
| Admin panel | Yes | Yes | Always on (when auth enabled) |
| WebDAV | Yes | Optional | Always on |
| CalDAV | Yes | Yes | Always on (when DB available) |
| CardDAV | Yes | Yes | Always on (when DB available) |
| Deduplication | No | No | Always on |
| Thumbnails | No | No | Always on |
| Chunked uploads | No | No | Always on |
# Kubernetes Setup
## 01 - Dependencies
- K8S Cluster
- Default StorageClass
- Ingress Controller
- Helm
## 02 - Deployment
Change the `charts/oxicloud/values.yaml` to match your setup.
To install the Chart run:
```bash
helm upgrade --install oxicloud charts/oxicloud -f charts/oxicloud/values.yaml
```
## 03 - Verification
To verify your deployment run:
```bash
kubectl get pods -n oxicloud
kubectl logs statefulset/oxicloud -n oxicloud
```
If you have collabora enabled you want to check the logs:
```bash
kubectl logs statefulset/oxicloud -n oxicloud | grep "WOPI discovery loaded"
```
doc/favorites-and-recent.md
-171
View File
@@ -1,171 +0,0 @@
# 13 - Favorites and Recent Items
Two per-user item tracking features:
- **Favorites** -- users mark files and folders for quick access.
- **Recent Items** -- automatically tracks recently accessed files and folders.
Both require PostgreSQL and are only available when a database connection is configured.
---
## Favorites
### Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **FavoritesUseCase**, **FavoritesRepositoryPort** | `src/application/ports/favorites_ports.rs` |
| Application Service | **FavoritesService** | `src/application/services/favorites_service.rs` |
| Application DTO | **FavoriteItemDto** | `src/application/dtos/favorites_dto.rs` |
| Infrastructure | **FavoritesPgRepository** | `src/infrastructure/repositories/pg/favorites_pg_repository.rs` |
| Interfaces | `favorites_handler` (free functions) | `src/interfaces/api/handlers/favorites_handler.rs` |
### DTO
```rust
pub struct FavoriteItemDto {
pub id: String,
pub user_id: String,
pub item_id: String,
pub item_type: String, // "file" | "folder"
pub created_at: DateTime<Utc>,
}
```
### REST API
All routes under `/api/favorites`, require authentication. User ID comes from the JWT token.
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/favorites/` | `get_favorites` | List all favorites for current user |
| `POST` | `/api/favorites/{item_type}/{item_id}` | `add_favorite` | Add a file or folder to favorites |
| `DELETE` | `/api/favorites/{item_type}/{item_id}` | `remove_favorite` | Remove from favorites |
- `item_type` must be `"file"` or `"folder"` (validated by service)
- Adding a duplicate is idempotent (`ON CONFLICT DO NOTHING`)
- Results ordered by `created_at DESC`
### Database Schema
```sql
CREATE TABLE IF NOT EXISTS auth.user_favorites (
id SERIAL PRIMARY KEY,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
item_id TEXT NOT NULL,
item_type TEXT NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, item_id, item_type)
);
```
Indexes: `user_id`, `item_id`, `item_type`, `created_at`, composite `(user_id, item_type)`.
### Example
```bash
# Add file to favorites
curl -X POST -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/favorites/file/abc-123"
# List favorites
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/favorites/"
# Remove from favorites
curl -X DELETE -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/favorites/folder/def-456"
```
---
## Recent Items
### Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **RecentItemsUseCase**, **RecentItemsRepositoryPort** | `src/application/ports/recent_ports.rs` |
| Application Service | **RecentService** | `src/application/services/recent_service.rs` |
| Application DTO | **RecentItemDto** | `src/application/dtos/recent_dto.rs` |
| Infrastructure | **RecentItemsPgRepository** | `src/infrastructure/repositories/pg/recent_items_pg_repository.rs` |
| Interfaces | `recent_handler` (free functions) | `src/interfaces/api/handlers/recent_handler.rs` |
### DTO
```rust
pub struct RecentItemDto {
pub id: String,
pub user_id: String,
pub item_id: String,
pub item_type: String, // "file" | "folder"
pub accessed_at: DateTime<Utc>,
}
```
### REST API
All routes under `/api/recent`, require authentication.
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/recent/` | `get_recent_items` | List recent items (optional `?limit=N`) |
| `POST` | `/api/recent/{item_type}/{item_id}` | `record_item_access` | Record an access (upsert) |
| `DELETE` | `/api/recent/{item_type}/{item_id}` | `remove_from_recent` | Remove specific item |
| `DELETE` | `/api/recent/clear` | `clear_recent_items` | Clear all recent items |
### Behavior
- **Max items per user**: 50 (configured in DI, clamped to 1-100)
- **Upsert**: re-accessing an item updates its `accessed_at` timestamp
- **Auto-prune**: after recording access, old items beyond the limit are automatically pruned
- **Ordering**: results ordered by `accessed_at DESC`
- **Limit parameter**: `?limit=N` caps results (defaults to and cannot exceed **max_recent_items**)
### Database Schema
```sql
CREATE TABLE IF NOT EXISTS auth.user_recent_files (
id SERIAL PRIMARY KEY,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
item_id TEXT NOT NULL,
item_type TEXT NOT NULL,
accessed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, item_id, item_type)
);
```
Indexes: `user_id`, `item_id`, `item_type`, `accessed_at`, composite `(user_id, accessed_at DESC)`.
### Example
```bash
# Record file access
curl -X POST -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/recent/file/abc-123"
# Get recent items (last 10)
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/recent/?limit=10"
# Clear history
curl -X DELETE -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/recent/clear"
```
## DI Wiring
Both services require PostgreSQL:
```rust
// Favorites
let repo = Arc::new(FavoritesPgRepository::new(db_pool.clone()));
let favorites_service = Arc::new(FavoritesService::new(repo));
// Recent
let repo = Arc::new(RecentItemsPgRepository::new(db_pool.clone()));
let recent_service = Arc::new(RecentService::new(repo, 50)); // max 50 items
```
Stored as `Option<Arc<...>>` in **AppState** -- only available when DB is connected.
doc/file-system-safety.md
-125
View File
@@ -1,125 +0,0 @@
# 03 - Storage Safety
OxiCloud ensures data integrity and durability through a combination of PostgreSQL transactional guarantees and atomic blob writes. The goal: writes either complete fully or not at all, data reaches persistent storage, and the system recovers from crashes or power loss.
---
## Storage Model
OxiCloud uses a **100% blob storage model**:
- **Metadata** (file names, folder hierarchy, sizes, MIME types, trash status) lives in **PostgreSQL** — protected by ACID transactions.
- **File content** is stored as content-addressed blobs via **DedupService** at `.blobs/{prefix}/{hash}.blob` — protected by atomic writes and fsync.
---
## PostgreSQL Safety (Metadata)
All file and folder metadata operations use PostgreSQL transactions:
- **Single-row operations** (INSERT, UPDATE, DELETE) are inherently atomic.
- **Multi-step operations** (e.g., move file: UPDATE folder_id + UPDATE path) use explicit transactions via `sqlx`.
- **Foreign key constraints** prevent orphaned records (e.g., files referencing non-existent folders).
- **Unique constraints** prevent duplicate names within the same parent folder.
- **Soft-delete** for trash (`is_trashed = TRUE`) preserves data until explicit permanent deletion.
The `storage.trash_items` VIEW provides a unified read interface over trashed files and folders without duplicating data.
---
## Blob Storage Safety (Content)
### DedupService Atomic Writes
**File:** `src/infrastructure/services/dedup_service.rs`
When storing file content, DedupService uses the following pattern:
1. **Hash computation** — SHA-256 hash of content determines the blob path
2. **Deduplication check** — if a blob with the same hash exists, only increment the reference counter (no write needed)
3. **Atomic write** — if new content:
- Write to a temporary file (`.blob.tmp`)
- Call `fsync` to ensure data reaches persistent storage
- Atomically rename temp file to final path (`.blobs/{prefix}/{hash}.blob`)
4. **Reference counting** — track how many files reference each blob
This ensures that a blob either fully exists or doesn't — no partial writes.
### FileSystemUtils
**File:** `src/infrastructure/services/file_system_utils.rs`
Low-level utilities used internally by DedupService and other infrastructure services:
```rust
/// Atomic write: temp file → fsync → rename
pub async fn atomic_write<P: AsRef<Path>>(path: P, contents: &[u8]) -> Result<(), IoError>
/// Directory creation with fsync
pub async fn create_dir_with_sync<P: AsRef<Path>>(path: P) -> Result<(), IoError>
/// Rename with directory sync
pub async fn rename_with_sync<P, Q>(from: P, to: Q) -> Result<(), IoError>
/// Delete with directory sync
pub async fn remove_file_with_sync<P: AsRef<Path>>(path: P) -> Result<(), IoError>
```
### fsync Guarantees
- `sync_all()` on written files ensures data and metadata reach the physical storage device
- Directory entries are synced after create/rename/delete operations
- Prevents data loss during crashes or power failures between OS buffer flush and disk write
---
## Transaction Flow: File Upload
```
1. DedupService.store_bytes(content)
→ Compute SHA-256 hash
→ Check if blob exists (dedup hit → increment ref, return hash)
→ Write to .blobs/{prefix}/{hash}.blob.tmp
→ fsync + rename → .blobs/{prefix}/{hash}.blob
2. FileBlobWriteRepository.save_file()
→ BEGIN TRANSACTION
→ INSERT INTO storage.files (name, folder_id, blob_hash, size, ...)
→ COMMIT
```
If step 1 fails, no metadata is written. If step 2 fails, the blob exists but is unreferenced (cleaned up by garbage collection). Data is never in an inconsistent state.
## Transaction Flow: File Deletion
```
1. FileBlobWriteRepository.delete_file_permanently()
→ BEGIN TRANSACTION
→ DELETE FROM storage.files WHERE id = $1 (captures blob_hash first)
→ COMMIT
2. DedupService.decrement_ref(blob_hash)
→ Decrement reference counter
→ If counter reaches 0, delete the blob file
```
If step 2 fails, an unreferenced blob may remain on disk (occupies space but is not a correctness issue). Future garbage collection can clean these up.
---
## Benefits
1. **ACID transactions** — metadata operations are atomic, consistent, isolated, and durable
2. **Content-addressable storage** — identical content is stored once, referenced by hash
3. **Crash resilience** — atomic blob writes + PostgreSQL WAL ensure recovery
4. **No partial writes** — temp file + rename pattern guarantees all-or-nothing
5. **Referential integrity** — foreign keys prevent orphaned metadata
---
## Performance Considerations
- PostgreSQL connection pooling (`sqlx::PgPool`) amortizes connection overhead
- Dedup hash computation is CPU-bound but avoids unnecessary disk writes for duplicate content
- Blob fsync adds latency vs. buffered writes, but ensures durability for critical user data
- Content cache (in-memory LRU) serves repeat reads without disk or DB access
doc/i18n.md
-100
View File
@@ -1,100 +0,0 @@
# 16 - Internationalization
JSON-based translation system. Translations are loaded from static files, cached in memory, and served via a public REST API (no auth required).
## Supported Languages
| Locale | Code | File |
|---|---|---|
| English | `en` | `static/locales/en.json` |
| Spanish | `es` | `static/locales/es.json` |
| French | `fr` | `static/locales/fr.json` |
| German | `de` | `static/locales/de.json` |
| Portuguese | `pt` | `static/locales/pt.json` |
| Persian | `fa` | `static/locales/fa.json` |
| Chinese | `zh` | `static/locales/zh.json` |
| Dutch | `nl` | `static/locales/nl.json` |
Default locale: **en** (English).
## Architecture
| Layer | Component | File |
|---|---|---|
| Domain Port | **I18nService** trait, **Locale** enum | `src/domain/services/i18n_service.rs` |
| Application Service | **I18nApplicationService** | `src/application/services/i18n_application_service.rs` |
| Application DTOs | **LocaleDto**, **TranslationRequestDto**, etc. | `src/application/dtos/i18n_dto.rs` |
| Infrastructure | **FileSystemI18nService** | `src/infrastructure/services/file_system_i18n_service.rs` |
| Interfaces | **I18nHandler** | `src/interfaces/api/handlers/i18n_handler.rs` |
## REST API
Public endpoints (no authentication), under `/api/i18n`:
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/i18n/locales` | `get_locales` | List available locales |
| `GET` | `/api/i18n/translate` | `translate` | Translate a key (`?key=...&locale=...`) |
| `GET` | `/api/i18n/locales/{locale_code}` | `get_translations_by_locale` | Get all translations for a locale |
### Examples
```bash
# List available locales
curl "https://oxicloud.example.com/api/i18n/locales"
# [{"code":"en","name":"English"},{"code":"es","name":"Spanish"}, ...]
# Translate a key
curl "https://oxicloud.example.com/api/i18n/translate?key=app.title&locale=es"
# {"key":"app.title","locale":"es","text":"OxiCloud"}
# Get all translations for a locale
curl "https://oxicloud.example.com/api/i18n/locales/en"
# { "app": { "title": "OxiCloud", ... }, "nav": { ... }, ... }
```
## Translation File Format
Nested JSON with dot-delimited key lookups:
```json
{
"app": {
"title": "OxiCloud",
"description": "Your personal cloud storage"
},
"nav": {
"files": "Files",
"shared": "Shared",
"recent": "Recent",
"favorites": "Favorites",
"trash": "Trash"
},
"actions": {
"search": "Search files...",
"new_folder": "New folder",
"upload": "Upload",
"download": "Download",
"delete": "Delete"
},
"share": { ... },
"user_menu": { ... }
}
```
Key lookup: `"nav.files"` resolves to `"Files"`.
## Fallback Behavior
If a key is missing in the requested locale, the system falls back to English (`en`). If still not found, returns an `I18nError::KeyNotFound`.
## Caching
Translations are cached in-memory via `RwLock<HashMap<Locale, serde_json::Value>>`. Loaded lazily on first request per locale.
## Frontend Integration
The frontend uses `static/js/i18n.js` and `static/js/languageSelector.js` to:
1. Detect the user's preferred language
2. Load translations via `/api/i18n/locales/{code}`
3. Apply translations to DOM elements
doc/important-delta-sync-implementation.md
-1152
View File
File diff suppressed because it is too large Load Diff
doc/internal-architecture.md
-474
View File
@@ -1,474 +0,0 @@
# 01 - Internal Architecture
OxiCloud follows a **hexagonal (ports & adapters) architecture** organized in four layers:
```
Domain → Application → Infrastructure → Interfaces
```
All cross-layer dependencies point inward via trait-based ports. The DI container (**AppServiceFactory**) wires concrete implementations at startup.
---
## Storage Model: 100% Blob Storage
OxiCloud uses a **100% blob storage model** where:
- **File metadata** (name, folder, size, user, timestamps, trash status) is stored in **PostgreSQL** (`storage.files` table).
- **File content** is stored as content-addressed blobs via **DedupService** at `.blobs/{prefix}/{hash}.blob`.
- **Folder structure** is purely virtual — represented as rows in `storage.folders` (no filesystem directories per user).
- **Trash** is a soft-delete flag (`is_trashed`, `trashed_at`) on files and folders, exposed via `storage.trash_items` VIEW.
There are no filesystem-based ID mappings, no `folder_ids.json`/`file_ids.json`, and no storage mediator.
---
## Dependency Injection Container
### AppServiceFactory
**File:** `src/common/di.rs`
```rust
pub struct AppServiceFactory {
storage_path: PathBuf,
locales_path: PathBuf,
config: AppConfig,
}
```
Initialization order in `build_app_state()`:
1. **Core services** — path, content cache, thumbnail, chunked upload, transcode, dedup, compression
2. **Repository services** — `FolderDbRepository`, `FileBlobReadRepository`, `FileBlobWriteRepository`, `TrashDbRepository` (all PgPool-backed)
3. **Trash service** (if **enable_trash** enabled)
4. **Application services** — folder, file upload/retrieval/management, search, i18n
5. **Share service** (if **enable_file_sharing** enabled)
6. **DB-dependent services** — favorites, recent, storage usage, auth (via **auth_factory**)
7. **Preload** translations
8. **ZIP service** (needs file retrieval + folder service, wired last)
9. **Assemble AppState** + admin settings + CalDAV/CardDAV
### AppState (Global State)
```rust
pub struct AppState {
pub core: CoreServices,
pub repositories: RepositoryServices,
pub applications: ApplicationServices,
pub db_pool: Option<Arc<PgPool>>,
pub auth_service: Option<AuthServices>,
pub admin_settings_service: Option<Arc<AdminSettingsService>>,
pub trash_service: Option<Arc<dyn TrashUseCase>>,
pub share_service: Option<Arc<dyn ShareUseCase>>,
pub favorites_service: Option<Arc<dyn FavoritesUseCase>>,
pub recent_service: Option<Arc<dyn RecentItemsUseCase>>,
pub storage_usage_service: Option<Arc<dyn StorageUsagePort>>,
pub calendar_service: Option<Arc<dyn StorageUseCase>>,
pub contact_service: Option<Arc<dyn StorageUseCase>>,
pub calendar_use_case: Option<Arc<dyn CalendarUseCase>>,
pub addressbook_use_case: Option<Arc<dyn AddressBookUseCase>>,
pub contact_use_case: Option<Arc<dyn ContactUseCase>>,
}
```
Builder pattern: `new()` → `with_database()` → `with_auth_services()` → `with_trash_service()` → ... → `for_routing()`. The `Default` impl uses stubs from `crate::common::stubs`.
### Service Groups
```rust
pub struct CoreServices {
pub path_service: Arc<PathService>,
pub file_content_cache: Arc<dyn ContentCachePort>,
pub thumbnail_service: Arc<dyn ThumbnailPort>,
pub chunked_upload_service: Arc<dyn ChunkedUploadPort>,
pub image_transcode_service: Arc<dyn ImageTranscodePort>,
pub dedup_service: Arc<dyn DedupPort>,
pub compression_service: Arc<dyn CompressionPort>,
pub zip_service: Arc<dyn ZipPort>,
pub config: AppConfig,
}
pub struct RepositoryServices {
pub folder_repository: Arc<dyn FolderStoragePort>,
pub folder_repo_concrete: Arc<FolderDbRepository>,
pub file_read_repository: Arc<dyn FileReadPort>,
pub file_write_repository: Arc<dyn FileWritePort>,
pub i18n_repository: Arc<dyn I18nService>,
pub trash_repository: Option<Arc<dyn TrashRepository>>,
}
pub struct ApplicationServices {
pub folder_service_concrete: Arc<FolderService>,
pub folder_service: Arc<dyn FolderUseCase>,
pub file_upload_service: Arc<dyn FileUploadUseCase>,
pub file_retrieval_service: Arc<dyn FileRetrievalUseCase>,
pub file_management_service: Arc<dyn FileManagementUseCase>,
pub file_use_case_factory: Arc<dyn FileUseCaseFactory>,
pub i18n_service: Arc<I18nApplicationService>,
pub trash_service: Option<Arc<dyn TrashUseCase>>,
pub search_service: Option<Arc<dyn SearchUseCase>>,
pub share_service: Option<Arc<dyn ShareUseCase>>,
pub favorites_service: Option<Arc<dyn FavoritesUseCase>>,
pub recent_service: Option<Arc<dyn RecentItemsUseCase>>,
}
pub struct AuthServices {
pub token_service: Arc<dyn TokenServicePort>,
pub auth_application_service: Arc<AuthApplicationService>,
}
```
---
## Database Schema (Storage)
All file and folder metadata lives in the `storage` PostgreSQL schema:
```sql
CREATE SCHEMA IF NOT EXISTS storage;
CREATE TABLE storage.folders (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
parent_id UUID REFERENCES storage.folders(id) ON DELETE CASCADE,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id),
is_trashed BOOLEAN NOT NULL DEFAULT FALSE,
trashed_at TIMESTAMPTZ,
original_parent_id UUID,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE storage.files (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
folder_id UUID NOT NULL REFERENCES storage.folders(id) ON DELETE CASCADE,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id),
blob_hash TEXT NOT NULL,
size BIGINT NOT NULL DEFAULT 0,
mime_type TEXT NOT NULL DEFAULT 'application/octet-stream',
is_trashed BOOLEAN NOT NULL DEFAULT FALSE,
trashed_at TIMESTAMPTZ,
original_folder_id UUID,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE OR REPLACE VIEW storage.trash_items AS
SELECT id, name, 'file' AS item_type, folder_id AS parent_id,
user_id, size, mime_type, trashed_at, created_at
FROM storage.files WHERE is_trashed = TRUE
UNION ALL
SELECT id, name, 'folder' AS item_type, parent_id,
user_id, 0 AS size, NULL AS mime_type, trashed_at, created_at
FROM storage.folders WHERE is_trashed = TRUE;
```
---
## Repository Layer (Infrastructure)
All repositories use **PgPool** for metadata and **DedupService** for blob content.
### FolderDbRepository
**File:** `src/infrastructure/repositories/pg/folder_db_repository.rs`
```rust
pub struct FolderDbRepository {
pool: Option<Arc<PgPool>>,
}
```
Implements `FolderRepository`. Uses recursive CTEs for path building, unique constraints for name dedup within parent, and soft-delete flags for trash operations.
Key methods: `create_folder`, `get_folder`, `get_folder_by_path`, `list_folders`, `rename_folder`, `move_folder`, `delete_folder`, `move_to_trash`, `restore_from_trash`, `create_home_folder`, `get_folder_user_id`.
`new_stub()` creates a pool-less instance for `AppState::default()`.
### FileBlobReadRepository
**File:** `src/infrastructure/repositories/pg/file_blob_read_repository.rs`
```rust
pub struct FileBlobReadRepository {
pool: Arc<PgPool>,
dedup: Arc<dyn DedupPort>,
folder_repo: Arc<FolderDbRepository>,
}
```
Implements `FileReadPort`. Reads metadata from `storage.files` and content from blob store via `dedup.read_blob()` / `read_blob_bytes()`.
Key methods: `get_file`, `list_files`, `get_file_content`, `get_file_stream`, `get_file_range_stream`, `get_file_mmap`, `get_file_path`, `get_parent_folder_id`.
### FileBlobWriteRepository
**File:** `src/infrastructure/repositories/pg/file_blob_write_repository.rs`
```rust
pub struct FileBlobWriteRepository {
pool: Arc<PgPool>,
dedup: Arc<dyn DedupPort>,
folder_repo: Arc<FolderDbRepository>,
}
```
Implements `FileWritePort`. Stores content via `dedup.store_bytes()` (returns hash), then INSERTs metadata into `storage.files`.
Key methods: `save_file`, `save_file_from_stream`, `move_file`, `rename_file`, `delete_file`, `update_file_content`, `move_to_trash`, `restore_from_trash`, `delete_file_permanently`.
### TrashDbRepository
**File:** `src/infrastructure/repositories/pg/trash_db_repository.rs`
```rust
pub struct TrashDbRepository {
pool: Arc<PgPool>,
retention_days: u32,
}
```
Implements `TrashRepository`. Reads from `storage.trash_items` VIEW. `clear_trash` DELETEs rows where `is_trashed = TRUE`. `get_expired_items` checks `trashed_at` against the configured retention period.
---
## Path Service
**File:** `src/infrastructure/services/path_service.rs`
```rust
pub struct PathService {
root_path: PathBuf, // e.g., ./storage
}
```
Used for resolving storage root paths (blob storage directory, thumbnail paths, etc.). Not used for per-user folder resolution — that is handled by `FolderDbRepository` via PostgreSQL.
### StoragePath (Domain Value Object)
**File:** `src/domain/services/path_service.rs`
```rust
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct StoragePath {
segments: Vec<String>,
}
```
| Method | Description |
|---|---|
| `root()` | Empty path (storage root) |
| `from_string(path)` | Parse from `/`-delimited string |
| `join(segment)` | Append a segment |
| `file_name()` | Last segment |
| `parent()` | All segments except last |
| `to_string()` | Join segments with `/` |
### Trait Implementations
- **StoragePort** — `resolve_path()`, `ensure_directory()`, `file_exists()`, `directory_exists()`
---
## Session Management
### Session Entity
**File:** `src/domain/entities/session.rs`
```rust
pub struct Session {
id: String, // UUID v4
user_id: String,
refresh_token: String,
expires_at: DateTime<Utc>,
ip_address: Option<String>,
user_agent: Option<String>,
created_at: DateTime<Utc>,
revoked: bool,
}
```
Constructors:
- `Session::new(user_id, refresh_token, ip_address, user_agent, expires_in_days)` — generates UUID, panics if **user_id** or **refresh_token** empty
- `Session::from_raw(...)` — for DB reconstruction
### SessionRepository (Domain Port)
**File:** `src/domain/repositories/session_repository.rs`
```rust
#[async_trait]
pub trait SessionRepository: Send + Sync + 'static {
async fn create_session(&self, session: Session) -> SessionRepositoryResult<Session>;
async fn get_session_by_id(&self, id: &str) -> SessionRepositoryResult<Session>;
async fn get_session_by_refresh_token(&self, token: &str) -> SessionRepositoryResult<Session>;
async fn get_sessions_by_user_id(&self, user_id: &str) -> SessionRepositoryResult<Vec<Session>>;
async fn revoke_session(&self, session_id: &str) -> SessionRepositoryResult<()>;
async fn revoke_all_user_sessions(&self, user_id: &str) -> SessionRepositoryResult<u64>;
async fn delete_expired_sessions(&self) -> SessionRepositoryResult<u64>;
}
```
### SessionStoragePort (Application Port)
**File:** `src/application/ports/auth_ports.rs`
```rust
#[async_trait]
pub trait SessionStoragePort: Send + Sync + 'static {
async fn create_session(&self, session: Session) -> Result<Session, DomainError>;
async fn get_session_by_refresh_token(&self, token: &str) -> Result<Session, DomainError>;
async fn revoke_session(&self, session_id: &str) -> Result<(), DomainError>;
async fn revoke_all_user_sessions(&self, user_id: &str) -> Result<u64, DomainError>;
}
```
### SessionPgRepository (Infrastructure)
**File:** `src/infrastructure/repositories/pg/session_pg_repository.rs`
```rust
pub struct SessionPgRepository {
pool: Arc<PgPool>,
}
```
Implements both **SessionRepository** and **SessionStoragePort**. Uses `with_transaction()` helper for write operations. `create_session` also updates `auth.users.last_login_at` within the same transaction.
### Database Schema
```sql
CREATE TABLE IF NOT EXISTS auth.sessions (
id VARCHAR(36) PRIMARY KEY,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
refresh_token TEXT NOT NULL UNIQUE,
expires_at TIMESTAMP WITH TIME ZONE NOT NULL,
ip_address TEXT,
user_agent TEXT,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
revoked BOOLEAN NOT NULL DEFAULT FALSE
);
CREATE INDEX idx_sessions_user_id ON auth.sessions(user_id);
CREATE INDEX idx_sessions_refresh_token ON auth.sessions(refresh_token);
CREATE INDEX idx_sessions_expires_at ON auth.sessions(expires_at);
CREATE INDEX idx_sessions_active ON auth.sessions(user_id, revoked)
WHERE NOT revoked AND is_session_active(expires_at);
```
### Auth Service
**File:** `src/application/services/auth_application_service.rs`
**AuthApplicationService** orchestrates authentication using:
- **UserStoragePort** — user CRUD
- **SessionStoragePort** — session lifecycle
- **PasswordHasherPort** — Argon2id hashing
- **TokenServicePort** — JWT generation/validation
- `RwLock<OidcState>` — hot-reloadable OIDC configuration
- `Mutex<HashMap<String, PendingOidcFlow>>` — in-flight OIDC login states
Wired by `auth_factory.rs`: **UserPgRepository** + **SessionPgRepository** + **Argon2PasswordHasher** + **JwtTokenService** → **AuthApplicationService**.
---
## File Use Case Factory
**File:** `src/application/services/file_use_case_factory.rs`
```rust
pub trait FileUseCaseFactory: Send + Sync + 'static {
fn create_file_upload_use_case(&self) -> Arc<dyn FileUploadUseCase>;
fn create_file_retrieval_use_case(&self) -> Arc<dyn FileRetrievalUseCase>;
fn create_file_management_use_case(&self) -> Arc<dyn FileManagementUseCase>;
}
```
**AppFileUseCaseFactory** creates lightweight service instances with only **FileReadPort** / **FileWritePort**.
### File Operation Port Hierarchy
| Port | Key Methods |
|---|---|
| **FileUploadUseCase** | `upload_file()`, `smart_upload()` (returns **UploadStrategy**: `Buffered` <1MB, `Streaming` ≥1MB), `create_file()`, `update_file()` |
| **FileRetrievalUseCase** | `get_file()`, `get_file_content()`, `get_file_stream()`, `get_file_optimized()` (content-cache → WebP transcode → mmap → streaming), `get_file_range_stream()` |
| **FileManagementUseCase** | `move_file()`, `rename_file()`, `delete_file()`, `delete_with_cleanup()` (trash-first with dedup reference cleanup) |
---
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────┐
│ Interfaces Layer │
│ Axum Router → API Routes + Middleware (Auth, Compress) │
└─────────────────────┬───────────────────────────────────────┘
│ Arc<AppState>
┌─────────────────────▼───────────────────────────────────────┐
│ Application Layer │
│ ┌──────────────┐ ┌──────────────┐ ┌─────────────────────┐ │
│ │ FileUpload │ │ FolderService│ │ AuthApplication │ │
│ │ FileRetrieval│ │ SearchService│ │ AdminSettings │ │
│ │ FileMgmt │ │ I18nService │ │ TrashService │ │
│ └──────┬───────┘ └──────┬───────┘ └──────────┬──────────┘ │
│ │ Ports (traits) │ │ │
└─────────┼────────────────┼─────────────────────┼────────────┘
│ │ │
┌─────────▼────────────────▼─────────────────────▼────────────┐
│ Infrastructure Layer │
│ ┌────────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ FileBlobRead │ │ PathService │ │ SessionPg │ │
│ │ FileBlobWrite │ │ DedupService │ │ UserPg │ │
│ │ FolderDb │ │ Thumbnail │ │ JwtTokenService │ │
│ │ TrashDb │ │ Transcode │ │ Argon2Hasher │ │
│ └────────────────┘ └──────────────┘ └──────────────────┘ │
│ ┌────────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ ContentCache │ │ Compression │ │ ChunkedUpload │ │
│ │ BufferPool │ │ ZipService │ │ ShareFsRepo │ │
│ └────────────────┘ └──────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────────┐
│ Domain Layer │
│ Entities: File, Folder, Session, User, Calendar, Contact │
│ Value Objects: StoragePath │
│ Repository Traits: FolderRepository, TrashRepository, ... │
│ Domain Errors │
└─────────────────────────────────────────────────────────────┘
```
### Data Flow: File Upload
```
HTTP Request (multipart)
→ FileUploadService.smart_upload()
→ FileBlobWriteRepository.save_file() / save_file_from_stream()
→ DedupService.store_bytes() → .blobs/{prefix}/{hash}.blob
→ INSERT INTO storage.files (name, folder_id, blob_hash, size, ...)
→ 201 Created (FileDto)
```
### Data Flow: File Download
```
HTTP Request (GET /api/files/{id}/download)
→ FileRetrievalService.get_file_optimized()
→ ContentCache hit? → serve from RAM
→ FileBlobReadRepository.get_file_content() / get_file_stream()
→ SELECT blob_hash FROM storage.files WHERE id = $1
→ DedupService.read_blob(hash) → bytes from .blobs/
→ Optional WebP transcode → response
```
### Data Flow: Folder Operations
```
HTTP Request (POST /api/folders)
→ FolderService.create_folder()
→ FolderDbRepository.create_folder()
→ INSERT INTO storage.folders (name, parent_id, user_id, ...)
→ 201 Created (FolderDto)
```
doc/lto-optimizations.md
-82
View File
@@ -1,82 +0,0 @@
# 05 - LTO Optimizations
OxiCloud uses Link Time Optimization (LTO) to improve runtime performance. LTO allows the compiler to optimize across module boundaries during linking -- better inlining, dead code elimination, and more efficient binaries.
---
## Implemented Optimizations
### Release Profile
```toml
[profile.release]
lto = "fat" # Full cross-module optimization
codegen-units = 1 # Maximum optimization but slower compile time
opt-level = 3 # Maximum optimization level
panic = "abort" # Smaller binary size by removing panic unwinding
strip = true # Removes debug symbols for smaller binary
```
### Development Profile
```toml
[profile.dev]
opt-level = 1 # Light optimization for faster build time
debug = true # Keep debug information for development
```
### Benchmark Profile
```toml
[profile.bench]
lto = "fat" # Full optimization for benchmarks
codegen-units = 1 # Maximum optimization
opt-level = 3 # Maximum optimization level
```
---
## Performance Effects
1. **Smaller binary size** -- unused code and metadata removed
2. **Faster execution** -- better inlining and code optimizations
3. **Reduced memory usage** -- more efficient code layout
---
## LTO Options
- **fat**: Full LTO across all crate boundaries. Maximum optimization, longest compile time.
- **thin**: Faster LTO that trades some optimization for compile speed. Good for development.
- **off**: No cross-module optimization.
---
## Build Time Impact
LTO increases compilation time. The tradeoff:
- Development builds: minimal LTO (`opt-level = 1`) for faster iteration
- Release builds: full LTO for maximum runtime performance
- Benchmark builds: full LTO to measure actual optimized performance
---
## Measuring Impact
```bash
# Run benchmarks with all optimizations
cargo bench
# Compare with non-optimized build (remove for comparison only)
RUSTFLAGS="-C lto=off" cargo bench
```
---
## When to Adjust
Consider changing these settings if:
1. You need faster compile times during development
2. You're experiencing unexpected runtime behavior
3. You want to experiment with optimization vs. binary size tradeoffs
The defaults work well for most cases.
doc/oidc-architecture.md
-149
View File
@@ -1,149 +0,0 @@
# 29 - OIDC Architecture
OpenID Connect (OIDC) authentication follows the Authorization Code Flow. The system supports multiple identity providers (Authentik, Authelia, KeyCloak) through a single configurable integration point.
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ IDENTITY PROVIDER │
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ │ │ │ │ │ │
│ │ Authentik │ │ Authelia │ │ KeyCloak │ │
│ │ │ │ │ │ │ │
│ └───────┬───────┘ └───────┬───────┘ └───────┬───────┘ │
│ │ │ │ │
└────────────┼──────────────────────┼──────────────────────┼─────────────┘
│ │ │
│ │ │
│ │ │
│ OIDC │
│ │ │
│ │ │
┌────────────┼──────────────────────┼──────────────────────┼─────────────┐
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌───────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ OXICLOUD │ │
│ │ │ │
│ │ ┌───────────────┐ ┌───────────────┐ │ │
│ │ │ │ │ │ │ │
│ │ │ OidcService │◄────►│ AuthService │ │ │
│ │ │ │ │ │ │ │
│ │ └───────┬───────┘ └───────┬───────┘ │ │
│ │ │ │ │ │
│ │ ▼ ▼ │ │
│ │ ┌───────────────────────────────────────────┐ │ │
│ │ │ │ │ │
│ │ │ AuthApplicationService │ │ │
│ │ │ │ │ │
│ │ └───────────────────┬───────────────────────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌───────────────────────────────────────────┐ │ │
│ │ │ │ │ │
│ │ │ Auth Handler │ │ │
│ │ │ │ │ │
│ │ └───────────────────────────────────────────┘ │ │
│ │ │ │
│ └───────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
▲
│
│ HTTP/HTTPS
│
│
┌────────────────────────────────────────────────────────────────────────┐
│ │
│ WEB BROWSER │
│ │
│ ┌───────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ User Interface │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ │ │ │ │ │
│ │ │ Login.html │ │ auth.js │ │ │
│ │ │ │ │ │ │ │
│ │ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ └───────────────────────────────────────────────────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────┘
```
## OIDC Authentication Flow
The flow follows the standard Authorization Code Flow:
1. **Authentication Start** -- the user clicks "Login with [Provider]" on the login page. The frontend generates a random state for CSRF protection and requests an authorization URL from the backend.
2. **Redirect to Identity Provider** -- the backend generates and returns the authorization URL. The browser redirects the user to the provider's login page.
3. **Authentication at the Provider** -- the user authenticates (password, 2FA, etc.). The provider redirects back with an authorization code.
4. **Authorization Code Exchange** -- the frontend sends the authorization code to the backend. The backend exchanges it for access and ID tokens with the provider, then verifies the ID token and extracts user info.
5. **User Creation/Retrieval** -- the backend looks up an existing user by the provider's external ID. If none exists and auto-provisioning is enabled, a new user is created. If disabled, an error is returned.
6. **Session Token Generation** -- the backend generates its own access and refresh tokens for the user. These tokens authenticate subsequent API requests.
7. **Response to Client** -- tokens and user info are returned to the frontend. The frontend stores them and redirects to the main page.
## Main Components
### OidcService
Handles communication with OIDC providers:
- Discovers provider OIDC endpoints
- Generates authorization URLs
- Exchanges authorization codes for tokens
- Verifies tokens and extracts user info
### AuthApplicationService
Coordinates the authentication process:
- Acts as interface between the API layer and domain services
- Manages user creation/retrieval
- Coordinates access token generation
### Auth Handler
Exposes HTTP endpoints for the OIDC auth flow:
- `GET /api/auth/oidc/providers` -- lists available OIDC providers
- `GET /api/auth/oidc/authorize` -- generates an authorization URL for the OIDC provider
- `GET /api/auth/oidc/callback` -- receives the redirect from the provider with the authorization code
- `POST /api/auth/oidc/exchange` -- exchanges the authorization code for session tokens
### Frontend (login.html + auth.js)
Handles the client-side of the auth flow:
- Shows SSO button for the configured OIDC provider in `login.html`
- Initiates the authentication flow via `auth.js`
- Handles the return redirect from the provider
- Processes and stores session tokens
## Provider Configuration
One OIDC provider is configured per instance via environment variables prefixed with **OXICLOUD_OIDC_***:
1. **Single provider** per instance.
2. **Environment variables**: **OXICLOUD_OIDC_ENABLED**, **OXICLOUD_OIDC_ISSUER_URL**, **OXICLOUD_OIDC_CLIENT_ID**, **OXICLOUD_OIDC_CLIENT_SECRET**, etc.
3. **Auto-provisioning**: users can be created automatically on first OIDC login (**OXICLOUD_OIDC_AUTO_PROVISION**).
4. **Role mapping**: admin groups are configured via **OXICLOUD_OIDC_ADMIN_GROUPS**.
See `oidc-config-examples.md` for provider-specific configuration examples.
## Security
The OIDC implementation includes several security measures:
1. **CSRF protection** -- random state parameter prevents CSRF attacks.
2. **Token validation** -- JWT signatures and expiration are verified.
3. **Authorization Code Flow** -- more secure than the implicit flow.
4. **HTTPS** -- required for all OIDC communications.
5. **Client secrets** -- stored securely, never exposed to the frontend.
doc/oidc-config-examples.md
-215
View File
@@ -1,215 +0,0 @@
# 31 - OIDC Config Examples
Configuration examples for integrating with different OIDC (OpenID Connect) providers. One OIDC provider per instance.
## Table of Contents
1. [General OIDC Configuration](#general-oidc-configuration)
2. [Authentik](#authentik)
3. [Authelia](#authelia)
4. [KeyCloak](#keycloak)
5. [Troubleshooting](#troubleshooting)
## General OIDC Configuration
To enable OIDC, set these environment variables:
```bash
# Enable OIDC
OXICLOUD_OIDC_ENABLED=true
# OIDC provider configuration
OXICLOUD_OIDC_PROVIDER_NAME="Display Name"
OXICLOUD_OIDC_ISSUER_URL="https://provider.example.com/realms/your-realm"
OXICLOUD_OIDC_CLIENT_ID="your-client-id"
OXICLOUD_OIDC_CLIENT_SECRET="your-client-secret"
OXICLOUD_OIDC_REDIRECT_URI="https://your-oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES="openid profile email"
OXICLOUD_OIDC_FRONTEND_URL="https://your-oxicloud.example.com"
OXICLOUD_OIDC_AUTO_PROVISION="true"
OXICLOUD_OIDC_ADMIN_GROUPS="admin-group"
OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN="false"
```
## Authentik
[Authentik](https://goauthentik.io/) is an open-source identity platform providing authentication, authorization, and user management.
### 1. Create an Application in Authentik
1. Log into the Authentik admin panel
2. Go to "Applications" -> "Create"
3. Enter a name for the application (e.g. "OxiCloud")
4. Select "OAuth2/OpenID Provider" as the provider type
5. In the OAuth2 configuration:
- **Redirect URI/Callback URL**: `https://your-oxicloud.example.com/api/auth/oidc/callback`
- **Client Type**: Confidential
- **Client ID**: auto-generated (note it down)
- **Client Secret**: auto-generated (note it down)
- **Scopes**: openid, email, profile
6. In the UI configuration:
- **Launch URL**: `https://your-oxicloud.example.com/`
- **Icon**: optional
### 2. Configure for Authentik
```yaml
# docker-compose.yml
version: '3'
services:
oxicloud:
image: diocrafts/oxicloud:latest
environment:
OXICLOUD_OIDC_ENABLED: "true"
OXICLOUD_OIDC_PROVIDER_NAME: "Authentik"
OXICLOUD_OIDC_ISSUER_URL: "https://authentik.example.com/application/o/oxicloud"
OXICLOUD_OIDC_CLIENT_ID: "your-authentik-client-id"
OXICLOUD_OIDC_CLIENT_SECRET: "your-authentik-client-secret"
OXICLOUD_OIDC_REDIRECT_URI: "https://oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES: "openid profile email"
OXICLOUD_OIDC_FRONTEND_URL: "https://oxicloud.example.com"
OXICLOUD_OIDC_AUTO_PROVISION: "true"
OXICLOUD_OIDC_ADMIN_GROUPS: ""
OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN: "false"
ports:
- "8086:8086"
volumes:
- ./storage:/app/storage
```
## Authelia
[Authelia](https://www.authelia.com/) is an open-source multi-factor authentication solution.
### 1. Configure Authelia
Edit your Authelia configuration (`configuration.yml`):
```yaml
identity_providers:
oidc:
hmac_secret: your-secure-secret # Change to a secure random value
issuer_private_key: /config/private.pem # Path to your private key
cors:
endpoints: ['authorization', 'token', 'revocation', 'introspection']
allowed_origins:
- https://oxicloud.example.com
clients:
- id: oxicloud
description: OxiCloud
secret: your-secure-client-secret # Change this
public: false
authorization_policy: two_factor
redirect_uris:
- https://oxicloud.example.com/api/auth/oidc/callback
scopes: ['openid', 'profile', 'email', 'groups']
userinfo_signing_algorithm: none
```
### 2. Configure for Authelia
```yaml
# docker-compose.yml
version: '3'
services:
oxicloud:
image: diocrafts/oxicloud:latest
environment:
OXICLOUD_OIDC_ENABLED: "true"
OXICLOUD_OIDC_PROVIDER_NAME: "Authelia"
OXICLOUD_OIDC_ISSUER_URL: "https://authelia.example.com"
OXICLOUD_OIDC_CLIENT_ID: "oxicloud"
OXICLOUD_OIDC_CLIENT_SECRET: "your-secure-client-secret"
OXICLOUD_OIDC_REDIRECT_URI: "https://oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES: "openid profile email groups"
OXICLOUD_OIDC_FRONTEND_URL: "https://oxicloud.example.com"
OXICLOUD_OIDC_AUTO_PROVISION: "true"
OXICLOUD_OIDC_ADMIN_GROUPS: ""
OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN: "false"
ports:
- "8086:8086"
volumes:
- ./storage:/app/storage
```
## KeyCloak
[KeyCloak](https://www.keycloak.org/) is an open-source identity and access management solution.
### 1. Create a Client in KeyCloak
1. Log into the KeyCloak admin console
2. Select your Realm
3. Go to "Clients" -> "Create"
4. Fill in the form:
- **Client ID**: `oxicloud`
- **Client Protocol**: `openid-connect`
- **Root URL**: `https://oxicloud.example.com`
5. In the client configuration:
- **Access Type**: `confidential`
- **Valid Redirect URIs**: `https://oxicloud.example.com/api/auth/oidc/callback`
- **Web Origins**: `https://oxicloud.example.com` (or `+` to allow all origins)
6. Save the configuration
7. Go to the "Credentials" tab and copy the generated "Secret"
### 2. Configure for KeyCloak
```yaml
# docker-compose.yml
version: '3'
services:
oxicloud:
image: diocrafts/oxicloud:latest
environment:
OXICLOUD_OIDC_ENABLED: "true"
OXICLOUD_OIDC_PROVIDER_NAME: "KeyCloak"
OXICLOUD_OIDC_ISSUER_URL: "https://keycloak.example.com/realms/your-realm"
OXICLOUD_OIDC_CLIENT_ID: "oxicloud"
OXICLOUD_OIDC_CLIENT_SECRET: "your-keycloak-client-secret"
OXICLOUD_OIDC_REDIRECT_URI: "https://oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES: "openid profile email"
OXICLOUD_OIDC_FRONTEND_URL: "https://oxicloud.example.com"
OXICLOUD_OIDC_AUTO_PROVISION: "true"
OXICLOUD_OIDC_ADMIN_GROUPS: "oxicloud-admins"
OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN: "false"
ports:
- "8086:8086"
volumes:
- ./storage:/app/storage
```
## Troubleshooting
### Error: "Failed to discover OIDC provider"
The backend cannot reach the provider's discovery endpoint.
**Fixes:**
1. Verify the discovery URL is correct
2. Check that the backend can reach the URL (firewalls, DNS, etc.)
3. If using a self-signed certificate, configure the appropriate trust
### Error: "Invalid redirect URI"
The OIDC provider is rejecting the redirect URI.
**Fixes:**
1. Make sure the redirect URI configured in the backend matches exactly what is registered in the provider
2. Check for protocol differences (http vs https), port, or path mismatches
### Error: "User does not exist and auto-creation is disabled"
**Fixes:**
1. Enable auto-provisioning: `OXICLOUD_OIDC_AUTO_PROVISION="true"`
2. Or manually create the user before attempting OIDC login
### Error: "Could not extract user ID from claim"
The backend cannot find the user ID attribute in the token claims.
**Fixes:**
1. Verify the provider returns the `sub` claim in tokens
2. Make sure scopes in **OXICLOUD_OIDC_SCOPES** include `openid`
3. Configure the provider to include the required claims in tokens
See `oidc-architecture.md` and `oidc-integration.md` for deeper technical details.
doc/oidc-integration.md
-266
View File
@@ -1,266 +0,0 @@
# 30 - OIDC Integration
OpenID Connect (OIDC) is an identity layer on top of OAuth 2.0. It lets clients verify user identity based on authentication performed by an authorization server and obtain basic profile information. Adding OIDC enables SSO with providers like Authentik, Authelia, and KeyCloak.
What it gives us:
1. Users authenticate with their existing IdP credentials
2. No need for separate username/password management
3. Modern auth best practices baked in
4. Seamless experience for users already on SSO
## OIDC Configuration
OIDC is configured separately from **AuthConfig** via **OidcConfig** in `src/common/config.rs`. This is a single-provider model -- one OIDC provider per instance:
```rust
/// OpenID Connect (OIDC) configuration
pub struct OidcConfig {
pub enabled: bool, // Whether OIDC is enabled
pub issuer_url: String, // OIDC Issuer URL
pub client_id: String, // OIDC Client ID
pub client_secret: String, // OIDC Client Secret
pub redirect_uri: String, // Redirect URI (default: http://localhost:8086/api/auth/oidc/callback)
pub scopes: String, // Scopes to request (default: "openid profile email")
pub frontend_url: String, // Frontend URL for post-login redirect
pub auto_provision: bool, // Auto-create users on first login (JIT provisioning)
pub admin_groups: String, // Comma-separated OIDC groups that map to admin role
pub disable_password_login: bool, // Disable password-based login entirely
pub provider_name: String, // Display name (default: "SSO")
}
```
Environment variables use the **OXICLOUD_OIDC_*** prefix:
```bash
OXICLOUD_OIDC_ENABLED=true
OXICLOUD_OIDC_ISSUER_URL="https://authentik.example.com/application/o/oxicloud/"
OXICLOUD_OIDC_CLIENT_ID="your-client-id"
OXICLOUD_OIDC_CLIENT_SECRET="your-client-secret"
OXICLOUD_OIDC_REDIRECT_URI="https://oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES="openid profile email"
OXICLOUD_OIDC_FRONTEND_URL="https://oxicloud.example.com"
OXICLOUD_OIDC_AUTO_PROVISION=true
OXICLOUD_OIDC_ADMIN_GROUPS="oxicloud-admins"
OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN=false
OXICLOUD_OIDC_PROVIDER_NAME="Authentik"
```
## OIDC Service Implementation
The OIDC service lives in the infrastructure layer at `src/infrastructure/services/oidc_service.rs` and implements the **OidcServicePort** trait defined in `src/application/ports/auth_ports.rs`:
```rust
// src/application/ports/auth_ports.rs — Port trait
#[async_trait]
pub trait OidcServicePort: Send + Sync + 'static {
fn enabled(&self) -> bool;
fn provider_name(&self) -> &str;
fn generate_auth_url(&self, state: &str) -> Result<String, DomainError>;
async fn exchange_code(&self, code: &str) -> Result<OidcTokenSet, DomainError>;
async fn get_user_info(&self, token_set: &OidcTokenSet) -> Result<OidcIdClaims, DomainError>;
}
// src/infrastructure/services/oidc_service.rs — Implementation
pub struct OidcService {
config: OidcConfig,
http_client: reqwest::Client,
// Discovery metadata cached after initialization
}
impl OidcService {
pub async fn new(config: OidcConfig) -> Result<Self, DomainError> {
// Discovers OIDC endpoints from issuer_url
// ...
}
}
```
Follows hexagonal architecture: the port (**OidcServicePort**) is in the application layer, and the implementation (**OidcService**) is in the infrastructure layer.
## User Entity OIDC Support
The **User** entity in `src/domain/entities/user.rs` supports OIDC users via two fields:
```rust
#[derive(Debug, Clone)]
pub struct User {
// ... standard fields ...
oidc_provider: Option<String>, // OIDC provider name (e.g., "authentik")
oidc_subject: Option<String>, // OIDC subject identifier (unique ID from provider)
}
impl User {
pub fn oidc_provider(&self) -> Option<&str> {
self.oidc_provider.as_deref()
}
pub fn oidc_subject(&self) -> Option<&str> {
self.oidc_subject.as_deref()
}
// Constructor for OIDC users
pub fn new_oidc(username, email, role, quota, oidc_provider, oidc_subject) -> Self;
}
```
## Database Schema
The **auth.users** table includes OIDC columns:
```sql
ALTER TABLE auth.users ADD COLUMN IF NOT EXISTS oidc_provider VARCHAR(255);
ALTER TABLE auth.users ADD COLUMN IF NOT EXISTS oidc_subject VARCHAR(255);
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_oidc ON auth.users(oidc_provider, oidc_subject) WHERE oidc_provider IS NOT NULL;
```
Users are matched by **oidc_provider** + **oidc_subject** combination. The **UserRepository** trait includes `get_user_by_oidc_subject()` for lookups.
## Auth Application Service
**AuthApplicationService** in `src/application/services/auth_application_service.rs` coordinates OIDC authentication:
```rust
impl AuthApplicationService {
// Initialize with OIDC support
pub fn with_oidc(self, oidc_service: Arc<dyn OidcServicePort>, oidc_config: OidcConfig) -> Self;
// Reload OIDC configuration (for admin settings changes)
pub async fn reload_oidc(&self, config: OidcConfig) -> Result<(), DomainError>;
// Disable OIDC
pub fn disable_oidc(&self);
// Check if OIDC is enabled
pub fn oidc_enabled(&self) -> bool;
// Get OIDC config
pub fn oidc_config(&self) -> Option<OidcConfig>;
// Get OIDC service reference
pub fn oidc_service(&self) -> Option<Arc<dyn OidcServicePort>>;
// Check if password login is disabled
pub fn password_login_disabled(&self) -> bool;
// Prepare OIDC authorization URL
pub fn prepare_oidc_authorize(&self) -> Result<OidcAuthorizeResponseDto, DomainError>;
// Handle OIDC callback (exchange code for tokens)
pub async fn oidc_callback(&self, code: &str, state: &str) -> Result<AuthResponseDto, DomainError>;
}
```
## Auth Handler Routes
OIDC endpoints in `src/interfaces/api/handlers/auth_handler.rs`:
```rust
// Public OIDC routes (no auth required) — nested under /api/auth/
.route("/status", get(get_system_status))
.route("/oidc/providers", get(oidc_providers))
.route("/oidc/authorize", get(oidc_authorize))
.route("/oidc/callback", get(oidc_callback))
.route("/oidc/exchange", post(oidc_exchange))
```
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/auth/oidc/providers` | Returns OIDC provider info (name, enabled state) |
| GET | `/api/auth/oidc/authorize` | Returns authorization URL for redirect to IdP |
| GET | `/api/auth/oidc/callback` | Receives callback redirect from IdP with auth code |
| POST | `/api/auth/oidc/exchange` | Exchanges auth code for JWT tokens |
## DTOs
DTOs in `src/application/dtos/user_dto.rs`:
```rust
// Response with authorization URL
#[derive(Debug, Clone, Serialize)]
pub struct OidcAuthorizeResponseDto {
pub authorize_url: String,
pub state: String,
}
// Query params received from IdP callback
#[derive(Debug, Clone, Deserialize)]
pub struct OidcCallbackQueryDto {
pub code: String,
pub state: String,
}
// Request to exchange code for tokens
#[derive(Debug, Clone, Deserialize)]
pub struct OidcExchangeDto {
pub code: String,
pub state: String,
}
// Provider info response
#[derive(Debug, Clone, Serialize)]
pub struct OidcProviderInfoDto {
pub enabled: bool,
pub provider_name: String,
pub disable_password_login: bool,
}
// User info from OIDC claims
#[derive(Debug, Clone, Serialize)]
pub struct OidcUserInfoDto {
pub subject: String,
pub email: Option<String>,
pub name: Option<String>,
pub preferred_username: Option<String>,
pub groups: Vec<String>,
}
```
## Frontend Integration
OIDC login is built directly into `static/login.html` and handled by `static/js/auth.js`. There is no separate `oidcAuth.js` file. The login page checks the system status endpoint to see if OIDC is enabled, then shows an SSO button accordingly.
```html
<!-- In login.html - SSO login button shown when OIDC is enabled -->
<button id="oidc-login-btn" class="btn btn-oidc" style="display:none">
Login with SSO
</button>
```
## Configuration Example
KeyCloak setup via docker-compose:
```yaml
# docker-compose.yml
version: '3'
services:
oxicloud:
image: diocrafts/oxicloud:latest
environment:
OXICLOUD_OIDC_ENABLED: "true"
OXICLOUD_OIDC_ISSUER_URL: "https://keycloak.example.com/realms/your-realm"
OXICLOUD_OIDC_CLIENT_ID: "oxicloud"
OXICLOUD_OIDC_CLIENT_SECRET: "your-client-secret"
OXICLOUD_OIDC_REDIRECT_URI: "https://oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES: "openid profile email"
OXICLOUD_OIDC_FRONTEND_URL: "https://oxicloud.example.com"
OXICLOUD_OIDC_AUTO_PROVISION: "true"
OXICLOUD_OIDC_ADMIN_GROUPS: "oxicloud-admins"
OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN: "false"
OXICLOUD_OIDC_PROVIDER_NAME: "KeyCloak"
ports:
- "8086:8086"
volumes:
- ./storage:/app/storage
```
See `oidc-config-examples.md` for more provider-specific configurations.
## Additional Notes
1. **Security** -- always use HTTPS for OIDC connections. Ensure proper TLS configuration.
2. **User mapping** -- OIDC users are identified by **oidc_provider** + **oidc_subject** in the **auth.users** table. Groups from OIDC can map to admin role via **OXICLOUD_OIDC_ADMIN_GROUPS**.
3. **Single provider** -- one OIDC provider per instance. Managed via admin settings UI or environment variables.
4. **Session management** -- after OIDC authentication, the backend generates its own JWT access/refresh tokens. Sessions work identically to password-based login from that point.
5. **Access control** -- OIDC users share the same permissions model as local users. Admin role can be auto-assigned based on OIDC group membership.
6. **Testing** -- use the admin settings UI (`/admin.html`) to configure and test OIDC connections.
doc/readme-auth.md
-197
View File
@@ -1,197 +0,0 @@
# 32 - Authentication
The auth system uses JWT (JSON Web Tokens) with Argon2id password hashing. Features include registration, login, role-based access control (Admin/User), token refresh, storage quotas, and file/folder ownership.
## API Endpoints
All auth endpoints live under `/api/auth`:
- **POST /api/auth/register** -- register a new user
- **POST /api/auth/login** -- login and get tokens
- **POST /api/auth/refresh** -- refresh access token
- **GET /api/auth/me** -- get current user info
- **PUT /api/auth/change-password** -- change user password
- **POST /api/auth/logout** -- logout and invalidate refresh token
- **GET /api/auth/status** -- system status (auth enabled, OIDC enabled, etc.)
- **GET /api/auth/oidc/providers** -- list available OIDC providers
- **GET /api/auth/oidc/authorize** -- generate OIDC authorization URL
- **GET /api/auth/oidc/callback** -- receive OIDC callback redirect
- **POST /api/auth/oidc/exchange** -- exchange authorization code for tokens
## Request/Response Examples
### Register
**Request:**
```json
POST /api/auth/register
{
"username": "testuser",
"email": "test@example.com",
"password": "SecurePassword123"
}
```
**Response:**
```json
201 Created
{
"userId": "d290f1ee-6c54-4b01-90e6-d701748f0851",
"username": "testuser",
"email": "test@example.com"
}
```
### Login
**Request:**
```json
POST /api/auth/login
{
"username": "testuser",
"password": "SecurePassword123"
}
```
**Response:**
```json
200 OK
{
"accessToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"refreshToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"expiresIn": 3600
}
```
### Refresh Token
**Request:**
```json
POST /api/auth/refresh
{
"refreshToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
}
```
**Response:**
```json
200 OK
{
"accessToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"refreshToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"expiresIn": 3600
}
```
### Get Current User
**Request:**
```
GET /api/auth/me
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```
**Response:**
```json
200 OK
{
"id": "d290f1ee-6c54-4b01-90e6-d701748f0851",
"username": "testuser",
"email": "test@example.com",
"role": "user",
"storageQuota": 10737418240,
"storageUsed": 1048576,
"createdAt": "2023-01-01T12:00:00Z"
}
```
### Change Password
**Request:**
```json
PUT /api/auth/change-password
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
{
"oldPassword": "SecurePassword123",
"newPassword": "NewSecurePassword456"
}
```
**Response:**
```
200 OK
```
### Logout
**Request:**
```
POST /api/auth/logout
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```
**Response:**
```
200 OK
```
## Testing
1. Start PostgreSQL and create the database:
```bash
createdb oxicloud
psql -d oxicloud -f db/schema.sql
```
2. Set environment variables:
```bash
source test-auth-env.sh
```
3. Start the server:
```bash
cargo run
```
4. Run the auth test script:
```bash
./test-auth-api.sh
```
## Database Schema
The auth system uses these tables in the **auth** schema:
- **auth.users** -- user info (includes **oidc_provider** and **oidc_subject** columns for OIDC users)
- **auth.sessions** -- refresh token sessions
- **auth.user_files** -- file ownership (user_id, file_path, file_id, size_bytes)
- **auth.user_favorites** -- user favorites (user_id, item_id, item_type)
- **auth.user_recent_files** -- recently accessed files (user_id, item_id, item_type, accessed_at)
- **auth.admin_settings** -- admin settings (key-value with category and secret flag)
## Implementation Details
- **Password hashing**: Argon2id with memory cost 65536 (64MB), time cost 3, parallelism 4
- **JWT secret**: configured via **OXICLOUD_JWT_SECRET** environment variable
- **Token expiry**: access token 1 hour, refresh token 30 days (configurable)
- **Database connection**: PostgreSQL with connection pooling
- **Middleware**: auth middleware for protected routes
## Security
- Passwords stored only as Argon2id hashes, never in plain text
- JWT tokens signed with a secret key
- Refresh tokens can be revoked to force logout
- Rate limiting should be applied to login attempts
- Password policy requires at least 8 characters
See `oidc-integration.md` for OIDC/SSO authentication details.
## Future Work
- Email verification for new registrations
- Password reset functionality
- Enhanced password policy
- Two-factor authentication
- OAuth integration for social logins
- Session management UI
doc/search.md
-120
View File
@@ -1,120 +0,0 @@
# 12 - Search
File and folder search with multi-criteria filtering, recursive traversal, pagination, and in-memory result caching. Two modes: a simple `GET` with query parameters, or an advanced `POST` with a full criteria body.
Controlled by the feature flag **OXICLOUD_ENABLE_SEARCH** (default: `true`).
## Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **SearchUseCase** trait | `src/application/ports/inbound.rs` |
| Application Service | **SearchService** | `src/application/services/search_service.rs` |
| Application DTOs | **SearchCriteriaDto**, **SearchResultsDto** | `src/application/dtos/search_dto.rs` |
| Interfaces | **SearchHandler** | `src/interfaces/api/handlers/search_handler.rs` |
## Search Criteria
```rust
pub struct SearchCriteriaDto {
pub name_contains: Option<String>, // text to search in file/folder names
pub file_types: Option<Vec<String>>, // extensions, e.g. ["pdf", "jpg"]
pub created_after: Option<u64>, // epoch seconds
pub created_before: Option<u64>,
pub modified_after: Option<u64>,
pub modified_before: Option<u64>,
pub min_size: Option<u64>, // bytes
pub max_size: Option<u64>,
pub folder_id: Option<String>, // scope to a specific folder
pub recursive: bool, // default: true
pub limit: usize, // default: 100
pub offset: usize, // default: 0
}
```
## Search Results
```rust
pub struct SearchResultsDto {
pub files: Vec<FileDto>,
pub folders: Vec<FolderDto>,
pub total_count: Option<usize>,
pub limit: usize,
pub offset: usize,
pub has_more: bool,
}
```
## REST API Endpoints
All routes under `/api/search`, require authentication.
| Method | Path | Handler | Description |
|---|---|---|---|
| `GET` | `/api/search/` | `SearchHandler::search_files_get` | Simple search via query params |
| `POST` | `/api/search/advanced` | `SearchHandler::search_files_post` | Advanced search via JSON body |
| `DELETE` | `/api/search/cache` | `SearchHandler::clear_search_cache` | Clear the result cache |
### GET Query Parameters
| Parameter | Type | Description |
|---|---|---|
| `query` | `String` | Text to search in names |
| `type` | `String` | File extension filter (comma-separated) |
| `created_after` | `u64` | Epoch seconds |
| `created_before` | `u64` | Epoch seconds |
| `modified_after` | `u64` | Epoch seconds |
| `modified_before` | `u64` | Epoch seconds |
| `min_size` | `u64` | Minimum size in bytes |
| `max_size` | `u64` | Maximum size in bytes |
| `folder_id` | `String` | Scope to folder |
| `recursive` | `bool` | Recursive search (default: true) |
| `limit` | `usize` | Max results (default: 100) |
| `offset` | `usize` | Pagination offset |
### Example
```bash
# Simple search
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/search/?query=report&type=pdf&limit=20"
# Advanced search
curl -X POST -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"name_contains":"report","file_types":["pdf","docx"],"min_size":1024,"recursive":true,"limit":50}' \
"https://oxicloud.example.com/api/search/advanced"
```
## Implementation Details
### Search Algorithm
1. If `folder_id` is provided, start from that folder. Otherwise start from root.
2. When `recursive` is true, traverse all subfolders depth-first via `search_recursive()`.
3. Files are filtered by name match, extension, date range, and size range.
4. Folders are filtered by name match and date range.
5. Results are paginated with `offset`/`limit`.
### Caching
Results are cached in-memory using a hash of the search criteria + user ID as the key.
- **Cache TTL**: 5 minutes (300 seconds)
- **Max entries**: 1000
- **Cleanup**: background `tokio` task runs periodically, evicts expired entries
- **Manual clear**: `DELETE /api/search/cache`
## DI Wiring
```rust
// In di.rs — SearchService creation
let search_service = SearchService::new(
repos.file_read_repository.clone(),
repos.folder_repository.clone(),
300, // cache TTL seconds
1000, // max cache entries
);
```
Stored in `AppState.applications.search_service: Option<Arc<dyn SearchUseCase>>`.
doc/share-integration.md
-367
View File
@@ -1,367 +0,0 @@
# 15 - Share Integration
File and folder sharing via public links. Users generate access links that work even for people without accounts. Supports optional password protection, expiration, and granular permissions. Follows hexagonal architecture throughout.
## Domain Entities
**Share** (`src/domain/entities/share.rs`) -- the core entity representing a shared resource:
```rust
pub struct Share {
pub id: String, // unique link identifier
pub item_id: String, // ID of shared file or folder
pub item_type: ShareItemType, // File or Folder
pub token: String, // unique token for public access
pub password_hash: Option<String>, // optional password hash
pub expires_at: Option<u64>, // optional expiration timestamp
pub permissions: SharePermissions, // granted permissions
pub created_at: u64, // creation timestamp
pub created_by: String, // creator user ID
pub access_count: u64, // access counter
}
pub enum ShareItemType {
File,
Folder
}
pub struct SharePermissions {
pub read: bool, // read permission
pub write: bool, // write permission
pub reshare: bool, // re-share permission
}
```
The entity has methods to validate expiration, verify passwords, increment the access counter, and modify properties (permissions, password, expiration).
## Repository Interface
**ShareRepository** (`src/domain/repositories/share_repository.rs`) defines persistence operations:
```rust
#[async_trait]
pub trait ShareRepository: Send + Sync + 'static {
async fn save(&self, share: &Share) -> Result<Share, ShareRepositoryError>;
async fn find_by_id(&self, id: &str) -> Result<Share, ShareRepositoryError>;
async fn find_by_token(&self, token: &str) -> Result<Share, ShareRepositoryError>;
async fn find_by_item(&self, item_id: &str, item_type: &ShareItemType) -> Result<Vec<Share>, ShareRepositoryError>;
async fn update(&self, share: &Share) -> Result<Share, ShareRepositoryError>;
async fn delete(&self, id: &str) -> Result<(), ShareRepositoryError>;
async fn find_by_user(&self, user_id: &str, offset: usize, limit: usize) -> Result<(Vec<Share>, usize), ShareRepositoryError>;
}
```
## Application Ports
**ShareUseCase** and **ShareStoragePort** (`src/application/ports/share_ports.rs`):
```rust
#[async_trait]
pub trait ShareUseCase: Send + Sync + 'static {
async fn create_shared_link(&self, user_id: &str, dto: CreateShareDto) -> Result<ShareDto, DomainError>;
async fn get_shared_link(&self, id: &str) -> Result<ShareDto, DomainError>;
async fn get_shared_link_by_token(&self, token: &str) -> Result<ShareDto, DomainError>;
async fn get_shared_links_for_item(&self, item_id: &str, item_type: &ShareItemType) -> Result<Vec<ShareDto>, DomainError>;
async fn update_shared_link(&self, id: &str, dto: UpdateShareDto) -> Result<ShareDto, DomainError>;
async fn delete_shared_link(&self, id: &str) -> Result<(), DomainError>;
async fn get_user_shared_links(&self, user_id: &str, page: usize, per_page: usize) -> Result<PaginatedResponseDto<ShareDto>, DomainError>;
async fn verify_shared_link_password(&self, token: &str, password: &str) -> Result<bool, DomainError>;
async fn register_shared_link_access(&self, token: &str) -> Result<(), DomainError>;
}
#[async_trait]
pub trait ShareStoragePort: Send + Sync + 'static {
async fn save_share(&self, share: &Share) -> Result<Share, DomainError>;
async fn find_share_by_id(&self, id: &str) -> Result<Share, DomainError>;
// ... other methods
}
```
## DTOs
**DTOs** (`src/application/dtos/share_dto.rs`):
```rust
pub struct CreateShareDto {
pub item_id: String,
pub item_type: String,
pub password: Option<String>,
pub expires_at: Option<u64>,
pub permissions: Option<SharePermissionsDto>,
}
pub struct UpdateShareDto {
pub password: Option<String>,
pub expires_at: Option<u64>,
pub permissions: Option<SharePermissionsDto>,
}
pub struct SharePermissionsDto {
pub read: bool,
pub write: bool,
pub reshare: bool,
}
pub struct ShareDto {
pub id: String,
pub item_id: String,
pub item_type: String,
pub token: String,
pub url: String,
pub password_protected: bool,
pub expires_at: Option<u64>,
pub permissions: SharePermissionsDto,
pub created_at: u64,
pub created_by: String,
pub access_count: u64,
}
```
## Application Service
**ShareService** (`src/application/services/share_service.rs`) implements the business logic:
```rust
pub struct ShareService {
config: Arc<AppConfig>,
share_repository: Arc<dyn ShareStoragePort>,
file_repository: Arc<dyn FileReadPort>,
folder_repository: Arc<dyn FolderStoragePort>,
}
```
Handles: shared element validation, permission management, unique link/token generation, password protection, expiration control, and access tracking.
## Infrastructure
**ShareFsRepository** (`src/infrastructure/repositories/share_fs_repository.rs`) persists share link metadata to a local JSON file:
```rust
pub struct ShareFsRepository {
config: Arc<AppConfig>,
}
struct ShareRecord {
id: String,
item_id: String,
item_type: String,
token: String,
password_hash: Option<String>,
expires_at: Option<u64>,
permissions_read: bool,
permissions_write: bool,
permissions_reshare: bool,
created_at: u64,
created_by: String,
access_count: u64,
}
```
Stores share link records in a JSON file. Supports queries and updates, search by ID/token/user, and pagination.
> **Note:** This repository stores *share link metadata* only (tokens, permissions, expiration). The actual file/folder content is accessed via `FileReadPort` / `FolderStoragePort` which use the blob storage model (PostgreSQL metadata + DedupService blobs).
## API Handlers and Routes
**Handlers** (`src/interfaces/api/handlers/share_handler.rs`):
```rust
pub async fn create_shared_link(
State(share_use_case): State<Arc<dyn ShareUseCase>>,
Json(dto): Json<CreateShareDto>,
) -> impl IntoResponse {
// ...
}
pub async fn get_shared_link(
State(share_use_case): State<Arc<dyn ShareUseCase>>,
Path(id): Path<String>,
) -> impl IntoResponse {
// ...
}
pub async fn get_user_shares(
State(share_use_case): State<Arc<dyn ShareUseCase>>,
Query(query): Query<GetSharesQuery>,
) -> impl IntoResponse {
// ...
}
pub async fn update_shared_link(
State(share_use_case): State<Arc<dyn ShareUseCase>>,
Path(id): Path<String>,
Json(dto): Json<UpdateShareDto>,
) -> impl IntoResponse {
// ...
}
pub async fn delete_shared_link(
State(share_use_case): State<Arc<dyn ShareUseCase>>,
Path(id): Path<String>,
) -> impl IntoResponse {
// ...
}
pub async fn access_shared_item(
State(share_use_case): State<Arc<dyn ShareUseCase>>,
Path(token): Path<String>,
) -> impl IntoResponse {
// ...
}
pub async fn verify_shared_item_password(
State(share_use_case): State<Arc<dyn ShareUseCase>>,
Path(token): Path<String>,
Json(req): Json<VerifyPasswordRequest>,
) -> impl IntoResponse {
// ...
}
```
**Routes** (`src/interfaces/api/routes.rs`):
```rust
// Private routes for managing shared links
let share_router = Router::new()
.route("/", post(share_handler::create_shared_link))
.route("/", get(share_handler::get_user_shares))
.route("/{id}", get(share_handler::get_shared_link))
.route("/{id}", put(share_handler::update_shared_link))
.route("/{id}", delete(share_handler::delete_shared_link));
// Public routes for accessing shared links
let public_share_router = Router::new()
.route("/{token}", get(share_handler::access_shared_item))
.route("/{token}/verify", post(share_handler::verify_shared_item_password));
// Main router configuration
router
.nest("/shares", share_router) // private API: /api/shares/...
.nest("/s", public_share_router); // public API: /api/s/...
```
## System Integration
### Configuration
```rust
pub struct FeaturesConfig {
// ...
pub enable_file_sharing: bool,
// ...
}
```
### Dependency Injection
The service is instantiated via **AppServiceFactory** in `src/common/di.rs` and injected into **AppState**:
```rust
// In AppServiceFactory::create_share_service()
pub fn create_share_service(&self, repos: &RepositoryServices)
-> Option<Arc<dyn ShareUseCase>>
{
if !self.config.features.enable_file_sharing {
return None;
}
let share_repository = Arc::new(ShareFsRepository::new(
Arc::new(self.config.clone())
));
let password_hasher: Arc<dyn PasswordHasherPort> =
Arc::new(Argon2PasswordHasher::new());
let service = Arc::new(ShareService::new(
Arc::new(self.config.clone()),
share_repository,
repos.file_read_repository.clone(), // FileBlobReadRepository
repos.folder_repository.clone(), // FolderDbRepository
password_hasher,
));
Some(service)
}
```
## Workflows
### Creating a Shared Link
1. User selects a file or folder to share.
2. Frontend sends a POST to `/api/shares/` with details (optional password, expiration, permissions).
3. `ShareService.create_shared_link()` validates data and verifies the item exists.
4. A unique token and access URL are generated.
5. The link is saved to the repository.
6. The URL and link details are returned.
### Accessing a Shared Resource
1. Someone opens a shared link (e.g., `http://oxicloud.example/api/s/{token}`).
2. Backend checks: valid token, not expired, password-protected or not.
3. If password-protected, the user is prompted.
4. Access counter increments.
5. Resource metadata is returned for display in the UI.
6. The user can access content according to the granted permissions.
## Security
**Password Protection** -- passwords are stored as hashes, not plaintext. Currently uses a simple hash but the design supports stronger algorithms like bcrypt.
**Expiration Control** -- links can be configured to expire automatically. The system checks expiration before granting access.
**Permission Control** -- granular permission model (read, write, reshare). Each operation validates permissions before allowing the action.
## Error Handling
```rust
pub enum ShareServiceError {
#[error("Share not found: {0}")]
NotFound(String),
#[error("Item not found: {0}")]
ItemNotFound(String),
#[error("Access denied: {0}")]
AccessDenied(String),
#[error("Invalid password: {0}")]
InvalidPassword(String),
#[error("Share expired")]
Expired,
#[error("Repository error: {0}")]
Repository(String),
#[error("Invalid item type: {0}")]
InvalidItemType(String),
#[error("Validation error: {0}")]
Validation(String),
}
```
HTTP status code mapping:
- `NotFound` -> HTTP 404
- `PasswordRequired` -> HTTP 401 + metadata
- `Expired` -> HTTP 410 Gone
- `AccessDenied` -> HTTP 403
- `ValidationError` -> HTTP 400
## Future Enhancements
1. **Notifications** -- alert users when their shared resources are accessed
2. **Activity Log** -- detailed audit trail of who accessed what and when
3. **Usage Limits** -- max access count or bandwidth per shared link
4. **Advanced Statistics** -- detailed metrics on shared resource usage
5. **Alternative Persistence** -- database or cloud storage backends (same interface)
## Technical Notes
- **Share metadata** is stored in a local JSON file via `ShareFsRepository`. This is separate from the 100% blob storage model used for file content.
- **File/folder lookups** during share access go through `FileReadPort` / `FolderStoragePort`, which read metadata from PostgreSQL and content from the DedupService blob store.
- **Scalability**: for higher load, share metadata could be migrated to PostgreSQL using the same hexagonal architecture (implement `ShareRepository` with PgPool).
- **Maintenance**: clear separation of concerns makes testing and maintenance straightforward.
The sharing feature is enabled via `OXICLOUD_ENABLE_FILE_SHARING` configuration flag.
doc/storage-quotas.md
-74
View File
@@ -1,74 +0,0 @@
# 11 - Storage Quotas
Per-user storage quotas track disk usage and can limit how much storage each user consumes. Controlled by the feature flag **OXICLOUD_ENABLE_USER_STORAGE_QUOTAS** (default: `false`).
## Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **StorageUsagePort** trait | `src/application/ports/storage_ports.rs` |
| Application Service | **StorageUsageService** | `src/application/services/storage_usage_service.rs` |
| Admin API | `/api/admin/users/{id}/quota` | `src/interfaces/api/handlers/admin_handler.rs` |
## Port Trait
```rust
#[async_trait]
pub trait StorageUsagePort: Send + Sync + 'static {
async fn update_user_storage_usage(&self, user_id: &str) -> Result<i64, DomainError>;
async fn update_all_users_storage_usage(&self) -> Result<(), DomainError>;
}
```
## How Usage is Calculated
1. Look up the user's username by ID.
2. Find the user's home folder: `"Mi Carpeta - {username}"` (naming convention).
3. Recursively traverse all subfolders, summing file sizes.
4. Skip directory entries (`mime_type = "directory"` or `"application/directory"`).
5. Update `auth.users.storage_used` via **UserStoragePort**.
`update_all_users_storage_usage()` processes all users concurrently via `tokio::spawn`.
## Admin Quota Management
Admins set per-user quotas through the admin API:
```bash
# Set 10 GB quota for a user
curl -X PUT -H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{"quota_bytes": 10737418240}' \
"https://oxicloud.example.com/api/admin/users/{user_id}/quota"
```
## Dashboard Stats
The admin dashboard (`GET /api/admin/dashboard`) includes quota-related metrics:
```json
{
"quotas_enabled": true,
"total_quota_bytes": 107374182400,
"total_used_bytes": 53687091200,
"storage_usage_percent": 50.0,
"users_over_80_percent": 5,
"users_over_quota": 1
}
```
## Configuration
```bash
# Enable storage quotas (default: false)
OXICLOUD_ENABLE_USER_STORAGE_QUOTAS=true
```
Part of **FeaturesConfig** in `src/common/config.rs`:
```rust
pub struct FeaturesConfig {
pub enable_user_storage_quotas: bool, // default: false
// ...
}
```
doc/thumbnails-and-transcoding.md
-191
View File
@@ -1,191 +0,0 @@
# 08 - Thumbnails and Transcoding
OxiCloud provides two image optimization features:
- **Thumbnails**: on-demand generation of WebP thumbnails in 3 sizes, with background pre-generation on upload
- **Image Transcoding**: automatic JPEG/PNG/GIF → WebP conversion based on browser `Accept` header
Both features use multi-level caching (memory LRU + disk) and are non-blocking.
---
## Thumbnails
### Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **ThumbnailPort** trait, **ThumbnailSize** enum | `src/application/ports/thumbnail_ports.rs` |
| Infrastructure | **ThumbnailService** | `src/infrastructure/services/thumbnail_service.rs` |
| Interfaces | Integrated in **FileHandler** | `src/interfaces/api/handlers/file_handler.rs` |
### Thumbnail Sizes
| Size | Dimensions | Directory Name |
|---|---|---|
| `Icon` | 150x150 | `icon` |
| `Preview` | 400x400 | `preview` |
| `Large` | 800x800 | `large` |
### Supported Formats
Input: `image/jpeg`, `image/jpg`, `image/png`, `image/gif`, `image/webp`
Output: always **WebP** (using Lanczos3 resize filter)
### Storage Layout
```
<storage_path>/
.thumbnails/
icon/
<file_id>.webp
preview/
<file_id>.webp
large/
<file_id>.webp
```
### REST API
| Method | Path | Description |
|---|---|---|
| `GET` | `/api/files/{id}/thumbnail/{size}` | Get thumbnail (`size` = `icon` / `preview` / `large`) |
| `POST` | `/api/files/upload` | Upload file + auto-generate thumbnails for images |
Response headers for thumbnail GET:
- `Content-Type: image/webp`
- `Cache-Control: public, max-age=31536000, immutable`
- `ETag: "thumb-{id}-{size}"`
### Generation Flow
1. **On upload** (`upload_file_with_thumbnails`): after successful file upload, if the MIME type is a supported image, thumbnails for all 3 sizes are generated in a background `tokio::spawn` task
2. **On GET** (lazy): if a thumbnail doesn't exist on disk, it's generated on-demand, cached in memory and on disk
### Caching
- **Memory LRU cache**: configurable max entries and max bytes
- **Disk cache**: persistent WebP files in `.thumbnails/`
- Cache lookup order: memory → disk → generate
### Port Trait
```rust
#[async_trait]
pub trait ThumbnailPort: Send + Sync + 'static {
fn is_supported_image(&self, mime_type: &str) -> bool;
async fn get_thumbnail(&self, file_id: &str, size: ThumbnailSize, original_path: &Path) -> Result<Bytes, DomainError>;
fn generate_all_sizes_background(self: Arc<Self>, file_id: String, original_path: PathBuf);
async fn delete_thumbnails(&self, file_id: &str) -> Result<(), DomainError>;
async fn get_stats(&self) -> ThumbnailStatsDto;
}
```
---
## Image Transcoding
### Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **ImageTranscodePort** trait, **OutputFormat**, **BrowserCapabilities** | `src/application/ports/transcode_ports.rs` |
| Infrastructure | **ImageTranscodeService** | `src/infrastructure/services/image_transcode_service.rs` |
| Interfaces | Integrated in download handler | `src/interfaces/api/handlers/file_handler.rs` |
### How It Works
1. Download handler reads the `Accept` header from the HTTP request
2. `BrowserCapabilities::from_accept_header()` determines if the browser supports WebP
3. If the file is a transcodable image (JPEG/PNG/GIF, ≤5 MB) and the browser supports WebP: check memory cache → check disk cache → transcode on miss
4. If WebP output is **larger** than the original, serve the original instead (smart skip)
### Constants
| Constant | Value | Description |
|---|---|---|
| `MAX_TRANSCODE_SIZE` | 5 MB | Files above this skip transcoding |
### Supported Input Formats
`image/jpeg`, `image/jpg`, `image/png`, `image/gif`
Not transcoded: `image/webp` (already optimal), `image/svg+xml`, `image/bmp`
### Storage Layout
```
<storage_path>/
.transcoded/
webp/
<file_id>.webp
```
### Caching
- **Memory LRU cache**: configurable max entries and max bytes
- **Disk cache**: persistent WebP files in `.transcoded/webp/`
- Disk writes are fire-and-forget via `tokio::spawn` (non-blocking)
- `invalidate(file_id)` evicts from both memory and disk
### Port Trait
```rust
#[async_trait]
pub trait ImageTranscodePort: Send + Sync + 'static {
fn can_transcode(&self, mime_type: &str) -> bool;
fn should_transcode(&self, mime_type: &str, file_size: u64) -> bool;
async fn get_transcoded(&self, file_id: &str, original_content: &[u8], original_mime: &str, target_format: OutputFormat)
-> Result<(Bytes, String, bool), DomainError>;
async fn invalidate(&self, file_id: &str);
async fn get_stats(&self) -> TranscodeStatsDto;
async fn clear_cache(&self) -> Result<(), DomainError>;
}
```
### Browser Detection
```rust
pub struct BrowserCapabilities {
pub supports_webp: bool,
pub supports_avif: bool, // reserved for future
}
impl BrowserCapabilities {
pub fn from_accept_header(accept: Option<&str>) -> Self;
pub fn best_format(&self) -> Option<OutputFormat>;
}
```
### Statistics
```rust
pub struct TranscodeStatsDto {
pub cache_hits: u64,
pub disk_hits: u64,
pub transcodes: u64,
pub bytes_saved: u64,
pub transcode_errors: u64,
}
```
### Example Flow
```
Client: GET /api/files/abc-123/download
Accept: image/webp, image/png, */*
Server: 1. File is "photo.jpg" (800KB) → can transcode yes, should transcode yes
2. Check memory cache → miss
3. Check disk cache (.transcoded/webp/abc-123.webp) → miss
4. Transcode JPEG → WebP (600KB) → smaller yes
5. Cache in memory + async write to disk
6. Respond with WebP (saves 200KB, 25% reduction)
```
## Dependencies
- `image = "0.25"` (with `jpeg`, `png`, `gif`, `webp` features) -- used by both thumbnail and transcode
- Thumbnail resize: `imageops::resize` with `FilterType::Lanczos3`
- WebP encoding: via `image` crate's WebP encoder
doc/trash-feature-summary.md
-117
View File
@@ -1,117 +0,0 @@
# 14 - Trash Feature
Soft-delete for files and folders. Items go to a per-user trash bin instead of being permanently removed. Configurable retention period with automatic cleanup.
## Architecture
Follows the hexagonal architecture:
1. **Domain Layer** (`/src/domain/`):
- Entities: **TrashedItem** representing files and folders in the trash
- Repository interfaces: **TrashRepository** defining trash management operations
2. **Application Layer** (`/src/application/`):
- DTOs: **TrashedItemDto** for data transfer between layers
- Ports: **TrashUseCase** defining available operations
- Services: **TrashService** implementing the trash use cases
3. **Infrastructure Layer** (`/src/infrastructure/`):
- Repositories: **TrashDbRepository** (PostgreSQL) — reads from `storage.trash_items` VIEW, manages soft-delete flags
- Trash-related methods in file/folder repositories: `FileBlobWriteRepository::move_to_trash()`, `FolderDbRepository::move_to_trash()`, etc.
- Services: **TrashCleanupService** for automatic cleanup of expired items
4. **Interface Layer** (`/src/interfaces/`):
- API handlers: `trash_handler.rs` with HTTP endpoints for trash operations
- Routes: updated `routes.rs` to include trash endpoints
## Storage Model
Trash uses a **soft-delete** model in PostgreSQL:
- Files and folders have `is_trashed` (BOOLEAN) and `trashed_at` (TIMESTAMPTZ) columns
- When an item is trashed, `is_trashed` is set to `TRUE` and `trashed_at` records the timestamp
- `original_parent_id` / `original_folder_id` stores the original location for restore
- The `storage.trash_items` VIEW provides a unified list of all trashed items (files + folders)
- No physical file movement occurs — blob content stays at `.blobs/{prefix}/{hash}.blob`
- Permanent deletion removes the DB row and decrements the blob reference counter
```sql
CREATE OR REPLACE VIEW storage.trash_items AS
SELECT id, name, 'file' AS item_type, folder_id AS parent_id,
user_id, size, mime_type, trashed_at, created_at
FROM storage.files WHERE is_trashed = TRUE
UNION ALL
SELECT id, name, 'folder' AS item_type, parent_id,
user_id, 0 AS size, NULL AS mime_type, trashed_at, created_at
FROM storage.folders WHERE is_trashed = TRUE;
```
## Key Features
1. **Soft Deletion** — files and folders are flagged as trashed, not immediately deleted
2. **Per-User Trash** — each user has an isolated trash bin (filtered by `user_id`)
3. **Retention Policy** — items auto-delete after a configurable period
4. **Restoration** — items can be restored to their original location via `original_parent_id`/`original_folder_id`
5. **Permanent Deletion** — items can be permanently deleted before retention expires (removes DB row + decrements blob ref)
6. **Empty Trash** — wipe everything in the trash at once
## API Endpoints
- `GET /api/trash` or `GET /api/trash/` — list all items in the user's trash
- `DELETE /api/trash/files/:id` — move a file to trash
- `DELETE /api/trash/folders/:id` — move a folder to trash
- `POST /api/trash/:id/restore` — restore an item to its original location
- `DELETE /api/trash/:id` — permanently delete an item from trash
- `DELETE /api/trash/empty` — empty the entire trash bin
## Implementation Details
### TrashDbRepository
**File:** `src/infrastructure/repositories/pg/trash_db_repository.rs`
```rust
pub struct TrashDbRepository {
pool: Arc<PgPool>,
retention_days: u32,
}
```
Key methods:
- `get_trash_items(user_id)` — SELECT from `storage.trash_items` WHERE `user_id = $1`
- `clear_trash(user_id)` — DELETE from `storage.files` and `storage.folders` WHERE `is_trashed = TRUE AND user_id = $1`
- `get_expired_items()` — finds items where `trashed_at + retention_days < NOW()`
### TrashService
**File:** `src/application/services/trash_service.rs`
Constructor: `TrashService::new(trash_repo, file_read, file_write, folder_repo, retention_days)`
Orchestrates trash operations by delegating to the appropriate repository:
- Moving a file to trash → `FileBlobWriteRepository::move_to_trash()`
- Moving a folder to trash → `FolderDbRepository::move_to_trash()`
- Permanent deletion → removes DB row + calls `DedupService::decrement_ref()` to clean blob if unreferenced
### TrashCleanupService
**File:** `src/infrastructure/services/trash_cleanup_service.rs`
Background job that runs every 24 hours to permanently delete items past the retention period.
## Testing
1. **Unit Tests** — testing **TrashService**:
- Move files and folders to trash
- Restore items from trash
- Permanent deletion
- Empty trash operation
2. **Integration Tests** — Python script hitting the API endpoints:
- End-to-end testing of all trash operations
- Verification of move, list, restore, and delete behavior
## Configuration
- **OXICLOUD_ENABLE_TRASH**: enable/disable the trash feature via **FeaturesConfig** (default: true)
- **OXICLOUD_TRASH_RETENTION_DAYS**: days to keep items before automatic deletion (default: 30, via **StorageConfig**)
doc/webdav-integration-guide.md
-697
View File
@@ -1,697 +0,0 @@
# 22 - WebDAV Integration Guide
The WebDAV interface exposes file operations over HTTP at a single base path. All standard WebDAV methods are supported: **PROPFIND**, **GET**, **PUT**, **MKCOL**, **MOVE**, **COPY**, **DELETE**. Authentication is HTTP Basic over TLS.
## Table of Contents
1. [Base URL and Endpoints](#base-url-and-endpoints)
2. [Authentication](#authentication)
3. [Common Operations](#common-operations)
- [Listing Directories](#listing-directories)
- [Downloading Files](#downloading-files)
- [Uploading Files](#uploading-files)
- [Creating Folders](#creating-folders)
- [Moving and Copying](#moving-and-copying)
- [Deleting Resources](#deleting-resources)
4. [XML Schemas](#xml-schemas)
5. [Code Examples](#code-examples)
6. [Extending WebDAV](#extending-webdav)
7. [Troubleshooting](#troubleshooting)
## Base URL and Endpoints
The WebDAV interface lives at:
```
https://[your-oxicloud-server]/webdav/
```
All file and folder operations hang off this base path. Append the resource path to the URL.
Examples:
- Root folder: `https://[your-oxicloud-server]/webdav/`
- File "document.pdf" in root: `https://[your-oxicloud-server]/webdav/document.pdf`
- Folder "projects": `https://[your-oxicloud-server]/webdav/projects/`
- File in subfolder: `https://[your-oxicloud-server]/webdav/projects/proposal.docx`
## Authentication
WebDAV uses HTTP Basic Authentication. Include the `Authorization` header with base64-encoded credentials:
```
Authorization: Basic base64(username:password)
```
Always use HTTPS.
## Common Operations
### Listing Directories
Use the **PROPFIND** method with a **Depth** header:
- `Depth: 0` -- info about the resource itself
- `Depth: 1` -- the resource and its immediate children (recommended)
- `Depth: infinity` -- the resource and all descendants (careful with large trees)
Request:
```http
PROPFIND /webdav/projects/ HTTP/1.1
Host: your-oxicloud-server
Depth: 1
Content-Type: application/xml
Authorization: Basic [credentials]
<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:allprop/>
</D:propfind>
```
Response:
```http
HTTP/1.1 207 Multi-Status
Content-Type: application/xml; charset=utf-8
```
### Downloading Files
Standard HTTP **GET**:
```http
GET /webdav/projects/document.pdf HTTP/1.1
Host: your-oxicloud-server
Authorization: Basic [credentials]
```
Response:
```http
HTTP/1.1 200 OK
Content-Type: application/pdf
Content-Length: 12345
Last-Modified: Wed, 15 Nov 2023 12:34:56 GMT
ETag: "abc123"
[File content]
```
### Uploading Files
Use HTTP **PUT** to upload or update a file:
```http
PUT /webdav/projects/document.pdf HTTP/1.1
Host: your-oxicloud-server
Content-Type: application/pdf
Content-Length: 12345
Authorization: Basic [credentials]
[File content]
```
New files return `201 Created`. Updates return `204 No Content`.
### Creating Folders
Use the **MKCOL** method:
```http
MKCOL /webdav/projects/new-folder HTTP/1.1
Host: your-oxicloud-server
Authorization: Basic [credentials]
```
Returns `201 Created` on success.
### Moving and Copying
**MOVE** a resource:
```http
MOVE /webdav/old-location.pdf HTTP/1.1
Host: your-oxicloud-server
Destination: https://your-oxicloud-server/webdav/new-location.pdf
Authorization: Basic [credentials]
```
**COPY** a resource:
```http
COPY /webdav/original.pdf HTTP/1.1
Host: your-oxicloud-server
Destination: https://your-oxicloud-server/webdav/copy.pdf
Authorization: Basic [credentials]
```
Both return `204 No Content` on success.
### Deleting Resources
Use HTTP **DELETE**:
```http
DELETE /webdav/projects/document.pdf HTTP/1.1
Host: your-oxicloud-server
Authorization: Basic [credentials]
```
Returns `204 No Content` on success.
## XML Schemas
### PROPFIND Request
Request all properties:
```xml
<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:allprop/>
</D:propfind>
```
Request specific properties:
```xml
<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:prop>
<D:displayname/>
<D:getcontentlength/>
<D:getlastmodified/>
</D:prop>
</D:propfind>
```
### PROPPATCH Request
Set and remove properties:
```xml
<?xml version="1.0" encoding="utf-8" ?>
<D:propertyupdate xmlns:D="DAV:" xmlns:Z="http://example.org/custom/">
<D:set>
<D:prop>
<Z:custom-property>Custom Value</Z:custom-property>
</D:prop>
</D:set>
<D:remove>
<D:prop>
<Z:old-property/>
</D:prop>
</D:remove>
</D:propertyupdate>
```
### LOCK Request
```xml
<?xml version="1.0" encoding="utf-8" ?>
<D:lockinfo xmlns:D="DAV:">
<D:lockscope><D:exclusive/></D:lockscope>
<D:locktype><D:write/></D:locktype>
<D:owner>
<D:href>mailto:user@example.com</D:href>
</D:owner>
</D:lockinfo>
```
## Code Examples
### Python Example
Using the `requests` library:
```python
import requests
from requests.auth import HTTPBasicAuth
import xml.etree.ElementTree as ET
# Set up authentication
auth = HTTPBasicAuth('username', 'password')
base_url = 'https://your-oxicloud-server/webdav'
# 1. List directory contents
headers = {'Depth': '1'}
body = '''<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:allprop/>
</D:propfind>'''
response = requests.request(
'PROPFIND',
f'{base_url}/projects/',
headers=headers,
data=body,
auth=auth
)
if response.status_code == 207: # Multi-Status
# Parse XML response
root = ET.fromstring(response.content)
for response_elem in root.findall('.//{DAV:}response'):
href = response_elem.find('.//{DAV:}href').text
print(f"Resource: {href}")
# Get displayname if available
displayname = response_elem.find('.//{DAV:}displayname')
if displayname is not None and displayname.text:
print(f" Name: {displayname.text}")
# Check if it's a collection (folder)
resourcetype = response_elem.find('.//{DAV:}resourcetype')
is_collection = resourcetype is not None and resourcetype.find('.//{DAV:}collection') is not None
print(f" Type: {'Folder' if is_collection else 'File'}")
# Get size if it's a file
if not is_collection:
contentlength = response_elem.find('.//{DAV:}getcontentlength')
if contentlength is not None and contentlength.text:
print(f" Size: {contentlength.text} bytes")
# 2. Upload a file
with open('local-file.pdf', 'rb') as f:
file_content = f.read()
response = requests.put(
f'{base_url}/projects/document.pdf',
data=file_content,
auth=auth
)
if response.status_code in (201, 204):
print("File uploaded successfully")
# 3. Download a file
response = requests.get(
f'{base_url}/projects/document.pdf',
auth=auth
)
if response.status_code == 200:
with open('downloaded-file.pdf', 'wb') as f:
f.write(response.content)
print("File downloaded successfully")
# 4. Create a folder
response = requests.request(
'MKCOL',
f'{base_url}/projects/new-folder',
auth=auth
)
if response.status_code == 201:
print("Folder created successfully")
# 5. Move a file
headers = {
'Destination': f'{base_url}/projects/new-location.pdf'
}
response = requests.request(
'MOVE',
f'{base_url}/projects/old-location.pdf',
headers=headers,
auth=auth
)
if response.status_code == 204:
print("File moved successfully")
# 6. Delete a file
response = requests.delete(
f'{base_url}/projects/document.pdf',
auth=auth
)
if response.status_code == 204:
print("File deleted successfully")
```
### JavaScript Example
Using the browser `fetch` API:
```javascript
// Base configuration
const baseUrl = 'https://your-oxicloud-server/webdav';
const credentials = btoa('username:password');
const headers = {
'Authorization': `Basic ${credentials}`
};
// 1. List directory contents
async function listDirectory(path) {
const response = await fetch(`${baseUrl}${path}`, {
method: 'PROPFIND',
headers: {
...headers,
'Depth': '1',
'Content-Type': 'application/xml'
},
body: `<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:allprop/>
</D:propfind>`
});
if (response.status === 207) {
const text = await response.text();
const parser = new DOMParser();
const xmlDoc = parser.parseFromString(text, 'text/xml');
const responses = xmlDoc.getElementsByTagNameNS('DAV:', 'response');
const resources = [];
for (let i = 0; i < responses.length; i++) {
const response = responses[i];
const href = response.getElementsByTagNameNS('DAV:', 'href')[0].textContent;
let displayName = '';
const displayNameElems = response.getElementsByTagNameNS('DAV:', 'displayname');
if (displayNameElems.length > 0) {
displayName = displayNameElems[0].textContent;
}
// Check if resource is a collection (folder)
const resourceTypeElem = response.getElementsByTagNameNS('DAV:', 'resourcetype')[0];
const isCollection = resourceTypeElem.getElementsByTagNameNS('DAV:', 'collection').length > 0;
// Get file size if it's a file
let size = null;
if (!isCollection) {
const contentLengthElems = response.getElementsByTagNameNS('DAV:', 'getcontentlength');
if (contentLengthElems.length > 0) {
size = parseInt(contentLengthElems[0].textContent, 10);
}
}
resources.push({
href,
displayName,
isCollection,
size
});
}
return resources;
} else {
throw new Error(`Failed to list directory: ${response.status}`);
}
}
// 2. Upload a file
async function uploadFile(path, fileContent) {
const response = await fetch(`${baseUrl}${path}`, {
method: 'PUT',
headers: {
...headers,
'Content-Type': 'application/octet-stream'
},
body: fileContent
});
return response.status === 201 || response.status === 204;
}
// Example usage with a File object from an input
const fileInput = document.getElementById('fileInput');
fileInput.addEventListener('change', async (event) => {
const file = event.target.files[0];
if (file) {
const result = await uploadFile(`/projects/${file.name}`, file);
console.log(`Upload ${result ? 'successful' : 'failed'}`);
}
});
// 3. Download a file
async function downloadFile(path) {
const response = await fetch(`${baseUrl}${path}`, {
method: 'GET',
headers
});
if (response.status === 200) {
return await response.blob();
} else {
throw new Error(`Failed to download: ${response.status}`);
}
}
// Example usage with download attribute
async function downloadAndSave(path, filename) {
try {
const blob = await downloadFile(path);
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
// Clean up
document.body.removeChild(a);
URL.revokeObjectURL(url);
} catch (error) {
console.error('Download failed:', error);
}
}
// 4. Create a folder
async function createFolder(path) {
const response = await fetch(`${baseUrl}${path}`, {
method: 'MKCOL',
headers
});
return response.status === 201;
}
// 5. Move a file
async function moveResource(fromPath, toPath) {
const response = await fetch(`${baseUrl}${fromPath}`, {
method: 'MOVE',
headers: {
...headers,
'Destination': `${baseUrl}${toPath}`
}
});
return response.status === 204;
}
// 6. Delete a resource
async function deleteResource(path) {
const response = await fetch(`${baseUrl}${path}`, {
method: 'DELETE',
headers
});
return response.status === 204;
}
```
### C# Example
```csharp
using System;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;
using System.Xml.Linq;
class WebDavClient
{
private readonly HttpClient _httpClient;
private readonly string _baseUrl;
public WebDavClient(string baseUrl, string username, string password)
{
_baseUrl = baseUrl.TrimEnd('/') + "/webdav";
_httpClient = new HttpClient();
// Set Basic Authentication
var credentials = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{username}:{password}"));
_httpClient.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Basic", credentials);
}
public async Task<XDocument> ListDirectoryAsync(string path)
{
var request = new HttpRequestMessage(new HttpMethod("PROPFIND"), $"{_baseUrl}/{path.TrimStart('/')}");
request.Headers.Add("Depth", "1");
request.Content = new StringContent(
@"<?xml version=""1.0"" encoding=""utf-8"" ?>
<D:propfind xmlns:D=""DAV:"">
<D:allprop/>
</D:propfind>",
Encoding.UTF8,
"application/xml"
);
var response = await _httpClient.SendAsync(request);
if (response.StatusCode == System.Net.HttpStatusCode.MultiStatus)
{
var content = await response.Content.ReadAsStringAsync();
return XDocument.Parse(content);
}
throw new Exception($"Failed to list directory: {response.StatusCode}");
}
public async Task<bool> UploadFileAsync(string path, byte[] content)
{
var request = new HttpRequestMessage(HttpMethod.Put, $"{_baseUrl}/{path.TrimStart('/')}");
request.Content = new ByteArrayContent(content);
var response = await _httpClient.SendAsync(request);
return response.StatusCode == System.Net.HttpStatusCode.Created ||
response.StatusCode == System.Net.HttpStatusCode.NoContent;
}
public async Task<byte[]> DownloadFileAsync(string path)
{
var response = await _httpClient.GetAsync($"{_baseUrl}/{path.TrimStart('/')}");
if (response.IsSuccessStatusCode)
{
return await response.Content.ReadAsByteArrayAsync();
}
throw new Exception($"Failed to download file: {response.StatusCode}");
}
public async Task<bool> CreateFolderAsync(string path)
{
var request = new HttpRequestMessage(new HttpMethod("MKCOL"), $"{_baseUrl}/{path.TrimStart('/')}");
var response = await _httpClient.SendAsync(request);
return response.StatusCode == System.Net.HttpStatusCode.Created;
}
public async Task<bool> MoveResourceAsync(string fromPath, string toPath)
{
var request = new HttpRequestMessage(new HttpMethod("MOVE"), $"{_baseUrl}/{fromPath.TrimStart('/')}");
request.Headers.Add("Destination", $"{_baseUrl}/{toPath.TrimStart('/')}");
var response = await _httpClient.SendAsync(request);
return response.StatusCode == System.Net.HttpStatusCode.NoContent;
}
public async Task<bool> DeleteResourceAsync(string path)
{
var response = await _httpClient.DeleteAsync($"{_baseUrl}/{path.TrimStart('/')}");
return response.StatusCode == System.Net.HttpStatusCode.NoContent;
}
}
// Example usage
async Task RunExampleAsync()
{
var client = new WebDavClient("https://your-oxicloud-server", "username", "password");
// List directory
try
{
var directoryListing = await client.ListDirectoryAsync("/projects");
// Process XML results...
Console.WriteLine("Directory listing successful");
}
catch (Exception ex)
{
Console.WriteLine($"Error listing directory: {ex.Message}");
}
// Upload a file
try
{
var fileContent = await File.ReadAllBytesAsync("local-file.pdf");
var result = await client.UploadFileAsync("/projects/document.pdf", fileContent);
Console.WriteLine($"Upload {(result ? "successful" : "failed")}");
}
catch (Exception ex)
{
Console.WriteLine($"Error uploading file: {ex.Message}");
}
// Download a file
try
{
var fileContent = await client.DownloadFileAsync("/projects/document.pdf");
await File.WriteAllBytesAsync("downloaded-file.pdf", fileContent);
Console.WriteLine("Download successful");
}
catch (Exception ex)
{
Console.WriteLine($"Error downloading file: {ex.Message}");
}
}
```
## Extending WebDAV
### Adding Custom Properties
To support custom WebDAV properties:
1. Define your XML namespace for custom properties.
2. Implement storage for them (database table recommended).
3. Update the WebDAV adapter to handle these properties.
Example adapter code for custom properties:
```rust
// Add to WebDavAdapter implementation
fn handle_custom_property(name: &QualifiedName, value: Option<&str>) -> Result<bool> {
if name.namespace == "http://example.org/custom/" {
// Store the custom property in your database
// ...
return Ok(true);
}
// Property not handled
Ok(false)
}
```
### Supporting CalDAV/CardDAV
To extend with CalDAV/CardDAV support:
1. Create additional adapters for calendar and contact data.
2. Implement the additional XML namespaces required.
3. Create handlers for the specialized methods.
4. Integrate with calendar and contacts storage.
See `caldav-technical-spec.md` and `carddav-technical-spec.md` for details.
## Troubleshooting
### Common Issues
1. **Authentication Failures**
- Check credentials are correctly Base64-encoded
- Ensure the Authorization header is formatted correctly
- Verify the user has the necessary permissions
1. Create additional adapters for calendar and contact data.
2. Implement the additional XML namespaces required.
3. Create handlers for the specialized methods.
4. Integrate with calendar and contacts storage.
See `caldav-technical-spec.md` and `carddav-technical-spec.md` for details.
## Troubleshooting
### Common Issues
1. **Authentication Failures**
- Check credentials are correctly Base64-encoded
- Ensure the Authorization header is formatted correctly
- Verify the user has the necessary permissions
doc/webdav-technical-spec.md
-166
View File
@@ -1,166 +0,0 @@
# 21 - WebDAV Technical Spec
OxiCloud's WebDAV implementation follows RFC 4918. Clients can perform file operations over HTTP, so desktop apps, mobile clients, and any WebDAV-compatible software can treat OxiCloud as a remote filesystem.
Supported operations: file/folder browsing, uploads, downloads, create/delete/move resources, metadata retrieval and modification.
## Architecture
The WebDAV layer follows the hexagonal architecture pattern used throughout the codebase:
```
┌────────────────────────────────────────────────────────────────────┐
│ INTERFACES │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ WebDAV Handler │ │
│ │ │ │
│ │ OPTIONS │ PROPFIND │ GET │ PUT │ DELETE │ MOVE │ COPY │ │
│ └─────────────────────────────┬─────────────────────────────┘ │
│ │ │
└─────────────────────────────────┼──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ APPLICATION │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ WebDAV Adapter │ │
│ │ │ │
│ │ XML Processing │ Protocol Translation │ DTOs Conversion │ │
│ └──────────────────────────────┬──────────────────────────────┘ │
│ │ │
│ ▼ │
│ │
│ ┌──────────────┐ ┌───────────────┐ ┌──────────────┐ ┌───────┐ │
│ │ │ │ │ │ │ │ │ │
│ │ FileService │ │ FolderService │ │ AuthService │ │ Other │ │
│ │ │ │ │ │ │ │ │ │
│ └──────┬───────┘ └───────┬───────┘ └──────┬───────┘ └───┬───┘ │
│ │ │ │ │ │
└─────────┼──────────────────┼─────────────────┼──────────────┼──────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────┐
│ DOMAIN │
│ │
│ ┌──────────┐ ┌───────────┐ ┌───────────┐ ┌────────┐ ┌──────┐ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ File │ │ Folder │ │ User │ │ Share │ │ etc. │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ └──────────┘ └───────────┘ └───────────┘ └────────┘ └──────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
### Key Components
1. **WebDAV Handler** (`src/interfaces/api/handlers/webdav_handler.rs`) -- processes HTTP requests for WebDAV methods, maps operations to service calls, manages response formatting.
2. **WebDAV Adapter** (`src/application/adapters/webdav_adapter.rs`) -- converts between WebDAV XML and domain objects, parses PROPFIND/PROPPATCH requests, generates XML responses.
## Data Flow
A typical WebDAV request flows through:
1. Client sends a WebDAV request (e.g., PROPFIND for directory listing)
2. **webdav_handler.rs** receives and authenticates
3. Handler identifies the operation type, passes to **WebDavAdapter**
4. Adapter parses XML, converts to domain objects
5. Handler calls appropriate service methods (e.g., **folder_service.list_folders()**)
6. Domain operations execute via existing services
7. Results go back to the adapter for XML formatting
8. Handler returns the response with proper HTTP headers
## Integration with OxiCloud
### File Operations
Uses **FileService** for uploads, downloads, and management.
### Folder Operations
Uses **FolderService** for directory listing and manipulation. Maintains consistent behavior with the REST API.
### Authentication
Same auth mechanisms as the rest of OxiCloud. Supports HTTP Basic Authentication for WebDAV clients.
### Trash Integration
Integrates with the trash system for file/folder deletion. WebDAV operations use the trash feature when available.
## Request Processing
### PROPFIND (Directory Listing)
```
┌─────────┐ ┌────────────────┐ ┌─────────────────┐ ┌───────────────┐
│ │ │ │ │ │ │ │
│ Client │────▶│ WebDAV Handler │────▶│ WebDAV Adapter │────▶│ FolderService │
│ │ │ │ │ │ │ │
└─────────┘ └────────────────┘ └─────────────────┘ └───────┬───────┘
│
┌─────────┐ ┌────────────────┐ ┌─────────────────┐ ┌───────▼───────┐
│ │ │ │ │ │ │ │
│ Client │◀────│ WebDAV Handler │◀────│ WebDAV Adapter │◀────│ FileService │
│ │ │ │ │ │ │ │
└─────────┘ └────────────────┘ └─────────────────┘ └───────────────┘
```
1. Client sends PROPFIND with Depth header
2. Handler extracts path and depth
3. Adapter parses XML to determine requested properties
4. **FolderService** retrieves folder contents
5. **FileService** retrieves file info if needed
6. Adapter generates XML response with all properties
7. Handler returns 207 Multi-Status
### PUT (File Upload)
```
┌─────────┐ ┌────────────────┐ ┌─────────────────┐
│ │ │ │ │ │
│ Client │────▶│ WebDAV Handler │────▶│ FileService │
│ │ │ │ │ │
└─────────┘ └────────────────┘ └─────────────────┘
│
┌─────────┐ ┌────────────────┐ ┌────────▼────────┐
│ │ │ │ │ │
│ Client │◀────│ WebDAV Handler │◀────│ Response │
│ │ │ │ │ │
└─────────┘ └────────────────┘ └─────────────────┘
```
1. Client sends PUT with file contents
2. Handler extracts path and parent folder info
3. **FileService** uploads the file
4. Handler returns 201 Created or 204 No Content
## Security
- **Authentication** -- same mechanisms as the REST API. Supports HTTP Basic Auth for WebDAV clients. Same permissions model applies.
- **Authorization** -- users can only access their own files through WebDAV. Shared resources keep the same permissions.
- **HTTPS** -- all WebDAV traffic should be served over HTTPS.
- **Input validation** -- all XML inputs strictly validated. Path traversal prevented by proper path normalization.
## Extension Points
1. **Property storage** -- support for custom WebDAV properties via a property database.
2. **CalDAV/CardDAV** -- the architecture allows extending to CalDAV (calendar) and CardDAV (contacts), both built on the WebDAV foundation.
3. **Advanced locking** -- full WebDAV locking for collaborative editing.
## Implementation Status
| Method | Status | Notes |
|-----------|-----------|------------------------------------------|
| OPTIONS | Complete | Advertises WebDAV capabilities |
| PROPFIND | Complete | Full directory listing with properties |
| GET | Complete | File download fully implemented |
| HEAD | Complete | Metadata retrieval implemented |
| PUT | Complete | File creation and update implemented |
| DELETE | Complete | Integration with trash features |
| MKCOL | Complete | Directory creation implemented |
| COPY | Complete | File/folder copying implemented |
| MOVE | Complete | File/folder moving/renaming implemented |
| PROPPATCH | Complete | Property updates implemented |
| LOCK | Complete | Basic locking capability implemented |
| UNLOCK | Complete | Basic unlocking capability implemented |
All WebDAV methods required by RFC 4918 are implemented. The server is compatible with all standard clients. Persistent property storage may be added later.
doc/wopi-integration.md
-1192
View File
File diff suppressed because it is too large Load Diff
doc/zip-and-compression.md
-101
View File
@@ -1,101 +0,0 @@
# 10 - ZIP and Compression
Two compression features exist: **ZIP download** (download folders as ZIP archives) and **Gzip compression** (transparent gzip for file content responses).
---
## ZIP Download
### Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **ZipPort** trait | `src/application/ports/zip_ports.rs` |
| Infrastructure | **ZipService** | `src/infrastructure/services/zip_service.rs` |
| Interfaces | `FolderHandler::download_folder_zip` | `src/interfaces/api/handlers/folder_handler.rs` |
### REST API
| Method | Path | Description |
|---|---|---|
| `GET` | `/api/folders/{id}/download` | Download folder as ZIP archive |
### Implementation
- Compression method: `Deflated` via the `zip` crate
- Traversal: iterative work-queue pattern (not recursive async) using a **PendingFolder** struct
- Cycle detection: `HashSet<String>` of processed folder IDs
- ZIP is built entirely in-memory (`ZipWriter<Cursor<Vec<u8>>>`)
- UNIX permissions: `0o755` for all entries
### Port Trait
```rust
#[async_trait]
pub trait ZipPort: Send + Sync + 'static {
async fn create_folder_zip(&self, folder_id: &str, folder_name: &str) -> Result<Vec<u8>, DomainError>;
}
```
### Example
```bash
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/folders/abc-123/download" \
-o my-folder.zip
```
---
## Gzip Compression
### Architecture
| Layer | Component | File |
|---|---|---|
| Application Port | **CompressionPort** trait | `src/application/ports/compression_ports.rs` |
| Infrastructure | **GzipCompressionService** | `src/infrastructure/services/compression_service.rs` |
### Configuration
| Constant | Value | Description |
|---|---|---|
| `COMPRESSION_SIZE_THRESHOLD` | 50 KB | Files below this are never compressed |
### Compression Levels
```rust
pub enum CompressionLevel {
None = 0,
Fast = 1,
Default = 6,
Best = 9,
}
```
### Port Trait
```rust
#[async_trait]
pub trait CompressionPort: Send + Sync + 'static {
async fn compress_data(&self, data: &[u8], level: CompressionLevel) -> Result<Vec<u8>, DomainError>;
async fn decompress_data(&self, compressed_data: &[u8]) -> Result<Vec<u8>, DomainError>;
fn should_compress(&self, mime_type: &str, size: u64) -> bool;
}
```
### Skip List
These MIME types are **never compressed** (already compressed or binary):
- `image/*` (except `svg`, `bmp`)
- `audio/*`, `video/*`
- `application/zip`, `application/gzip`, `application/x-compressed`
- `application/x-7z-compressed`, `application/x-rar-compressed`
- `application/x-bzip2`, `application/x-xz`
### Implementation Details
- Uses the `flate2` crate (`GzEncoder` / `GzDecoder`)
- Compress/decompress run inside `spawn_blocking` to avoid blocking the async runtime
- Optional **BufferPool** integration for buffer reuse
- Buffer pool estimates: 80% of input size for compression, 5x for decompression
docs/.vitepress/config.mts
+12 -59
View File
@@ -28,65 +28,6 @@ export default defineConfig({
label: "English",
lang: "en",
},
es: {
label: "Español",
lang: "es",
link: "/es/",
title: "OxiCloud",
description: "Almacenamiento en la nube autoalojado, calendario y contactos — increíblemente rápido",
themeConfig: {
nav: [
{ text: "Inicio", link: "/es/" },
{ text: "Guía", link: "/es/guide/" },
{ text: "Configuración", link: "/es/config/" },
{ text: "FAQ", link: "/es/faq" },
],
editLink: {
pattern: "https://github.com/DioCrafts/OxiCloud/tree/main/docs/:path",
text: "Editar esta página en GitHub",
},
sidebar: {
"/es/": [
{
text: "Introducción",
items: [
{ text: "¿Qué es OxiCloud?", link: "/es/guide/" },
{ text: "Inicio Rápido", link: "/es/guide/installation" },
],
},
{
text: "Configuración",
items: [
{ text: "Despliegue & Docker", link: "/es/config/deployment" },
{ text: "Variables de Entorno", link: "/es/config/env" },
{ text: "OIDC / SSO", link: "/es/config/oidc" },
{ text: "WOPI (Office)", link: "/es/config/wopi" },
],
},
{
text: "Características",
items: [
{ text: "WebDAV", link: "/es/guide/webdav" },
{ text: "CalDAV & CardDAV", link: "/es/guide/caldav-carddav" },
{ text: "Subida Chunked", link: "/es/guide/chunked-uploads" },
{ text: "Deduplicación", link: "/es/guide/deduplication" },
{ text: "Búsqueda", link: "/es/guide/search" },
{ text: "Papelera", link: "/es/guide/trash" },
],
},
{
text: "Arquitectura",
items: [
{ text: "Arquitectura Interna", link: "/es/architecture/" },
{ text: "Caché", link: "/es/architecture/caching" },
{ text: "Cuotas de Almacenamiento", link: "/es/architecture/storage-quotas" },
],
},
{ text: "FAQ", link: "/es/faq" },
],
},
},
},
},
head: [
@@ -126,7 +67,10 @@ export default defineConfig({
items: [
{ text: "Deployment & Docker", link: "/config/deployment" },
{ text: "Environment Variables", link: "/config/env" },
{ text: "Authentication", link: "/config/authentication" },
{ text: "OIDC / SSO", link: "/config/oidc" },
{ text: "OIDC Config Examples", link: "/config/oidc-config-examples" },
{ text: "Admin Settings", link: "/config/admin-settings" },
{ text: "WOPI (Office Editing)", link: "/config/wopi" },
],
},
@@ -135,10 +79,16 @@ export default defineConfig({
items: [
{ text: "WebDAV", link: "/guide/webdav" },
{ text: "CalDAV & CardDAV", link: "/guide/caldav-carddav" },
{ text: "DAV Client Setup", link: "/guide/dav-client-setup" },
{ text: "Chunked Uploads", link: "/guide/chunked-uploads" },
{ text: "Batch Operations", link: "/guide/batch-operations" },
{ text: "Deduplication", link: "/guide/deduplication" },
{ text: "Favorites & Recent", link: "/guide/favorites-and-recent" },
{ text: "Search", link: "/guide/search" },
{ text: "Thumbnails & Transcoding", link: "/guide/thumbnails-and-transcoding" },
{ text: "Trash & Recycle Bin", link: "/guide/trash" },
{ text: "ZIP & Compression", link: "/guide/zip-and-compression" },
{ text: "Internationalization", link: "/guide/i18n" },
],
},
{
@@ -146,6 +96,9 @@ export default defineConfig({
items: [
{ text: "Internal Architecture", link: "/architecture/" },
{ text: "Caching", link: "/architecture/caching" },
{ text: "Storage Safety", link: "/architecture/file-system-safety" },
{ text: "Database Transactions", link: "/architecture/database-transactions" },
{ text: "Share Integration", link: "/architecture/share-integration" },
{ text: "Storage Quotas", link: "/architecture/storage-quotas" },
],
},
docs/architecture/database-transactions.md
+74
View File
@@ -0,0 +1,74 @@
# Database Transactions
OxiCloud uses explicit PostgreSQL transactions for multi-step operations that must either commit together or fail together.
## ACID Guarantees
- Atomicity: all work succeeds or the entire transaction rolls back
- Consistency: constraints and invariants remain valid before and after commit
- Isolation: concurrent work behaves predictably
- Durability: committed writes survive process and system failures
## Transaction Helper
The PostgreSQL repositories use a helper like `with_transaction` to standardize the transaction lifecycle:
```rust
pub async fn with_transaction<F, T, E>(
pool: &Arc<PgPool>,
operation_name: &str,
operation: F,
) -> Result<T, E>
where
F: for<'c> FnOnce(&'c mut Transaction<'_, Postgres>) -> futures::future::BoxFuture<'c, Result<T, E>>,
E: From<SqlxError> + std::fmt::Display,
{ /* ... */ }
```
That wrapper handles begin, commit, rollback, and lifecycle logging so repository code can focus on the actual domain operation.
## Common Use Cases
### User management
Transactions keep related user changes together, such as creating a user and attaching the dependent records required for a valid account.
### Session management
Session creation and session revocation can update multiple tables in a single logical step, which avoids stale or mismatched security state.
### File and folder workflows
Moves, renames, trash operations, and other multi-step metadata changes rely on transactions so the tree stays consistent.
## Isolation Levels
OxiCloud can use different isolation levels depending on the operation.
| Level | Use case |
| --- | --- |
| `Read Committed` | Default application work |
| `Repeatable Read` | Stable reads during a longer unit of work |
| `Serializable` | Highest safety for conflict-prone critical operations |
Higher isolation can introduce retries or contention, so it should be reserved for the few flows that need it.
## Best Practices
- Keep transactions short
- Avoid heavy I/O inside a transaction when possible
- Group only operations that must commit together
- Choose the lowest isolation level that preserves correctness
- Log and surface rollback causes clearly
## Why It Matters
- Prevents partial metadata updates
- Keeps concurrent user activity predictable
- Reduces race conditions in critical operations
- Makes failures recoverable and easier to reason about
## Related Pages
- [Storage Safety](/architecture/file-system-safety)
- [Internal Architecture](/architecture/)
docs/architecture/file-system-safety.md
+77
View File
@@ -0,0 +1,77 @@
# Storage Safety
OxiCloud protects file integrity with two layers working together:
- PostgreSQL transactions for metadata
- Atomic blob writes for content
The result is a simple guarantee: operations either complete fully or fail cleanly enough to recover without corrupting user data.
## Storage Model
- Metadata such as names, folders, MIME types, quotas, and trash state lives in PostgreSQL
- File content is stored as content-addressed blobs under the storage backend
- Deduplication metadata is tracked separately so multiple files can reference the same content safely
## Metadata Safety
PostgreSQL protects metadata with ACID transactions.
- Single-row writes are atomic by default
- Multi-step operations use explicit transactions
- Foreign keys prevent orphaned references
- Unique constraints prevent illegal duplicates in the same scope
- Trash uses soft-delete semantics until permanent deletion is requested
## Content Safety
Blob writes rely on an atomic write pattern:
1. Write new content to a temporary file
2. `fsync` the file to push data and metadata to durable storage
3. Rename the temp file into its final content-addressed path
4. Sync parent directory metadata when needed
This prevents partially written blobs from appearing as valid stored content.
## Deduplication Notes
OxiCloud's deduplication pipeline uses BLAKE3 hashing and chunk manifest tracking. The storage layer can therefore reuse identical content while still keeping metadata changes transactional.
If content is stored successfully but the later metadata transaction fails, the content may remain as an unreferenced blob. That is a space leak, not a consistency leak, and can be cleaned up later.
## Upload Flow
```text
1. Receive content and spool it safely to storage
2. Finalize the content-addressed blob write
3. Begin metadata transaction
4. Insert or update file metadata in PostgreSQL
5. Commit
```
If step 1 or 2 fails, no metadata is committed. If step 4 or 5 fails, metadata rolls back and the storage layer can clean up unreferenced content later.
## Delete Flow
```text
1. Begin metadata transaction
2. Remove or soft-delete the metadata row
3. Commit
4. Decrement blob references and remove physical content when the refcount reaches zero
```
If the metadata transaction fails, the physical file is not considered deleted. If the refcount cleanup fails, the system may keep extra content on disk, but user-visible metadata remains correct.
## Why This Matters
- Crash resilience during uploads and deletes
- Safe recovery after power loss or host restarts
- Clean separation between metadata correctness and background storage cleanup
- Predictable behavior for trash, deduplication, and shared storage backends
## Related Pages
- [Internal Architecture](/architecture/)
- [Database Transactions](/architecture/database-transactions)
- [Storage Quotas](/architecture/storage-quotas)
docs/architecture/index.md
+11
View File
@@ -37,6 +37,17 @@ All cross-layer dependencies point **inward** via trait-based ports. The DI cont
8. **ZIP service** (last, depends on file & folder services)
9. **Assemble `AppState`**
## AppState Shape
The assembled `AppState` groups the application into a few stable buckets:
- `core` for cross-cutting runtime services such as path resolution, caching, chunked uploads, deduplication, compression, thumbnails, and ZIP handling
- `repositories` for PostgreSQL-backed folder, file, trash, and i18n persistence
- `applications` for the use-case layer exposed to handlers
- optional auth, admin, trash, share, favorites, recent, storage usage, calendar, and contact services when those features are enabled
This lets handlers depend on stable interfaces while the concrete implementation details stay inside the DI container.
## Project Structure
```
docs/architecture/share-integration.md
+74
View File
@@ -0,0 +1,74 @@
# Share Integration
OxiCloud supports public file and folder sharing through signed share links. A share can be public, password-protected, time-limited, or scoped by permissions.
## What a Share Contains
A share record tracks:
- The shared item ID and whether it is a file or folder
- A public token used in the share URL
- Optional password protection
- Optional expiration timestamp
- Permissions for read, write, and reshare
- The creator and access count
## Public and Private Routes
### Authenticated management routes
| Method | Path | Description |
| --- | --- | --- |
| `POST` | `/api/shares/` | Create a new share |
| `GET` | `/api/shares/` | List current user's shares |
| `GET` | `/api/shares/{id}` | Fetch one share |
| `PUT` | `/api/shares/{id}` | Update permissions, password, or expiration |
| `DELETE` | `/api/shares/{id}` | Delete a share |
### Public access routes
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/s/{token}` | Access a shared item |
| `POST` | `/api/s/{token}/verify` | Verify a password-protected share |
## Service Responsibilities
The share service is responsible for:
- Validating that the underlying file or folder exists
- Generating unique share IDs and public tokens
- Enforcing password checks and expiration rules
- Mapping domain permissions into API DTOs
- Recording access counts
Share metadata is persisted separately from the file content itself. The shared resource still uses the normal storage model for files and folders.
## Example Workflow
### Creating a share link
1. A user selects a file or folder in the UI
2. The frontend submits a request to `/api/shares/`
3. OxiCloud validates the target and requested permissions
4. The backend generates a token and public URL
5. The share metadata is saved and returned to the caller
### Opening a share link
1. A guest opens `/api/s/{token}`
2. OxiCloud verifies the token and checks expiration
3. If the share is password protected, the client verifies the password first
4. Access is counted and the shared resource is returned according to the granted permissions
## Security Notes
- Passwords are stored as hashes, never as plaintext
- Expired shares are rejected before content access
- Permissions are checked per action, not only when the share is created
## Related Pages
- [OIDC / SSO](/config/oidc)
- [Admin Settings](/config/admin-settings)
- [Internal Architecture](/architecture/)
docs/architecture/storage-quotas.md
+12
View File
@@ -15,6 +15,10 @@ OXICLOUD_ENABLE_USER_STORAGE_QUOTAS=true
3. If the upload would exceed the quota, it's rejected with a `413 Payload Too Large` error
4. Admins can view and set quotas via the admin panel or API
## Usage Calculation
The storage usage service recalculates logical usage from the user's home folder tree and sums file sizes recursively. Directory entries are skipped and the final value is written back to `auth.users.storage_used`.
## API
| Method | Endpoint | Description |
@@ -26,6 +30,14 @@ OXICLOUD_ENABLE_USER_STORAGE_QUOTAS=true
The admin panel (`/admin.html`) shows each user's current usage vs. quota with a visual progress bar.
The dashboard also exposes aggregate quota stats such as:
- total quota bytes
- total used bytes
- overall storage usage percent
- users above 80% usage
- users over quota
## Deduplication Interaction
Storage usage is calculated based on **logical file size** (what the user uploaded), not physical blob size. This means:
docs/config/admin-settings.md
+86
View File
@@ -0,0 +1,86 @@
# Admin Settings
OxiCloud exposes an admin API for runtime configuration, dashboard stats, and user administration. All routes live under `/api/admin` and require an authenticated admin JWT.
## Settings Endpoints
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/admin/settings/oidc` | Read current OIDC settings |
| `PUT` | `/api/admin/settings/oidc` | Save OIDC settings |
| `POST` | `/api/admin/settings/oidc/test` | Test provider connectivity |
| `GET` | `/api/admin/settings/general` | Read general server settings |
The OIDC runtime UI complements the base configuration described in [OIDC / SSO](/config/oidc) and the provider samples in [OIDC Config Examples](/config/oidc-config-examples).
## Dashboard Endpoint
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/admin/dashboard` | Read server statistics and feature state |
Typical dashboard fields include:
- server version
- whether auth and OIDC are enabled
- whether quotas are enabled
- total, active, and admin user counts
- quota usage totals and percentage
## User Management Endpoints
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/admin/users` | List users |
| `GET` | `/api/admin/users/{id}` | Get one user |
| `DELETE` | `/api/admin/users/{id}` | Delete a user |
| `PUT` | `/api/admin/users/{id}/role` | Change role |
| `PUT` | `/api/admin/users/{id}/active` | Activate or deactivate a user |
| `PUT` | `/api/admin/users/{id}/quota` | Update a storage quota |
### Built-in safety guards
- Admins cannot delete their own account
- Admins cannot change their own role
- Admins cannot deactivate themselves
## OIDC Settings Priority
When the same setting exists in multiple places, OxiCloud resolves it in this order:
1. Environment variables such as `OXICLOUD_OIDC_*`
2. Values stored in the admin settings table
3. Built-in defaults
If a value is overridden by environment variables, the admin API can expose that in the response so operators know why a saved value is not taking effect.
## Test Connection Example
```json
{
"issuer_url": "https://keycloak.example.com/realms/main"
}
```
Successful responses include discovered endpoints such as the authorization endpoint, token endpoint, and userinfo endpoint.
## Data Storage
Runtime settings are stored in `auth.admin_settings`.
```sql
CREATE TABLE IF NOT EXISTS auth.admin_settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
category TEXT NOT NULL,
is_secret BOOLEAN DEFAULT FALSE,
updated_by VARCHAR(36),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);
```
## Related Pages
- [OIDC / SSO](/config/oidc)
- [OIDC Config Examples](/config/oidc-config-examples)
- [Environment Variables](/config/env)
docs/config/authentication.md
+72
View File
@@ -0,0 +1,72 @@
# Authentication
OxiCloud ships with JWT-based authentication and Argon2id password hashing for local accounts. It also exposes status and OIDC-related auth endpoints under the same `/api/auth` namespace.
## Core Endpoints
| Method | Endpoint | Description |
| --- | --- | --- |
| `POST` | `/api/auth/register` | Create a local user account |
| `POST` | `/api/auth/login` | Exchange username and password for access and refresh tokens |
| `POST` | `/api/auth/refresh` | Refresh the session tokens |
| `GET` | `/api/auth/me` | Return the current authenticated user |
| `PUT` | `/api/auth/change-password` | Change the current user's password |
| `POST` | `/api/auth/logout` | Invalidate the current session |
| `GET` | `/api/auth/status` | Return auth system state, including OIDC availability |
## OIDC Endpoints Under Auth
| Method | Endpoint | Description |
| --- | --- | --- |
| `GET` | `/api/auth/oidc/providers` | List configured OIDC provider info |
| `GET` | `/api/auth/oidc/authorize` | Build the authorization redirect URL |
| `GET` | `/api/auth/oidc/callback` | Handle provider redirect callback |
| `POST` | `/api/auth/oidc/exchange` | Exchange the auth code for OxiCloud session tokens |
## Example Flows
### Register
```json
{
"username": "testuser",
"email": "test@example.com",
"password": "SecurePassword123"
}
```
### Login
```json
{
"username": "testuser",
"password": "SecurePassword123"
}
```
Typical successful login response:
```json
{
"accessToken": "...",
"refreshToken": "...",
"expiresIn": 3600
}
```
### Current User
`GET /api/auth/me` returns the authenticated user's identity, role, and storage information.
## Security Model
- local passwords are hashed with Argon2id
- access control is role-based (`admin` and `user`)
- refresh tokens support session renewal without forcing frequent re-login
- OIDC can coexist with local auth or disable password login entirely
## Related Pages
- [OIDC / SSO](/config/oidc)
- [Admin Settings](/config/admin-settings)
- [Environment Variables](/config/env)
docs/config/deployment.md
+21 -5
View File
@@ -16,14 +16,18 @@ The final image runs as non-root user `oxicloud` (UID/GID 1001). Exposed port: *
```yaml
services:
postgres:
image: postgres:17.4-alpine
image: postgres:18.2-alpine3.23
restart: always
environment:
POSTGRES_DB: oxicloud
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres # change in production!
POSTGRES_PASSWORD: postgres
ports:
- "5432:5432"
networks:
- oxicloud
volumes:
- pg_data:/var/lib/postgresql/data
- ./db/schema.sql:/docker-entrypoint-initdb.d/schema.sql
- pg_data:/var/lib/postgresql/
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
@@ -31,9 +35,15 @@ services:
retries: 5
oxicloud:
image: ghcr.io/diocrafts/oxicloud:latest
image: diocrafts/oxicloud:latest
restart: always
build:
context: .
dockerfile: Dockerfile
ports:
- "8086:8086"
networks:
- oxicloud
env_file:
- .env
volumes:
@@ -42,11 +52,17 @@ services:
postgres:
condition: service_healthy
networks:
oxicloud:
driver: bridge
volumes:
pg_data:
storage_data:
```
This example mirrors the repository's current `docker-compose.yml`. If you deploy from a registry-only setup, you can keep the `image:` line and remove the `build:` stanza.
## Kubernetes (Helm)
### Prerequisites
docs/config/env.md
+18 -1
View File
@@ -1,6 +1,6 @@
# Environment Variables
All variables use the `OXICLOUD_` prefix.
Most runtime variables use the `OXICLOUD_` prefix. A few build-time or allocator variables do not.
## Server
@@ -19,6 +19,14 @@ All variables use the `OXICLOUD_` prefix.
| `OXICLOUD_DB_CONNECTION_STRING` | `postgres://postgres:postgres@localhost:5432/oxicloud` | PostgreSQL connection string |
| `OXICLOUD_DB_MAX_CONNECTIONS` | `20` | Max pool connections |
| `OXICLOUD_DB_MIN_CONNECTIONS` | `5` | Min pool connections |
| `OXICLOUD_DB_MAINTENANCE_MAX_CONNECTIONS` | `5` | Max connections in the isolated maintenance pool |
| `OXICLOUD_DB_MAINTENANCE_MIN_CONNECTIONS` | `1` | Min connections in the isolated maintenance pool |
## Build-Time SQLx
| Variable | Default | Description |
|---|---|---|
| `DATABASE_URL` | — | Build-time database URL for SQLx compile-time checks |
## Authentication
@@ -68,6 +76,15 @@ See the [WOPI configuration guide](/config/wopi) for details.
| `OXICLOUD_WOPI_TOKEN_TTL_SECS` | `86400` | Token lifetime |
| `OXICLOUD_WOPI_LOCK_TTL_SECS` | `1800` | Lock expiration |
## Allocator Tuning
These variables are read directly by **mimalloc**, not by OxiCloud's config parser.
| Variable | Default | Description |
|---|---|---|
| `MIMALLOC_PURGE_DELAY` | `0` | Delay in ms before freed memory is returned to the OS |
| `MIMALLOC_ALLOW_LARGE_OS_PAGES` | `0` | Enable or disable large OS pages for allocations |
## Internal Defaults (not configurable via env)
| Parameter | Default |
docs/config/index.md
+1
View File
@@ -6,6 +6,7 @@ OxiCloud is configured entirely via **environment variables** (no config files n
- [Deployment & Docker](/config/deployment) — Docker Compose, Kubernetes Helm chart, image details
- [Environment Variables](/config/env) — complete reference of all `OXICLOUD_*` variables
- [Authentication](/config/authentication) — JWT auth, login, refresh, password changes, and auth status
- [OIDC / SSO](/config/oidc) — single sign-on with Keycloak, Authentik, Authelia, Google, Azure AD
- [WOPI (Office Editing)](/config/wopi) — Collabora Online / OnlyOffice integration
docs/config/oidc-config-examples.md
+107
View File
@@ -0,0 +1,107 @@
# OIDC Config Examples
This page collects provider-specific OpenID Connect examples for OxiCloud. Use it together with the base reference in [OIDC / SSO](/config/oidc).
## Base Environment Variables
```bash
OXICLOUD_OIDC_ENABLED=true
OXICLOUD_OIDC_PROVIDER_NAME="Display Name"
OXICLOUD_OIDC_ISSUER_URL="https://provider.example.com/realms/your-realm"
OXICLOUD_OIDC_CLIENT_ID="your-client-id"
OXICLOUD_OIDC_CLIENT_SECRET="your-client-secret"
OXICLOUD_OIDC_REDIRECT_URI="https://your-oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES="openid profile email"
OXICLOUD_OIDC_FRONTEND_URL="https://your-oxicloud.example.com"
OXICLOUD_OIDC_AUTO_PROVISION="true"
OXICLOUD_OIDC_ADMIN_GROUPS="admin-group"
OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN="false"
```
## Authentik
1. Create an OAuth2 or OpenID Connect application for OxiCloud
2. Register `https://your-oxicloud.example.com/api/auth/oidc/callback` as the callback URL
3. Copy the generated client ID and client secret
```yaml
services:
oxicloud:
image: diocrafts/oxicloud:latest
environment:
OXICLOUD_OIDC_ENABLED: "true"
OXICLOUD_OIDC_PROVIDER_NAME: "Authentik"
OXICLOUD_OIDC_ISSUER_URL: "https://authentik.example.com/application/o/oxicloud"
OXICLOUD_OIDC_CLIENT_ID: "your-authentik-client-id"
OXICLOUD_OIDC_CLIENT_SECRET: "your-authentik-client-secret"
OXICLOUD_OIDC_REDIRECT_URI: "https://oxicloud.example.com/api/auth/oidc/callback"
OXICLOUD_OIDC_SCOPES: "openid profile email"
OXICLOUD_OIDC_FRONTEND_URL: "https://oxicloud.example.com"
```
## Authelia
Configure an OIDC client in Authelia and allow OxiCloud's redirect URI.
```yaml
identity_providers:
oidc:
clients:
- id: oxicloud
description: OxiCloud
public: false
redirect_uris:
- https://oxicloud.example.com/api/auth/oidc/callback
scopes: [openid, profile, email, groups]
```
```yaml
services:
oxicloud:
image: diocrafts/oxicloud:latest
environment:
OXICLOUD_OIDC_PROVIDER_NAME: "Authelia"
OXICLOUD_OIDC_ISSUER_URL: "https://authelia.example.com"
OXICLOUD_OIDC_CLIENT_ID: "oxicloud"
OXICLOUD_OIDC_CLIENT_SECRET: "your-client-secret"
```
## Keycloak
1. Create a confidential client named `oxicloud`
2. Set the valid redirect URI to `https://oxicloud.example.com/api/auth/oidc/callback`
3. Copy the generated client secret
```yaml
services:
oxicloud:
image: diocrafts/oxicloud:latest
environment:
OXICLOUD_OIDC_PROVIDER_NAME: "Keycloak"
OXICLOUD_OIDC_ISSUER_URL: "https://keycloak.example.com/realms/your-realm"
OXICLOUD_OIDC_CLIENT_ID: "oxicloud"
OXICLOUD_OIDC_CLIENT_SECRET: "your-keycloak-client-secret"
OXICLOUD_OIDC_ADMIN_GROUPS: "oxicloud-admins"
```
## Troubleshooting
### Failed to discover the provider
- Verify the issuer URL exactly matches the provider's discovery URL base
- Check DNS, TLS, and network reachability from the OxiCloud container or host
### Invalid redirect URI
- Make the configured callback match exactly on scheme, host, port, and path
- Check for `http` versus `https` mismatches
### Auto-provisioning problems
- Enable `OXICLOUD_OIDC_AUTO_PROVISION=true` if first-login account creation is expected
- Make sure `openid` is included in the configured scopes
## Related Pages
- [OIDC / SSO](/config/oidc)
- [Admin Settings](/config/admin-settings)
docs/config/oidc.md
+19
View File
@@ -10,6 +10,16 @@ OxiCloud supports OpenID Connect for single sign-on with providers like **Keyclo
4. IdP redirects back to OxiCloud with an auth code
5. OxiCloud exchanges the code for user info and issues its own JWT tokens
## Architecture
OIDC follows the Authorization Code Flow and keeps a clear split between provider communication and local session handling.
- `OidcService` discovers provider metadata, builds authorization URLs, exchanges authorization codes, and validates the token response
- `AuthApplicationService` coordinates user lookup or auto-provisioning and then issues OxiCloud's own access and refresh tokens
- the auth handler exposes the public OIDC endpoints under `/api/auth/oidc/*`
After the browser returns from the IdP, OxiCloud does not reuse the provider token for app requests. It converts the identity into its own JWT session model.
## Configuration
```bash
@@ -54,6 +64,15 @@ If `OXICLOUD_OIDC_ENABLED=true` but `issuer_url`, `client_id`, or `client_secret
| GET | `/api/auth/oidc/callback` | Callback from IdP with auth code |
| POST | `/api/auth/oidc/exchange` | Exchange auth code for JWT tokens |
## Identity Mapping
OIDC users are matched by the pair:
- `oidc_provider`
- `oidc_subject`
This allows one external identity to map to one local user record and supports just-in-time provisioning when `OXICLOUD_OIDC_AUTO_PROVISION=true`.
## Provider Examples
### Keycloak
docs/config/wopi.md
+28 -1
View File
@@ -9,6 +9,15 @@ OxiCloud integrates with **Collabora Online** and **OnlyOffice** via the WOPI pr
3. The editor fetches the file from OxiCloud via WOPI endpoints
4. Edits are saved back via `PutFile`
## Host / Client Flow
OxiCloud acts as the **WOPI host** and Collabora or OnlyOffice acts as the **WOPI client**.
1. OxiCloud loads the discovery document from the configured WOPI client
2. The frontend opens a host page that embeds the editor in an iframe
3. The iframe URL includes `WOPISrc`, which points back to OxiCloud's `/wopi/files/*` endpoints
4. The editor calls back into OxiCloud with the WOPI access token to read, lock, and save the file
## Configuration
```bash
@@ -68,7 +77,25 @@ services:
| GET | `/wopi/files/{id}` | CheckFileInfo — file metadata |
| GET | `/wopi/files/{id}/contents` | GetFile — download file content |
| POST | `/wopi/files/{id}/contents` | PutFile — save edited content |
| POST | `/wopi/files/{id}` | Lock / Unlock / RefreshLock |
| POST | `/wopi/files/{id}` | Lock / Unlock / RefreshLock / Rename / Delete / Save As |
### Common `X-WOPI-Override` operations
| Override | Purpose |
| --- | --- |
| `LOCK` | Acquire or refresh an editor lock |
| `UNLOCK` | Release a lock |
| `REFRESH_LOCK` | Extend the current lock |
| `PUT_RELATIVE` | Save as a related file |
| `RENAME_FILE` | Rename from inside the editor |
| `DELETE` | Delete from the editor when supported |
## Notes
- `CheckFileInfo` is required for every WOPI action
- `PutFile` uploads the full file content back to OxiCloud
- lock conflicts return `409 Conflict` with the current lock value
- OxiCloud uses query parameter `?access_token=` authentication for WOPI callbacks instead of the standard JWT middleware
## Supported Formats
docs/guide/batch-operations.md
+120
View File
@@ -0,0 +1,120 @@
# Batch Operations
OxiCloud exposes batch endpoints for bulk file and folder operations under `/api/batch`. Batch requests reduce round-trips, run concurrently behind a semaphore, and return per-item success and failure details instead of aborting on the first error.
## What You Can Do
### File operations
| Method | Path | Description |
| --- | --- | --- |
| `POST` | `/api/batch/files/move` | Move multiple files into a target folder |
| `POST` | `/api/batch/files/copy` | Copy multiple files into a target folder |
| `POST` | `/api/batch/files/delete` | Delete multiple files |
| `POST` | `/api/batch/files/get` | Fetch metadata for multiple files |
### Folder operations
| Method | Path | Description |
| --- | --- | --- |
| `POST` | `/api/batch/folders/delete` | Delete multiple folders |
| `POST` | `/api/batch/folders/create` | Create multiple folders |
| `POST` | `/api/batch/folders/get` | Fetch metadata for multiple folders |
| `POST` | `/api/batch/folders/move` | Move multiple folders |
### Additional batch endpoints
| Method | Path | Description |
| --- | --- | --- |
| `POST` | `/api/batch/trash` | Trash multiple items in one request |
| `POST` | `/api/batch/download` | Build a batch download |
## Request Shapes
### File move or copy
```json
{
"file_ids": ["id-1", "id-2", "id-3"],
"target_folder_id": "folder-abc"
}
```
### Folder delete
```json
{
"folder_ids": ["folder-1", "folder-2"],
"recursive": true
}
```
### Folder create
```json
{
"folders": [
{ "name": "Documents", "parent_id": null },
{ "name": "Photos", "parent_id": "folder-abc" }
]
}
```
## Response Format
All batch endpoints return the same envelope:
```json
{
"successful": [
{ "id": "id-1" }
],
"failed": [
{ "id": "bad-id", "error": "File not found" }
],
"stats": {
"total": 5,
"successful": 4,
"failed": 1,
"execution_time_ms": 245
}
}
```
### Status codes
| Code | Meaning |
| --- | --- |
| `200 OK` or `201 Created` | Every operation succeeded |
| `206 Partial Content` | Some operations succeeded and some failed |
| `400 Bad Request` | Every operation failed |
## Concurrency Model
Batch work is coordinated by `BatchOperationService` and a `tokio::sync::Semaphore`. By default, OxiCloud caps concurrent work with `max_concurrent_files = 10` so large batches do not starve the rest of the application.
Individual failures are collected in the `failed` array. One bad item does not cancel the whole request unless the batch cannot start at all.
## Example
```bash
# Move three files into a folder
curl -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"file_ids":["id-1","id-2","id-3"],"target_folder_id":"folder-abc"}' \
"https://oxicloud.example.com/api/batch/files/move"
# Delete multiple folders recursively
curl -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"folder_ids":["old-1","old-2"],"recursive":true}' \
"https://oxicloud.example.com/api/batch/folders/delete"
```
## Related Pages
- [Search](/guide/search)
- [Trash & Recycle Bin](/guide/trash)
- [ZIP and Compression](/guide/zip-and-compression)
docs/guide/caldav-carddav.md
+56
View File
@@ -16,6 +16,33 @@ https://your-server:8086/caldav/
- RFC 5545 (iCalendar format)
- DAV capabilities: `1, 2, calendar-access`
### Route Structure
CalDAV is mounted at the top level, not under `/api`:
- `/caldav`
- `/caldav/`
- `/caldav/{*path}`
OxiCloud also exposes `/.well-known/caldav` and redirects it to `/caldav/`.
Typical resource shapes:
- `/caldav/` for the calendar home
- `/caldav/{calendar_id}/` for one calendar
- `/caldav/{calendar_id}/{ical_uid}.ics` for one event
### Supported Methods
- `OPTIONS`
- `PROPFIND`
- `REPORT`
- `MKCALENDAR`
- `PUT`
- `GET`
- `DELETE`
- `PROPPATCH`
### Client Setup
| Client | URL |
@@ -48,6 +75,31 @@ https://your-server:8086/carddav/
- RFC 6352 (CardDAV)
- RFC 6350 (vCard 4.0)
### Route Structure
CardDAV is also mounted at the top level:
- `/carddav`
- `/carddav/`
- `/carddav/{*path}`
Typical resource shapes:
- `/carddav/` for the address book home
- `/carddav/{addressBookId}/` for one address book
- `/carddav/{addressBookId}/{contactId}.vcf` for one contact
### Supported Methods
- `OPTIONS`
- `PROPFIND`
- `REPORT`
- `MKCOL`
- `PUT`
- `GET`
- `DELETE`
- `PROPPATCH`
### Client Setup
| Client | URL |
@@ -68,3 +120,7 @@ https://your-server:8086/carddav/
::: info
DAVx⁵ file sync works. CalDAV/CardDAV support on DAVx⁵ is still being refined.
:::
## Client Setup
For platform-specific instructions, see [DAV Client Setup](/guide/dav-client-setup).
docs/guide/chunked-uploads.md
+82 -38
View File
@@ -1,63 +1,107 @@
# Chunked Uploads
OxiCloud supports TUS-like chunked uploads for large files. Uploads are parallel, resumable, and have MD5 integrity checks.
OxiCloud exposes resumable chunked uploads under `/api/uploads`. The protocol is TUS-like in spirit, but the concrete API is OxiCloud-specific: create a session, stream chunks with `PATCH`, inspect progress with `HEAD`, then finalize the assembled file.
## How It Works
## Upload Flow
1. Client sends `POST /api/files/upload/init` with file metadata → receives an `upload_id`
2. Client splits the file into chunks and uploads them in parallel via `POST /api/files/upload/chunk`
3. Each chunk includes its index, MD5 hash, and the `upload_id`
4. When all chunks are uploaded, client calls `POST /api/files/upload/complete`
5. Server reassembles the file, verifies integrity, and runs deduplication
1. Create an upload session with `POST /api/uploads`
2. Upload each chunk with `PATCH /api/uploads/{upload_id}?chunk_index=N`
3. Optionally inspect progress with `HEAD /api/uploads/{upload_id}`
4. Finalize with `POST /api/uploads/{upload_id}/complete`
5. Cancel an in-flight upload with `DELETE /api/uploads/{upload_id}` if needed
## API Endpoints
### Initialize Upload
### Create upload session
```http
POST /api/files/upload/init
POST /api/uploads
Content-Type: application/json
{
"file_name": "large-video.mp4",
"filename": "large-video.mp4",
"folder_id": "folder-uuid",
"content_type": "video/mp4",
"total_size": 524288000,
"chunk_size": 8388608,
"total_chunks": 63
"chunk_size": 8388608
}
```
### Upload Chunk
```http
POST /api/files/upload/chunk
Content-Type: multipart/form-data
upload_id: "uuid"
chunk_index: 0
chunk_hash: "md5-hex"
file: <binary>
```
### Complete Upload
```http
POST /api/files/upload/complete
Content-Type: application/json
Typical response:
```json
{
"upload_id": "uuid"
"upload_id": "uuid",
"chunk_size": 8388608,
"total_chunks": 63,
"expires_at": 86400
}
```
## Configuration
### Upload a chunk
| Parameter | Default | Description |
|---|---|---|
| Max parallel chunks | 8 | Concurrent chunk uploads |
| Min size for chunking | 200 MB | Below this, single-shot upload is used |
| Chunk size | 8 MB | Default chunk size |
Chunks are sent as raw bytes, not multipart form uploads.
## Frontend Behaviour
```http
PATCH /api/uploads/{upload_id}?chunk_index=0&checksum=md5-hex
Content-Type: application/octet-stream
Content-MD5: md5-hex
The OxiCloud web UI automatically selects chunked upload for large files. A progress bar shows overall completion and current chunk status.
<binary chunk bytes>
```
Notes:
- `chunk_index` is required and zero-based
- `checksum` is optional and can also be supplied with the `Content-MD5` header
- Successful responses include progress headers such as `Upload-Offset`, `Upload-Progress`, and `Upload-Complete`
### Inspect upload status
```http
HEAD /api/uploads/{upload_id}
```
The response includes upload metadata in headers such as:
- `Upload-Offset`
- `Upload-Length`
- `Upload-Progress`
- `Upload-Chunks-Total`
- `Upload-Chunks-Complete`
### Finalize upload
```http
POST /api/uploads/{upload_id}/complete
```
Successful responses return the created file metadata:
```json
{
"file_id": "uuid",
"filename": "large-video.mp4",
"size": 524288000,
"path": "/Videos/large-video.mp4"
}
```
### Cancel upload
```http
DELETE /api/uploads/{upload_id}
```
This removes the in-progress session and temporary chunk data.
## Validation Rules
- `filename` is required
- `total_size` must be greater than zero
- `chunk_size` must be at least 1 MB when provided
- Storage quota checks can reject the session before upload starts
## Frontend Behavior
The OxiCloud web UI can switch to chunked uploads for larger files, track aggregate progress, and retry individual chunks without restarting the full transfer.
docs/guide/dav-client-setup.md
+101
View File
@@ -0,0 +1,101 @@
# DAV Client Setup
This page collects platform-specific connection steps for OxiCloud's WebDAV, CalDAV, and CardDAV endpoints.
## Connection Summary
| Use case | URL |
| --- | --- |
| WebDAV file access | `https://your-oxicloud-server/webdav/` |
| CalDAV calendar sync | `https://your-oxicloud-server/caldav` |
| CardDAV contact sync | `https://your-oxicloud-server/carddav` |
## WebDAV
### Windows Explorer
1. Open File Explorer
2. Right-click This PC and choose Add a network location or Map network drive
3. Enter `https://your-oxicloud-server/webdav/`
4. Provide your OxiCloud username and password
If Windows refuses the connection, check the `WebClient` service and verify these registry values under `HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters`:
- `BasicAuthLevel = 2` when Basic auth is required
- `FileSizeLimitInBytes` if you need to allow larger transfers
### macOS Finder
1. Open Finder
2. Choose Go -> Connect to Server or press Cmd+K
3. Enter `https://your-oxicloud-server/webdav/`
4. Sign in with your OxiCloud credentials
### Linux
- GNOME Files: use `davs://your-oxicloud-server/webdav/`
- KDE Dolphin: use `webdavs://your-oxicloud-server/webdav/`
- `davfs2`: mount `https://your-oxicloud-server/webdav/` to a local directory
## CalDAV
### Apple Calendar
Use an advanced CalDAV account and point it at `https://your-oxicloud-server/caldav`.
### Thunderbird
Create a network calendar and use a CalDAV location such as:
```text
https://your-oxicloud-server/caldav/calendars/your-calendar-id
```
### Android with DAVx5
Use Login with URL and username, then point the base URL at `https://your-oxicloud-server/caldav`.
### Outlook on Windows
Use a CalDAV plugin such as CalDAV Synchronizer and register the calendar endpoint explicitly.
## CardDAV
### Apple Contacts
Create a CardDAV account using `https://your-oxicloud-server/carddav`.
### Thunderbird
Use a remote address book with a URL such as:
```text
https://your-oxicloud-server/carddav/address-books/your-address-book-id
```
### Android with DAVx5
Use the CardDAV base URL `https://your-oxicloud-server/carddav`.
### Outlook on Windows
Use a CardDAV-capable synchronizer and configure the remote address book endpoint explicitly.
## Troubleshooting
### WebDAV
- Make sure the URL includes `/webdav/`
- Use HTTPS in production
- Recheck credentials and the WebClient service on Windows
### CalDAV and CardDAV
- Use the full `/caldav` or `/carddav` base path
- Verify the calendar or address book identifier when the client asks for one
- If sync works on one client and not another, compare the exact URLs being used
## Related Pages
- [WebDAV](/guide/webdav)
- [CalDAV & CardDAV](/guide/caldav-carddav)
docs/guide/deduplication.md
+50 -31
View File
@@ -1,46 +1,65 @@
# File Deduplication
OxiCloud uses **SHA-256 content-addressable storage** to avoid storing duplicate files. If two users upload the same file, only one copy is stored on disk.
OxiCloud uses **content-defined chunking (CDC)** with **FastCDC** and **BLAKE3** to deduplicate files at the sub-file level. Instead of storing only whole-file blobs, OxiCloud can split a file into variable-size chunks, reuse unchanged chunks across versions, and track the ordered chunk list in PostgreSQL.
## How It Works
1. When a file is uploaded, its SHA-256 hash is computed
2. The hash is checked against the blob store (`.blobs/{prefix}/{hash}.blob`)
3. If a blob with that hash already exists, the file metadata points to the existing blob (no extra disk usage)
4. If not, the content is saved as a new blob
5. A reference counter tracks how many files point to each blob
1. OxiCloud analyzes the uploaded file with FastCDC
2. The file is split into variable-size chunks from **64 KB** to **1 MB**, targeting an average of **256 KB**
3. Each chunk is hashed with **BLAKE3** and checked against the blob index
4. Only new chunks are written to the blob backend
5. A manifest in PostgreSQL maps the whole-file hash to the ordered chunk hash list
6. Reference counts are updated so identical chunks are stored once even across multiple files or edited versions
## Automatic Cleanup
## Storage Model
```text
storage.files -> metadata rows that reference content
chunk_manifests -> file_hash -> [chunk_hashes] + chunk_sizes + ref_count
storage.blobs -> per-chunk blob metadata and reference counts
blob backend -> actual chunk bytes on disk or remote storage
```
The manifest table is created in `migrations/20260414000000_chunk_manifests.sql` and keeps:
- `file_hash`
- ordered `chunk_hashes`
- `chunk_sizes`
- `total_size`
- `chunk_count`
- `ref_count`
## Why CDC Matters
Whole-file dedup only helps when two files are byte-for-byte identical. CDC helps when files are similar but not identical, for example:
- edited office documents
- versioned project archives
- large media files with partial changes
In those cases, unchanged chunks can be reused and only the modified portions need new storage.
## Backward Compatibility
Older uploads stored before CDC are still readable. When OxiCloud does not find a matching manifest row, it falls back to legacy whole-file blob reads.
## Cleanup Behavior
When a file is permanently deleted:
1. The blob's reference count is decremented
2. If the reference count reaches zero, the blob is removed from disk
1. OxiCloud decrements the manifest reference count
2. If the last manifest reference disappears, chunk refcounts are decremented
3. Chunks with `ref_count = 0` are removed from the blob index and then deleted from the backend
This means disk space is only freed when the **last** reference to a blob is removed.
## Storage Layout
```
storage/
├── .blobs/
│ ├── a1/
│ │ └── a1b2c3d4...sha256.blob
│ ├── f8/
│ │ └── f8e7d6c5...sha256.blob
│ └── ...
```
The first two hex characters of the hash are used as a directory prefix to avoid having millions of files in a single directory.
This keeps storage correct even when multiple files share the same chunk set.
## Benefits
- **Disk savings** — identical files across users consume storage only once
- **Instant uploads** — if the blob already exists, the upload completes immediately
- **Integrity** — SHA-256 ensures bit-for-bit correctness
- Better storage savings for edited and versioned files
- Faster repeat uploads when many chunks already exist
- BLAKE3 hashing for fast content verification
- PostgreSQL-backed manifests for durable indexing and cleanup
## Limitations
## Related Endpoints
- Deduplication is based on exact content match (byte-identical files)
- Near-duplicate files (e.g., a JPEG re-saved at slightly different quality) are stored separately
- Encryption at rest would require per-user keys, which breaks deduplication (planned as opt-in)
The dedup subsystem is also exposed through helper endpoints under `/api/dedup` for hash checks, deduplicated uploads, statistics, and maintenance operations.
docs/guide/favorites-and-recent.md
+100
View File
@@ -0,0 +1,100 @@
# Favorites and Recent Items
OxiCloud includes two per-user tracking features backed by PostgreSQL:
- Favorites for pinning files and folders you want to reach quickly
- Recent items for tracking the files and folders you accessed most recently
Both features are enabled when the instance has a database connection.
## Favorites
### API
All routes live under `/api/favorites` and require authentication.
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/favorites/` | List all favorites for the current user |
| `POST` | `/api/favorites/{item_type}/{item_id}` | Add a file or folder to favorites |
| `DELETE` | `/api/favorites/{item_type}/{item_id}` | Remove a favorite |
`item_type` must be either `file` or `folder`.
### Behavior
- Adding the same item twice is idempotent
- Results are ordered by `created_at DESC`
- User identity comes from the JWT, not the request body
### Storage model
Favorites are stored in `auth.user_favorites` with a uniqueness constraint on `(user_id, item_id, item_type)`.
```sql
CREATE TABLE IF NOT EXISTS auth.user_favorites (
id SERIAL PRIMARY KEY,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
item_id TEXT NOT NULL,
item_type TEXT NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, item_id, item_type)
);
```
## Recent Items
### API
All routes live under `/api/recent` and require authentication.
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/recent/` | List recent items, optionally with `?limit=N` |
| `POST` | `/api/recent/{item_type}/{item_id}` | Record an access |
| `DELETE` | `/api/recent/{item_type}/{item_id}` | Remove one item from history |
| `DELETE` | `/api/recent/clear` | Clear all recent items |
### Behavior
- Default maximum per user: 50 items
- Re-accessing an item updates its `accessed_at` timestamp
- Old items are automatically pruned after inserts
- Results are ordered by `accessed_at DESC`
### Storage model
Recent items are stored in `auth.user_recent_files`.
```sql
CREATE TABLE IF NOT EXISTS auth.user_recent_files (
id SERIAL PRIMARY KEY,
user_id VARCHAR(36) NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
item_id TEXT NOT NULL,
item_type TEXT NOT NULL,
accessed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, item_id, item_type)
);
```
## Example
```bash
# Add a file to favorites
curl -X POST -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/favorites/file/abc-123"
# List recent items
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/recent/?limit=10"
# Clear recent history
curl -X DELETE -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/recent/clear"
```
## Related Pages
- [Search](/guide/search)
- [Trash & Recycle Bin](/guide/trash)
- [Batch Operations](/guide/batch-operations)
docs/guide/i18n.md
+65
View File
@@ -0,0 +1,65 @@
# Internationalization
OxiCloud exposes a public translation API backed by JSON locale files on disk. Locales are loaded lazily, cached in memory, and served without authentication.
## Public API
All routes live under `/api/i18n`.
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/i18n/locales` | List available locales |
| `GET` | `/api/i18n/translate?key=...&locale=...` | Resolve a single key |
| `GET` | `/api/i18n/locales/{locale_code}` | Fetch all translations for one locale |
## Locale Files
Translations are stored as nested JSON files under `static/locales/`.
Example shape:
```json
{
"app": {
"title": "OxiCloud"
},
"nav": {
"files": "Files",
"trash": "Trash"
}
}
```
Keys are resolved with dot notation, so `nav.files` maps to `Files`.
## Fallback Rules
- If a key is missing in the requested locale, OxiCloud falls back to English
- If the key is missing there as well, the API returns a not-found error for that key
## Caching Model
Translations are cached in memory with an `RwLock<HashMap<Locale, serde_json::Value>>` and loaded on first use for each locale.
## Frontend Usage
Typical frontend flow:
1. Detect the preferred locale
2. Request `/api/i18n/locales/{code}`
3. Apply translated strings to UI elements
## Example
```bash
# List locales
curl "https://oxicloud.example.com/api/i18n/locales"
# Fetch one translation
curl "https://oxicloud.example.com/api/i18n/translate?key=app.title&locale=es"
```
## Related Pages
- [Internal Architecture](/architecture/)
- [Environment Variables](/config/env)
docs/guide/installation.md
+27 -16
View File
@@ -3,8 +3,8 @@
## Docker (recommended)
```bash
git clone https://github.com/DioCrafts/oxicloud.git
cd oxicloud
git clone https://github.com/DioCrafts/OxiCloud.git
cd OxiCloud
cp example.env .env
docker compose up -d
```
@@ -16,14 +16,15 @@ Open **http://localhost:8086**. That's it.
```yaml
services:
postgres:
image: postgres:17.4-alpine
image: postgres:18.2-alpine3.23
environment:
POSTGRES_DB: oxicloud
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- "5432:5432"
volumes:
- pg_data:/var/lib/postgresql/data
- ./db/schema.sql:/docker-entrypoint-initdb.d/schema.sql
- pg_data:/var/lib/postgresql/
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
@@ -32,6 +33,9 @@ services:
oxicloud:
image: diocrafts/oxicloud:latest
build:
context: .
dockerfile: Dockerfile
ports:
- "8086:8086"
env_file:
@@ -52,13 +56,19 @@ volumes:
Requires **Rust 1.93+** and **PostgreSQL 13+**.
```bash
git clone https://github.com/DioCrafts/oxicloud.git
cd oxicloud
echo "DATABASE_URL=postgres://user:pass@localhost/oxicloud" > .env
git clone https://github.com/DioCrafts/OxiCloud.git
cd OxiCloud
cp example.env .env
# Edit .env and set OXICLOUD_DB_CONNECTION_STRING for runtime
export DATABASE_URL=postgres://user:pass@localhost:5432/oxicloud
cargo build --release
cargo run --release
```
`OXICLOUD_DB_CONNECTION_STRING` is the runtime setting read by OxiCloud. `DATABASE_URL` is only needed for SQLx build-time checks.
## Kubernetes (Helm)
```bash
@@ -77,14 +87,14 @@ kubectl logs statefulset/oxicloud -n oxicloud
| Client | Protocol | URL |
|--------|----------|-----|
| Windows Explorer | WebDAV | `http://host:8086/webdav/` |
| macOS Finder | WebDAV | `http://host:8086/webdav/` |
| Nautilus / Dolphin | WebDAV | `dav://host:8086/webdav/` |
| Thunderbird (calendar) | CalDAV | `http://host:8086/caldav/` |
| Thunderbird (contacts) | CardDAV | `http://host:8086/carddav/` |
| DAVx⁵ (Android) | CalDAV + CardDAV | `http://host:8086/` |
| GNOME Calendar | CalDAV | `http://host:8086/caldav/` |
| GNOME Contacts | CardDAV | `http://host:8086/carddav/` |
| Windows Explorer | WebDAV | `https://host/webdav/` |
| macOS Finder | WebDAV | `https://host/webdav/` |
| Nautilus / Dolphin | WebDAV | `davs://host/webdav/` |
| Thunderbird (calendar) | CalDAV | `https://host/caldav/` |
| Thunderbird (contacts) | CardDAV | `https://host/carddav/` |
| DAVx⁵ (Android) | CalDAV + CardDAV | `https://host/` |
| GNOME Calendar | CalDAV | `https://host/caldav/` |
| GNOME Contacts | CardDAV | `https://host/carddav/` |
| Collabora / OnlyOffice | WOPI | See [WOPI configuration](/config/wopi) |
## What's Next?
@@ -92,3 +102,4 @@ kubectl logs statefulset/oxicloud -n oxicloud
- [Environment Variables →](/config/env)
- [OIDC / SSO Setup →](/config/oidc)
- [WebDAV Guide →](/guide/webdav)
- [DAV Client Setup →](/guide/dav-client-setup)
docs/guide/search.md
+50 -27
View File
@@ -1,43 +1,66 @@
# Search
OxiCloud provides full-text search across your files with multiple filter options.
OxiCloud provides authenticated file and folder search with simple query parameters, advanced JSON criteria, pagination, recursive traversal, and in-memory result caching.
## Endpoint
## Endpoints
```http
GET /api/search?q=report&type_filter=pdf,docx&recursive=true
```
| Method | Endpoint | Description |
| --- | --- | --- |
| `GET` | `/api/search/` | Simple search using query parameters |
| `POST` | `/api/search/advanced` | Advanced search with a JSON body |
| `GET` | `/api/search/suggest` | Lightweight autocomplete suggestions |
| `DELETE` | `/api/search/cache` | Clear the search results cache |
## Query Parameters
All search endpoints require authentication.
## Simple Search Parameters
| Parameter | Description |
|---|---|
| `q` | Search query (matches file name) |
| `type_filter` | Comma-separated file extensions to filter by |
| `folder_id` | Restrict search to a specific folder |
| `recursive` | `true` to search subfolders |
| `date_from` / `date_to` | Filter by modification date |
| `size_min` / `size_max` | Filter by file size (bytes) |
| `limit` | Maximum results to return |
| --- | --- |
| `query` | Text to search in file and folder names |
| `type` | Comma-separated file extensions |
| `created_after` / `created_before` | Filter by creation time |
| `modified_after` / `modified_before` | Filter by modification time |
| `min_size` / `max_size` | Filter by file size in bytes |
| `folder_id` | Restrict search scope to one folder |
| `recursive` | Search subfolders, defaults to `true` |
| `limit` | Maximum results, defaults to `100` |
| `offset` | Pagination offset |
| `sort_by` | `relevance`, `name`, `name_desc`, `date`, `date_desc`, `size`, or `size_desc` |
## How It Works
### Example
OxiCloud stores file metadata in PostgreSQL with an `ltree` path column, enabling efficient recursive subtree queries:
```sql
SELECT * FROM storage.files
WHERE path <@ 'root.folder_id'
AND LOWER(name) LIKE '%query%'
AND LOWER(extension) = ANY('{pdf,docx}')
ORDER BY updated_at DESC
LIMIT 50;
```bash
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/search/?query=report&type=pdf,docx&recursive=true&limit=20"
```
## Frontend
## Advanced Search
The web UI includes a search bar in the toolbar. Results appear instantly with file name, path, size, and type. Clicking a result navigates to the file's location.
```json
{
"name_contains": "report",
"file_types": ["pdf", "docx"],
"min_size": 1024,
"folder_id": "folder-uuid",
"recursive": true,
"limit": 50,
"offset": 0
}
```
## Suggestions
Use `/api/search/suggest?query=rep&limit=10` for quick autocomplete-style results. Suggestions can also be scoped to a folder with `folder_id`.
## Result Caching
Search results are cached in memory using the search criteria and user ID as the cache key.
- Cache TTL: 5 minutes
- Max entries: 1000
- Manual invalidation: `DELETE /api/search/cache`
## Feature Flag
Search can be disabled via `OXICLOUD_ENABLE_SEARCH=false`.
Search can be disabled with `OXICLOUD_ENABLE_SEARCH=false`.
docs/guide/thumbnails-and-transcoding.md
+96
View File
@@ -0,0 +1,96 @@
# Thumbnails and Transcoding
OxiCloud optimizes image delivery with two complementary features:
- WebP thumbnail generation in three sizes
- On-the-fly image transcoding for browsers that advertise WebP support
Both features use a memory cache plus a persistent disk cache and are designed to stay off the request hot path whenever possible.
## Thumbnails
### Supported sizes
| Size | Dimensions | Directory |
| --- | --- | --- |
| `icon` | 150 x 150 | `.thumbnails/icon/` |
| `preview` | 400 x 400 | `.thumbnails/preview/` |
| `large` | 800 x 800 | `.thumbnails/large/` |
### Supported inputs
`image/jpeg`, `image/jpg`, `image/png`, `image/gif`, and `image/webp`
All thumbnail outputs are stored as WebP.
### API
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/files/{id}/thumbnail/{size}` | Fetch a thumbnail |
| `POST` | `/api/files/upload` | Upload a file and pre-generate thumbnails for supported images |
Thumbnail responses include:
- `Content-Type: image/webp`
- `Cache-Control: public, max-age=31536000, immutable`
- `ETag: "thumb-{id}-{size}"`
### Generation flow
1. Upload succeeds through the file API
2. If the MIME type is supported, OxiCloud starts thumbnail generation in a background task
3. If a thumbnail is requested before pre-generation completes, the request can generate it lazily
4. Future requests are served from memory or disk cache
## Image Transcoding
OxiCloud can serve a smaller WebP version of uploaded JPEG, PNG, or GIF files when the client advertises WebP support in the `Accept` header.
### Rules
- Files over 5 MB skip transcoding
- Existing WebP files are not transcoded again
- SVG and BMP are not transcoded
- If the WebP output is larger than the original, OxiCloud serves the original file instead
### Storage layout
```text
<storage_path>/
.transcoded/
webp/
<file_id>.webp
```
### Statistics tracked by the service
- Cache hits
- Disk hits
- Successful transcodes
- Bytes saved
- Transcode errors
## Caching
Both thumbnail and transcode services use:
- An in-memory LRU cache for hot assets
- A disk cache for persistent reuse across restarts
- Fire-and-forget background writes for cache warmup
For the broader cache model across metadata and listings, see [Caching Architecture](/architecture/caching).
## Example
```text
Client: GET /api/files/abc-123/download
Accept: image/webp, image/png, */*
Server: checks cache -> transcodes if needed -> returns the smaller asset
```
## Related Pages
- [Caching Architecture](/architecture/caching)
- [ZIP and Compression](/guide/zip-and-compression)
docs/guide/trash.md
+12 -1
View File
@@ -9,12 +9,21 @@ OxiCloud provides a trash system that soft-deletes files and folders, allowing u
3. Users can browse the trash, restore items, or permanently delete them
4. Items older than the retention period (default: **30 days**) are automatically purged
## Storage Model
- files and folders keep their original rows in PostgreSQL
- deletion into trash only flips soft-delete state and records the original parent location for restore
- blob content is not moved when an item enters the trash
- the unified `storage.trash_items` view is used to list trashed files and folders together
## API Endpoints
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/trash` | List trashed items |
| POST | `/api/trash/restore/{id}` | Restore a trashed item |
| DELETE | `/api/trash/files/{id}` | Move a file to the trash |
| DELETE | `/api/trash/folders/{id}` | Move a folder to the trash |
| POST | `/api/trash/{id}/restore` | Restore a trashed item |
| DELETE | `/api/trash/{id}` | Permanently delete |
| DELETE | `/api/trash/empty` | Empty the entire trash |
@@ -25,3 +34,5 @@ Permanent deletion decrements the blob reference count. If no other file points
## Feature Flag
Trash can be disabled via `OXICLOUD_ENABLE_TRASH=false`. When disabled, deletions are permanent.
Retention is controlled by `OXICLOUD_TRASH_RETENTION_DAYS`.
docs/guide/webdav.md
+72
View File
@@ -33,6 +33,65 @@ Always use HTTPS in production — Basic auth sends credentials in every request
| `DELETE` | Delete a file/folder |
| `LOCK` / `UNLOCK` | File locking |
## Common Operations
### List a directory
Use `PROPFIND` with a `Depth` header:
```http
PROPFIND /webdav/projects/ HTTP/1.1
Depth: 1
Content-Type: application/xml
<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:allprop/>
</D:propfind>
```
Successful directory listings return `207 Multi-Status`.
### Download a file
```http
GET /webdav/projects/document.pdf HTTP/1.1
Authorization: Basic base64(username:password)
```
### Upload or replace a file
```http
PUT /webdav/projects/document.pdf HTTP/1.1
Content-Type: application/pdf
<file bytes>
```
### Create a folder
```http
MKCOL /webdav/projects/new-folder HTTP/1.1
```
### Move or copy
```http
MOVE /webdav/old-location.pdf HTTP/1.1
Destination: https://your-server/webdav/new-location.pdf
```
```http
COPY /webdav/original.pdf HTTP/1.1
Destination: https://your-server/webdav/copy.pdf
```
### Delete a resource
```http
DELETE /webdav/projects/document.pdf HTTP/1.1
```
## Client Setup
### Windows Explorer
@@ -78,3 +137,16 @@ curl -u user:pass -X MKCOL https://your-server:8086/webdav/new-folder/
## Streaming PROPFIND
OxiCloud streams PROPFIND responses, so listing directories with thousands of files doesn't consume excessive memory.
## Integration Notes
- the WebDAV handler is only an HTTP adapter; file and folder operations still go through the same application services used by the REST API
- HTTP Basic Authentication is supported for DAV clients, while authorization rules remain the same as the rest of OxiCloud
- delete operations integrate with trash when the trash feature is enabled
## Troubleshooting
- Always use the `/webdav/` base path
- Prefer HTTPS because WebDAV uses Basic Authentication
- On Windows, make sure the `WebClient` service is enabled
- OxiCloud rejects path traversal segments such as `.` and `..` at the HTTP boundary
docs/guide/zip-and-compression.md
+67
View File
@@ -0,0 +1,67 @@
# ZIP and Compression
OxiCloud ships two compression-related features:
- ZIP download for folders
- Gzip compression for suitable file responses
## ZIP Download
### Endpoint
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/api/folders/{id}/download` | Download a folder as a ZIP archive |
### How it works
- The ZIP archive is built in memory
- Folder traversal uses an iterative queue rather than recursive async calls
- Cycle detection prevents loops while walking nested folders
- Entries are written with UNIX mode `0o755`
- Compression uses the `Deflated` method from the `zip` crate
### Example
```bash
curl -H "Authorization: Bearer $TOKEN" \
"https://oxicloud.example.com/api/folders/abc-123/download" \
-o my-folder.zip
```
## Gzip Compression
OxiCloud can compress responses when it is worth doing so.
### Compression threshold
Files below 50 KB are skipped.
### Compression levels
| Level | Value |
| --- | --- |
| `None` | `0` |
| `Fast` | `1` |
| `Default` | `6` |
| `Best` | `9` |
### Skip list
These types are not gzipped because they are already compressed or because compression provides poor returns:
- `image/*` except SVG and BMP
- `audio/*`
- `video/*`
- archive formats such as ZIP, gzip, 7z, RAR, bzip2, and XZ
### Runtime behavior
- Compression and decompression run in `spawn_blocking`
- The implementation uses `flate2`
- Optional buffer pooling reduces allocation churn under load
## Related Pages
- [Batch Operations](/guide/batch-operations)
- [Thumbnails and Transcoding](/guide/thumbnails-and-transcoding)
doc/images/Captura de pantalla 2025-03-23 230739.png → images/Captura de pantalla 2025-03-23 230739.png
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 622 KiB

After

Width:  |  Height:  |  Size: 622 KiB

src/interfaces/api/handlers/wopi_handler.rs
+1 -1
View File
@@ -6,7 +6,7 @@
//! These endpoints use `?access_token=` query parameter auth, NOT the
//! regular JWT auth middleware.
//!
//! Reference: doc/wopi-integration.md
//! Reference: docs/config/wopi.md
use crate::interfaces::middleware::auth::AuthUser;
use axum::{