Commit Graph

13 Commits

Author SHA1 Message Date
Diocrafts 761d159a92 feat(dedup): CDC sub-file deduplication with FastCDC + parallel chunk storage + dedup skip
- Replace whole-file SHA-256 dedup with FastCDC 2020 content-defined chunking
  (min 64KB, avg 256KB, max 1MB) + BLAKE3 hashing
- Add chunk_manifests table (file_hash → chunk_hashes[] + chunk_sizes[])
- Add put_blob_from_bytes to BlobStorageBackend trait (all 7 backends)
- 3-phase store_chunks pipeline:
  Phase 0: batch-check existing chunks (single PG query)
  Phase 1: selective disk read (skip existing chunks entirely)
  Phase 2: parallel upload with buffer_unordered(8)
- CDC-aware read_blob_stream and read_blob_range_stream with legacy fallback
- Transactional manifest + chunk ref-count cascade on remove_reference
- 12 CDC tests (determinism, reassembly, contiguity, sub-file dedup, etc.)
- Update deduplication.md to reflect new architecture
2026-04-14 23:17:39 +02:00
Andrey Tkachenko da066f47fa Music Player & Playlist Manager 2026-04-08 15:14:03 +03:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
Jared Wolff 9ec0f0e0d6 fix(migrations): add ALTER TABLE fallback for media_sort_date column
When the storage.files table already exists from a manual migration,
CREATE TABLE IF NOT EXISTS is skipped entirely, so the new
media_sort_date column is never added. Add an ALTER TABLE ADD COLUMN
IF NOT EXISTS immediately after the CREATE TABLE to handle both fresh
and pre-existing databases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:17:34 -04:00
Jared Wolff 036390a242 feat: add SQL migration system using sqlx::migrate!()
Replace manual schema.sql application with sqlx's built-in migration
system. Migrations are embedded at compile time and tracked in the
_sqlx_migrations table. Pending migrations run automatically on startup.

- Move db/schema.sql → migrations/20260307000000_initial_schema.sql
- Remove apply_schema() and split_sql_statements() from db.rs
- Add run_migrations() using sqlx::migrate!() macro
- Remove docker-compose schema.sql mount (app handles it now)
- Enable sqlx "migrate" feature in Cargo.toml

Future schema changes: add a new timestamped .sql in migrations/.

Closes #190

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:30:52 -04:00
Dionisio 8f2b0a354c big refactoring 2026-02-03 17:59:04 +01:00
DioCrafts 52d8250d51 adding card dav and cald dav 2025-04-13 01:04:04 +02:00
DioCrafts 5d67bc4d84 fix several bugs 2025-04-10 01:43:25 +02:00
DioCrafts 8f1d213526 improve postgresql performance 2025-04-09 00:21:20 +02:00
DioCrafts fb276d9b24 adding ui sharing 2025-03-28 08:09:18 +01:00
DioCrafts 38b0e9594b fix auth errors and add primigenial paper trash 2025-03-24 16:47:42 +01:00
DioCrafts 9a9fd72f61 fixing bugs 2025-03-23 22:44:18 +01:00
DioCrafts cafad0fbfd adding user authentication 2025-03-20 09:22:31 +01:00