Commit Graph

495 Commits

Author SHA1 Message Date
Edouard Vanbelle 8b0fb03b5c feat(recoverable jobs): clarify life cycle pause vs cancel
a job can be paused/resumed
    a job as an exclusibity by it's name
    if you want to run another job with same name:
    either cancel the first one, or wait of it's terminaison

    pause does not permit to run the other job, this can create
    race conditions
2026-08-02 17:38:44 +02:00
Edouard Vanbelle 5eec0fb36e fix(migration): recover the main banner progression on server restart 2026-08-02 16:25:47 +02:00
Edouard Vanbelle 07802e01f8 feat(recoverable job): add pause/resume capability 2026-08-02 16:17:15 +02:00
Edouard Vanbelle 58c24bd150 feat(storage migration): make logs more verbose 2026-08-02 15:55:30 +02:00
Edouard Vanbelle 015f2da0f7 refactor(backend): normalize naming convention to backend rather storage
no ambiguity with the backend rather storage
2026-08-02 14:56:29 +02:00
Edouard Vanbelle a10d3254bf fix(storage migration): overwrite target blob if mismatch
if target blob already exists, migration will check the blob
    if header is already with the targeted key (or no key if not ciphered)  no need write
    otherwise write the blob (that will convert any blob with no header into the correct version)
2026-08-02 14:56:29 +02:00
Edouard Vanbelle 9902a6f8fe refactor(usage_reconcile): explicit naming to prevent confusion with storage (backend) 2026-08-02 14:56:29 +02:00
Edouard Vanbelle db5ae1cc90 fix(storage migration): do not flip storage backend on migration error 2026-08-02 14:56:29 +02:00
Edouard Vanbelle 6b3d36730c fix(blob consistency): fix report of unreadable blob 2026-08-02 14:56:26 +02:00
Edouard Vanbelle d4cb0c53d5 feat(blob): add a safety net on blob without header
function read_dispatch:

    the reader use th header to determined if file is encrypted
    and which key (fingerprint) was used

    if blob was recorded in legacy format (no header), it tries all attached keys on storage

    last safety net: if none matched, do a blake3 on file, if it matches the file is
    stored in clear (no cipher)

    otherwise:
        - blob is corrupted
        - key is lost
2026-08-02 14:56:26 +02:00
Edouard Vanbelle 4297d8139a feat(admin dashboard): clarify quota usage vs backend usage 2026-08-02 14:56:22 +02:00
Edouard Vanbelle dd1528de92 feat(rotate-key): show finger print + way to know if can remove key 2026-08-02 04:26:47 +02:00
Edouard Vanbelle 47246592b7 fix(blob_consistency): raise a finding if decipher fails 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 120dd09885 fix(decypher): try to decipher even with old format 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 0a41f561d0 feat(rotate-key): add report + key fingerprint in hexdigit fmt 2026-08-02 02:40:33 +02:00
Edouard Vanbelle ddede168f9 feat(job): delete temp files at boot time (only possible case: crash) 2026-08-02 02:40:33 +02:00
Edouard Vanbelle bc481bdd99 feat(job): fix key rotation on local storage (replace blob) 2026-08-02 02:40:33 +02:00
Edouard Vanbelle a58351b7ad feat(storage key rot): add admin panel 2026-08-02 02:40:33 +02:00
Edouard Vanbelle a9d5aae781 feat(storage key rot): add rotate services 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 30b0000c26 feat(storage key rot): add blob/chunk creation with the header 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 9485ee5540 feat(storage key rot): add blob header engine 2026-08-02 02:40:33 +02:00
Edouard Vanbelle e164689771 feat(storage key rot): remove dead born OXICLOUD_STORAGE_<N>_ENCRYPTION_CIPHER
+ alway ovewrite on storage migration (got issue when migrating with blob already existing and a key change)
2026-08-02 02:40:33 +02:00
Edouard Vanbelle bbfb106a32 feat(maintenance): add a maintenance notification during backend migration 2026-08-01 18:46:17 +02:00
Edouard Vanbelle 142afecbbf feat(storage): add a guide on storage 2026-08-01 17:50:35 +02:00
Edouard Vanbelle f409658c96 feat(storage): hot swap backend on migrate 2026-08-01 17:11:07 +02:00
Edouard Vanbelle 8329b4aa56 feat(storage): improve admin panel 2026-08-01 17:10:06 +02:00
Edouard Vanbelle cc439aaff9 fix(dedup): fix informations 2026-08-01 16:20:49 +02:00
Edouard Vanbelle f949939508 feat(storage): add cmd option --select-storage 2026-08-01 14:36:24 +02:00
Edouard Vanbelle d7c19570a5 feat(storage): wire choice of storage 2026-08-01 14:19:14 +02:00
Edouard Vanbelle 2de71b6d9a feat(storage): add readonly during storage migration 2026-08-01 13:43:04 +02:00
Edouard Vanbelle 6b7bb67500 feat(storage): jobs can choose storage to migrate/scan 2026-08-01 13:06:09 +02:00
Edouard Vanbelle 354e058114 feat(storage): add multi entry in config 2026-08-01 12:58:43 +02:00
Edouard Vanbelle 82a50936d4 feat(storage-migration): move storage mig. to recoverable job 2026-08-01 11:53:51 +02:00
Edouard Vanbelle 5a87999949 fix(integration test): fix drive test used bytes with grace period 2026-07-30 01:29:09 +02:00
Edouard Vanbelle 9fe47a53ea feat(jobs): add admin call to purge jubs result 2026-07-30 00:39:11 +02:00
Edouard Vanbelle 507bc2e98d feat(recoverable-job): add backend_consistency (storage) 2026-07-30 00:15:20 +02:00
Edouard Vanbelle 8de50ec404 feat(jobs/ui): blobs_consistency add b3sum check 2026-07-29 23:15:45 +02:00
Edouard Vanbelle e822714b50 feat(jobs/ui): reorder job names + fix(blobs_consistency) 2026-07-29 23:01:20 +02:00
Edouard Vanbelle 5527d09618 feat(recoverable-job): check if old blob (no cdc) still remains: notice only 2026-07-29 22:47:07 +02:00
Edouard Vanbelle 61b8571a48 feat(recoverable-job): add grace period for drives, files, folders consistency 2026-07-29 22:29:20 +02:00
Edouard Vanbelle 44356b14f2 feat(recoverable-job): add blobs_consistency job 2026-07-29 22:25:05 +02:00
Edouard Vanbelle 0f12399a48 feat(recoverable-job): fix files_consistency to check blob chunk consistency 2026-07-29 22:17:15 +02:00
Edouard Vanbelle 5881968f50 feat(recoverable-job): add progress view 2026-07-29 08:41:19 +02:00
Edouard Vanbelle 4336eca4d1 feat(recoverable-job): add admin page 2026-07-29 08:03:43 +02:00
Edouard Vanbelle e1556e3d36 feat(recoverable-job): add findings 2026-07-29 07:57:06 +02:00
Edouard Vanbelle 41d83b3053 feat(recoverable-job): add consistency_batch (runs all consistency check) 2026-07-29 01:44:52 +02:00
Edouard Vanbelle 782a5c99bd feat(recoverable-job): add folder_consistency 2026-07-29 01:39:58 +02:00
Edouard Vanbelle 394708c9a1 feat(recov. job): add drive consistency service 2026-07-29 01:25:15 +02:00
Edouard Vanbelle 8177b04786 feat(recoverable-job): wire API (cancel, view run, ...) 2026-07-28 23:25:37 +02:00
Edouard Vanbelle 302d2ff80f feat(recoverable-job): add core engine 2026-07-28 22:39:07 +02:00