feat(drive): stop writing in {file,folder}.user_id
plan:
```
What step 5 is actually doing
Goal: stop writing to storage.files.user_id / storage.folders.user_id on new INSERTs. This is a prerequisite for step 6 (dropping the column entirely). The column has existed since the pre-D0 single-owner era; every read that used to key on it has already been migrated to drive-membership grants over the last N days.
What has to change together for this to be safe:
┌──────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Piece │ What changes │ Why │
├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Migration │ Swap storage.files uniqueness indexes from (folder_id, name, user_id) to (drive_id, folder_id, name) │ Otherwise, two new rows with user_id = NULL would both be allowed (PG treats NULLs as distinct) — │
│ (a) │ │ uniqueness silently breaks │
├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Migration │ ALTER user_id DROP NOT NULL on both tables │ Otherwise, dropping the INSERT bind violates NOT NULL and every write 500s │
│ (b) │ │ │
├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Migration │ Drop dead user_id-leading indexes │ Cheap cleanup — nothing scans them anymore │
│ (c) │ │ │
├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ PL/pgSQL (d) │ Rewrite storage.copy_folder_tree without user_id in the INSERT column list │ Cross-drive copy runs entirely in SQL, needs the same treatment │
├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Rust (e) │ ~12 INSERT column-list drops across folder_db_repository, file_blob_write_repository, drive_pg_repository, dedup_service, │ Actual write path │
│ │ folder_service, load-seed.rs │ │
├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Rust (f) │ Simplify folder_db_repository::create_folder's parent lookup to only fetch drive_id (was fetching (user_id, drive_id)) │ It's fetching a value it no longer needs │
└──────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────┘
```
This commit is contained in:
Edouard Vanbelle
@@ -0,0 +1,108 @@
|
||||
-- ─────────────────────────────────────────────────────────────────────────
|
||||
-- D7 step 5 — retire `user_id` as a write/uniqueness axis on
|
||||
-- `storage.files` and `storage.folders`.
|
||||
--
|
||||
-- Every read that used to filter by `files.user_id = $caller` or
|
||||
-- `folders.user_id = $caller` has already been migrated to a
|
||||
-- drive-membership predicate (see D7-pass §6/§10 changes:
|
||||
-- `file_blob_read_repository`, `folder_db_repository`,
|
||||
-- `path_resolver_service`, `dedup_service`, plus `authz.require(Read, …)`
|
||||
-- at every WebDAV consumer site). This migration removes the last
|
||||
-- reason to keep binding `user_id` on writes:
|
||||
--
|
||||
-- 1. Files uniqueness indexes swap from `(folder_id, name, user_id)` /
|
||||
-- `(name, user_id)` → `(drive_id, folder_id, name)` /
|
||||
-- `(drive_id, name)`. Post-D0 `files.drive_id` is `NOT NULL`, so
|
||||
-- the drive-scoped form is strictly stronger — a file is unique
|
||||
-- by its position within its drive, not by "who used to own it".
|
||||
-- The folder side already got this treatment in D0
|
||||
-- (`20260802100002_drives_not_null.sql`).
|
||||
--
|
||||
-- 2. Dead user_id-leading indexes get dropped:
|
||||
-- - `idx_files_user_id`, `idx_folders_user_id` — nothing scans by
|
||||
-- `WHERE user_id = $1` any more.
|
||||
-- - `idx_folders_trashed` — was `(user_id, is_trashed)`; the
|
||||
-- trash listing moved to `(drive_id, is_trashed)` via the
|
||||
-- same D7 rewrite.
|
||||
-- - `idx_files_user_size_active` — was the per-user storage
|
||||
-- usage summary; the reconciliation sweep now GROUPs by
|
||||
-- `drive_id` (`storage_usage_service::update_all_drives_storage_usage`).
|
||||
--
|
||||
-- 3. `ALTER COLUMN user_id DROP NOT NULL` on both tables. The
|
||||
-- column stays for compat with the follow-up column-drop
|
||||
-- migration (D7 step 6) but new INSERTs will leave it NULL.
|
||||
-- Existing rows keep their backfilled values until the drop.
|
||||
--
|
||||
-- Steps 4-6 (Rust INSERT binds dropped + PL/pgSQL copy_folder_tree
|
||||
-- update) ship in the same commit so no in-flight INSERT ever
|
||||
-- tries to bind a NOT NULL that just went away.
|
||||
|
||||
-- ── 1. Swap files uniqueness indexes ─────────────────────────────────────
|
||||
--
|
||||
-- Pre-D7: name unique within (folder, user). Post-D7: name unique within
|
||||
-- (drive, folder). Since a drive has exactly one root folder tree and
|
||||
-- a given file lives in exactly one drive, this is a strict tightening.
|
||||
--
|
||||
-- The `IF EXISTS` guards let this migration re-run cleanly against a DB
|
||||
-- that's already been partially migrated (dev workflow).
|
||||
|
||||
DROP INDEX IF EXISTS storage.idx_files_unique_name_in_folder;
|
||||
DROP INDEX IF EXISTS storage.idx_files_unique_name_at_root;
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_files_unique_name_in_folder
|
||||
ON storage.files (drive_id, folder_id, name)
|
||||
WHERE NOT is_trashed AND folder_id IS NOT NULL;
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_files_unique_name_at_root
|
||||
ON storage.files (drive_id, name)
|
||||
WHERE NOT is_trashed AND folder_id IS NULL;
|
||||
|
||||
-- ── 2. Drop dead user_id-leading indexes ─────────────────────────────────
|
||||
|
||||
DROP INDEX IF EXISTS storage.idx_files_user_id;
|
||||
DROP INDEX IF EXISTS storage.idx_files_user_size_active;
|
||||
DROP INDEX IF EXISTS storage.idx_folders_user_id;
|
||||
DROP INDEX IF EXISTS storage.idx_folders_trashed;
|
||||
|
||||
-- ── 3. Allow NULL user_id on both tables ─────────────────────────────────
|
||||
|
||||
ALTER TABLE storage.files ALTER COLUMN user_id DROP NOT NULL;
|
||||
ALTER TABLE storage.folders ALTER COLUMN user_id DROP NOT NULL;
|
||||
|
||||
-- ── 4. Post-flight sanity ────────────────────────────────────────────────
|
||||
|
||||
DO $BODY$
|
||||
DECLARE
|
||||
files_nullable BOOLEAN;
|
||||
folders_nullable BOOLEAN;
|
||||
new_files_uniq BOOLEAN;
|
||||
BEGIN
|
||||
SELECT is_nullable::boolean INTO files_nullable
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'storage'
|
||||
AND table_name = 'files'
|
||||
AND column_name = 'user_id';
|
||||
|
||||
SELECT is_nullable::boolean INTO folders_nullable
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'storage'
|
||||
AND table_name = 'folders'
|
||||
AND column_name = 'user_id';
|
||||
|
||||
SELECT EXISTS (
|
||||
SELECT 1 FROM pg_indexes
|
||||
WHERE schemaname = 'storage'
|
||||
AND indexname = 'idx_files_unique_name_in_folder'
|
||||
) INTO new_files_uniq;
|
||||
|
||||
IF NOT files_nullable THEN
|
||||
RAISE EXCEPTION 'storage.files.user_id NOT NULL constraint did not drop';
|
||||
END IF;
|
||||
IF NOT folders_nullable THEN
|
||||
RAISE EXCEPTION 'storage.folders.user_id NOT NULL constraint did not drop';
|
||||
END IF;
|
||||
IF NOT new_files_uniq THEN
|
||||
RAISE EXCEPTION 'drive-scoped files uniqueness index did not land';
|
||||
END IF;
|
||||
END;
|
||||
$BODY$;
|
||||
@@ -0,0 +1,170 @@
|
||||
-- ─────────────────────────────────────────────────────────────────────────
|
||||
-- D7 step 5 — drop `user_id` from `storage.copy_folder_tree` INSERTs.
|
||||
--
|
||||
-- Companion to `20260902000000_files_folders_user_id_nullable.sql`. That
|
||||
-- migration made both `storage.files.user_id` and `storage.folders.user_id`
|
||||
-- nullable; this one stops writing to them from the copy-tree flow so
|
||||
-- copied rows leave the column NULL — provenance moves entirely to the
|
||||
-- `created_by` / `updated_by` §14 columns, which the PL/pgSQL already
|
||||
-- preserved from source.
|
||||
--
|
||||
-- No behavioural change apart from the write-time projection: reads no
|
||||
-- longer key on `files.user_id` (all migrated to drive-membership
|
||||
-- predicates), the uniqueness constraints don't include `user_id`
|
||||
-- (companion migration swapped them to drive-scoped), and provenance
|
||||
-- was already flowing through `created_by` / `updated_by`.
|
||||
--
|
||||
-- Identical function signature and return shape — no caller update
|
||||
-- needed.
|
||||
|
||||
CREATE OR REPLACE FUNCTION storage.copy_folder_tree(
|
||||
p_source_id UUID,
|
||||
p_target_parent_id UUID, -- NULL = copy to root (keeps source drive)
|
||||
p_dest_name TEXT DEFAULT NULL -- NULL = keep source folder name
|
||||
) RETURNS TABLE(new_root_id TEXT, folders_copied BIGINT, files_copied BIGINT) AS $$
|
||||
DECLARE
|
||||
v_root_lpath ltree;
|
||||
v_root_depth INT;
|
||||
v_max_depth INT;
|
||||
v_level INT;
|
||||
v_folders BIGINT := 0;
|
||||
v_files BIGINT := 0;
|
||||
v_inserted BIGINT;
|
||||
v_new_root UUID;
|
||||
v_dest_drive_id UUID;
|
||||
BEGIN
|
||||
-- Validate source exists.
|
||||
SELECT fo.lpath, nlevel(fo.lpath)
|
||||
INTO v_root_lpath, v_root_depth
|
||||
FROM storage.folders fo
|
||||
WHERE fo.id = p_source_id AND NOT fo.is_trashed;
|
||||
|
||||
IF v_root_lpath IS NULL THEN
|
||||
RAISE EXCEPTION 'Source folder not found: %', p_source_id
|
||||
USING ERRCODE = 'P0002'; -- no_data_found
|
||||
END IF;
|
||||
|
||||
-- Resolve destination drive_id once up front (cross-drive copy path).
|
||||
IF p_target_parent_id IS NULL THEN
|
||||
SELECT fo.drive_id INTO v_dest_drive_id
|
||||
FROM storage.folders fo
|
||||
WHERE fo.id = p_source_id;
|
||||
ELSE
|
||||
SELECT fo.drive_id INTO v_dest_drive_id
|
||||
FROM storage.folders fo
|
||||
WHERE fo.id = p_target_parent_id AND NOT fo.is_trashed;
|
||||
IF v_dest_drive_id IS NULL THEN
|
||||
RAISE EXCEPTION 'Target parent folder not found: %', p_target_parent_id
|
||||
USING ERRCODE = 'P0002';
|
||||
END IF;
|
||||
END IF;
|
||||
|
||||
-- Temp mapping: every folder in the subtree → new UUID.
|
||||
CREATE TEMP TABLE IF NOT EXISTS _copy_map(
|
||||
old_id UUID PRIMARY KEY,
|
||||
new_id UUID NOT NULL DEFAULT gen_random_uuid()
|
||||
) ON COMMIT DROP;
|
||||
TRUNCATE _copy_map;
|
||||
|
||||
INSERT INTO _copy_map(old_id)
|
||||
SELECT fo.id
|
||||
FROM storage.folders fo
|
||||
WHERE NOT fo.is_trashed
|
||||
AND fo.lpath <@ v_root_lpath;
|
||||
|
||||
SELECT cm.new_id INTO v_new_root
|
||||
FROM _copy_map cm WHERE cm.old_id = p_source_id;
|
||||
|
||||
SELECT MAX(nlevel(fo.lpath))
|
||||
INTO v_max_depth
|
||||
FROM storage.folders fo
|
||||
JOIN _copy_map cm ON fo.id = cm.old_id;
|
||||
|
||||
-- ── Insert folders level by level ──
|
||||
-- Post-D7: `user_id` intentionally omitted from the column list so
|
||||
-- copied rows leave the (now-nullable) column NULL. Provenance is
|
||||
-- carried by `created_by` / `updated_by` (§14 columns) — preserved
|
||||
-- from source so authorship survives the copy.
|
||||
FOR v_level IN v_root_depth .. v_max_depth LOOP
|
||||
INSERT INTO storage.folders(
|
||||
id, name, parent_id,
|
||||
drive_id, created_by, updated_by
|
||||
)
|
||||
SELECT cm.new_id,
|
||||
CASE WHEN fo.id = p_source_id AND p_dest_name IS NOT NULL
|
||||
THEN p_dest_name ELSE fo.name END,
|
||||
CASE WHEN fo.id = p_source_id THEN p_target_parent_id
|
||||
ELSE pm.new_id END,
|
||||
v_dest_drive_id,
|
||||
fo.created_by,
|
||||
fo.updated_by
|
||||
FROM storage.folders fo
|
||||
JOIN _copy_map cm ON fo.id = cm.old_id
|
||||
LEFT JOIN _copy_map pm ON fo.parent_id = pm.old_id
|
||||
WHERE NOT fo.is_trashed
|
||||
AND nlevel(fo.lpath) = v_level;
|
||||
|
||||
GET DIAGNOSTICS v_inserted = ROW_COUNT;
|
||||
v_folders := v_folders + v_inserted;
|
||||
END LOOP;
|
||||
|
||||
-- Temp mapping for files src→dst (dst ids pre-allocated so we can
|
||||
-- reference them in the dead-property duplication below).
|
||||
CREATE TEMP TABLE IF NOT EXISTS _copy_file_map(
|
||||
old_id UUID PRIMARY KEY,
|
||||
new_id UUID NOT NULL DEFAULT gen_random_uuid()
|
||||
) ON COMMIT DROP;
|
||||
TRUNCATE _copy_file_map;
|
||||
|
||||
INSERT INTO _copy_file_map(old_id)
|
||||
SELECT f.id
|
||||
FROM storage.files f
|
||||
JOIN _copy_map cm ON f.folder_id = cm.old_id
|
||||
WHERE NOT f.is_trashed;
|
||||
|
||||
-- ── Batch copy all files (zero-copy: same blob_hash) ──
|
||||
-- Post-D7: `user_id` omitted. Provenance via `created_by`/`updated_by`.
|
||||
INSERT INTO storage.files(
|
||||
id, name, folder_id, blob_hash, size, mime_type,
|
||||
media_sort_date, drive_id, created_by, updated_by
|
||||
)
|
||||
SELECT fm.new_id, f.name, cm.new_id, f.blob_hash, f.size,
|
||||
f.mime_type, f.media_sort_date, v_dest_drive_id, f.created_by,
|
||||
f.updated_by
|
||||
FROM storage.files f
|
||||
JOIN _copy_map cm ON f.folder_id = cm.old_id
|
||||
JOIN _copy_file_map fm ON fm.old_id = f.id
|
||||
WHERE NOT f.is_trashed;
|
||||
|
||||
GET DIAGNOSTICS v_files = ROW_COUNT;
|
||||
|
||||
-- Batch increment blob ref_counts.
|
||||
IF v_files > 0 THEN
|
||||
UPDATE storage.blobs b
|
||||
SET ref_count = ref_count + hc.cnt
|
||||
FROM (
|
||||
SELECT f.blob_hash, COUNT(*)::int AS cnt
|
||||
FROM storage.files f
|
||||
JOIN _copy_map cm ON f.folder_id = cm.new_id
|
||||
WHERE NOT f.is_trashed
|
||||
GROUP BY f.blob_hash
|
||||
) hc
|
||||
WHERE b.hash = hc.blob_hash;
|
||||
END IF;
|
||||
|
||||
-- Duplicate dead properties per RFC 4918 §8.8 — id-keyed store.
|
||||
INSERT INTO storage.webdav_dead_properties
|
||||
(folder_id, namespace, local_name, value)
|
||||
SELECT cm.new_id, dp.namespace, dp.local_name, dp.value
|
||||
FROM storage.webdav_dead_properties dp
|
||||
JOIN _copy_map cm ON dp.folder_id = cm.old_id;
|
||||
|
||||
INSERT INTO storage.webdav_dead_properties
|
||||
(file_id, namespace, local_name, value)
|
||||
SELECT fm.new_id, dp.namespace, dp.local_name, dp.value
|
||||
FROM storage.webdav_dead_properties dp
|
||||
JOIN _copy_file_map fm ON dp.file_id = fm.old_id;
|
||||
|
||||
RETURN QUERY SELECT v_new_root::text, v_folders, v_files;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
@@ -0,0 +1,108 @@
|
||||
-- ═══════════════════════════════════════════════════════════════════════════
|
||||
-- D0-step-8 companion — cascade-delete guard for the orphan-root check.
|
||||
--
|
||||
-- Fixes a latent bug in `storage.check_no_orphan_root_folder` that
|
||||
-- surfaced during user-delete tests. Repro (verified against a fresh
|
||||
-- test DB with no other data):
|
||||
--
|
||||
-- INSERT INTO auth.users … one user
|
||||
-- Run the atomic personal-drive create (drive + root folder +
|
||||
-- drives.root_folder_id wire-up + owner role_grant)
|
||||
-- DELETE FROM auth.users WHERE id = <that user>
|
||||
-- → ERROR: Orphan root folder rejected …
|
||||
--
|
||||
-- Root cause — the FK columns `storage.folders.created_by` and
|
||||
-- `storage.folders.updated_by` are declared
|
||||
-- `REFERENCES auth.users(id) ON DELETE SET NULL` (D0/M1 migration
|
||||
-- `20260802100000_drives_schema_additive.sql`, lines 117-129). So when
|
||||
-- `DELETE FROM auth.users` runs, PostgreSQL cascades a SET NULL
|
||||
-- update onto every folder row referencing that user — including that
|
||||
-- user's own personal-drive root folder. That UPDATE fires the
|
||||
-- DEFERRED `trg_no_orphan_root_folder` constraint trigger, which queues
|
||||
-- a check on the row's `NEW` state.
|
||||
--
|
||||
-- Cascade order (all inside the same transaction) is: SET NULL on the
|
||||
-- folder → cascade DELETE storage.drives (default_for_user FK) →
|
||||
-- cascade DELETE storage.folders (drive_id FK). By COMMIT, the drive
|
||||
-- and the folder are both gone. When the deferred trigger fires, its
|
||||
-- query `EXISTS (drive d WHERE d.id = NEW.drive_id AND d.root_folder_id
|
||||
-- = NEW.id)` finds no drive, so it raises. The check is correct in
|
||||
-- isolation — but the row it's checking no longer exists, so the
|
||||
-- invariant it's protecting no longer applies.
|
||||
--
|
||||
-- Fix: add an existence guard before the drive lookup. If the row has
|
||||
-- been deleted in the same transaction, skip the check — a deleted row
|
||||
-- can't be an orphan by definition.
|
||||
--
|
||||
-- This preserves the original invariant on all live rows:
|
||||
-- * The atomic four-write create transaction still gets checked at
|
||||
-- COMMIT and still requires the drive→folder wire-up (the folder
|
||||
-- row exists at COMMIT because we didn't delete it).
|
||||
-- * Direct SQL that tries to insert an orphan root folder is still
|
||||
-- rejected (the INSERT queues a check, the row exists at COMMIT,
|
||||
-- the drive lookup fails, exception raised).
|
||||
-- * The only new behaviour is "if this row was deleted before COMMIT,
|
||||
-- silently skip" — which is what the caller wanted anyway.
|
||||
--
|
||||
-- No table changes, no data changes, no reverse migration needed —
|
||||
-- `CREATE OR REPLACE FUNCTION` is idempotent, and every future call
|
||||
-- of the trigger picks up the new body immediately.
|
||||
|
||||
CREATE OR REPLACE FUNCTION storage.check_no_orphan_root_folder()
|
||||
RETURNS trigger AS $$
|
||||
BEGIN
|
||||
-- Non-root rows are guaranteed correct by their parent_id FK.
|
||||
IF NEW.parent_id IS NOT NULL THEN
|
||||
RETURN NULL;
|
||||
END IF;
|
||||
|
||||
-- Trashed root folders are soft-deleted in place — the resolver
|
||||
-- never lands on them, and they were valid roots before they got
|
||||
-- trashed. Skip enforcement; the row's history is preserved.
|
||||
IF NEW.is_trashed THEN
|
||||
RETURN NULL;
|
||||
END IF;
|
||||
|
||||
-- Cascade-delete guard (NEW in this migration).
|
||||
--
|
||||
-- The trigger is DEFERRABLE INITIALLY DEFERRED — it fires at COMMIT
|
||||
-- with `NEW` captured at trigger-queue time. If the row was
|
||||
-- subsequently deleted in the same transaction (e.g. the cascade
|
||||
-- path from `DELETE FROM auth.users` → SET NULL on created_by /
|
||||
-- updated_by → cascade DELETE storage.drives → cascade DELETE
|
||||
-- storage.folders), the invariant no longer applies: there's no
|
||||
-- orphan because the row itself is gone.
|
||||
IF NOT EXISTS (SELECT 1 FROM storage.folders WHERE id = NEW.id) THEN
|
||||
RETURN NULL;
|
||||
END IF;
|
||||
|
||||
-- The core check: some drive must point at this row as its
|
||||
-- root_folder_id, AND that drive must be the same one carrying
|
||||
-- our drive_id (the 1:1 bidirectional invariant from §3).
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM storage.drives d
|
||||
WHERE d.id = NEW.drive_id
|
||||
AND d.root_folder_id = NEW.id
|
||||
) THEN
|
||||
RAISE EXCEPTION
|
||||
'Orphan root folder rejected: storage.folders id=% has '
|
||||
'parent_id IS NULL and drive_id=%, but no drive has '
|
||||
'root_folder_id pointing at it. Root folders must be '
|
||||
'created via the atomic four-write transaction (see '
|
||||
'docs/plan/drive.md §3 and DrivePgRepository::'
|
||||
'create_personal_drive_atomic); direct SQL is not '
|
||||
'supported.',
|
||||
NEW.id, NEW.drive_id;
|
||||
END IF;
|
||||
|
||||
RETURN NULL;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
|
||||
COMMENT ON FUNCTION storage.check_no_orphan_root_folder() IS
|
||||
'DB-level guard for the "every root folder belongs to a drive" '
|
||||
'invariant. Wired as a DEFERRABLE INITIALLY DEFERRED constraint '
|
||||
'trigger so the atomic create transaction (folder INSERTed before '
|
||||
'drive UPDATEd) commits cleanly. Skips the check on rows that were '
|
||||
'deleted in the same tx (cascade path from user delete). See '
|
||||
'docs/plan/drive.md §3.';
|
||||
Reference in New Issue
Block a user