feat(drive): clarify photo, music, search strategies
+ request non enumeration on search (a user must not guess existance
of any files he does not have access)
```
┌─────────────────────────────┬──────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────┐
│ Section │ Scope │ Why │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Photos │ Default Personal Drive (capability flag to include other drives) │ High noise risk from non-photo images in shared drives │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Music (library + playlists) │ Cross-drive (capability flag to exclude) │ Low noise risk; audio in shared drives is usually intentional content │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Favorites │ Cross-drive │ Personal curation │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Recent items │ Cross-drive │ Personal history │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Tantivy search │ Cross-drive │ Discovery tool │
├─────────────────────────────┼──────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Trash (D2) │ Per-drive (owner-actioned) │ Already specified │
└─────────────────────────────┴──────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────┘
```
This commit is contained in:
Edouard Vanbelle
+119
-17
@@ -575,7 +575,7 @@ users with one drive auto-select Personal silently, users with
|
|||||||
N drives get a real picker. No POC change needed — it just sees
|
N drives get a real picker. No POC change needed — it just sees
|
||||||
real drive rows instead of folder UUIDs.
|
real drive rows instead of folder UUIDs.
|
||||||
|
|
||||||
### 11. Content search index — drive-aware filtering
|
### 11. Content search index — cross-drive with anti-enum filtering
|
||||||
|
|
||||||
v0.7.0 added an embedded Tantivy full-text content index (see
|
v0.7.0 added an embedded Tantivy full-text content index (see
|
||||||
`infrastructure/services/search_index/tantivy_content_index.rs`
|
`infrastructure/services/search_index/tantivy_content_index.rs`
|
||||||
@@ -583,24 +583,60 @@ and migration `20260701000000_content_search_index.sql`). Today
|
|||||||
every indexed document carries the owning user as a filter field
|
every indexed document carries the owning user as a filter field
|
||||||
and queries restrict by that field at query time.
|
and queries restrict by that field at query time.
|
||||||
|
|
||||||
When ownership pivots to `drive_id`, the index has to follow —
|
The pivot to drives keeps **one global endpoint, `/api/search`,
|
||||||
otherwise search leaks content across drives the moment D7 drops
|
that aggregates across every drive the caller can read**. There
|
||||||
`user_id`:
|
is no per-drive search route in D0; the URL/picker UI in D1 may
|
||||||
|
add an optional `?drive_id=<uuid>` narrowing parameter, but the
|
||||||
|
default surface stays cross-drive.
|
||||||
|
|
||||||
|
The security primitive that makes cross-drive search safe is the
|
||||||
|
**`Occur::Must` clause applied at query time**, not after.
|
||||||
|
Tantivy's collector only sees documents that satisfy the Must
|
||||||
|
clause — stored fields (`preview`), counts, and pagination
|
||||||
|
cursors all reflect the filtered set. This is the same shape the
|
||||||
|
existing `user_id` filter uses today; we keep it and swap the
|
||||||
|
field.
|
||||||
|
|
||||||
1. **Schema update**: every indexed document gains a `drive_id`
|
1. **Schema update**: every indexed document gains a `drive_id`
|
||||||
field stamped at ingest time. Existing documents need a
|
STRING field stamped at ingest time. The existing `user_id`
|
||||||
one-shot reindex pass during the D0 migration (read each row,
|
field is kept during the dual-write window for rollback
|
||||||
look up its new `drive_id`, update the index entry). Cheap on
|
safety and dropped in D7.
|
||||||
small instances; the migration script should report a progress
|
2. **Query path**: replace the `Must user_id = caller` clause
|
||||||
count for larger ones.
|
with a `Must drive_id ∈ accessible_drives` set-membership
|
||||||
2. **Query path**: instead of filtering by `user_id = caller`,
|
clause. `accessible_drives` is computed fresh per query
|
||||||
expand `caller → set of drive_ids the caller can read`
|
(personal + every shared-drive membership), reusing the
|
||||||
(personal + every shared-drive membership) and filter by
|
`expand_user` cache that `PgAclEngine` already maintains and
|
||||||
`drive_id ∈ that set`. Expansion reuses the drive-role check
|
`AuthzCacheLifecycleHook` already invalidates on membership
|
||||||
already required by `PgAclEngine`.
|
change.
|
||||||
3. **Treat as a blocking step of D0**, not a D4/D5-era polish
|
3. **Handler-side ReBAC re-verification** (defense in depth):
|
||||||
item — otherwise the index is the silent leak path during the
|
after Tantivy returns hits, the `/api/search` handler
|
||||||
dual-write window.
|
re-checks each `file_id` with the engine. Catches two cases
|
||||||
|
the drive_id filter can't:
|
||||||
|
- **Index staleness** — file just moved to a drive the
|
||||||
|
caller can't access; indexer hasn't caught up.
|
||||||
|
- **Per-file grants** — ReBAC can grant access to a single
|
||||||
|
file inside a drive the caller doesn't otherwise have. The
|
||||||
|
filter is drive-only; the re-check restores per-file
|
||||||
|
resolution.
|
||||||
|
4. **Token subjects cannot search**. `/api/search` returns 401
|
||||||
|
for token-authenticated callers (anonymous link tokens have
|
||||||
|
access to one resource, not a drive — there is no meaningful
|
||||||
|
"search my drives" surface for them). The 401 is consistent;
|
||||||
|
"empty results" would leak nothing but would be operationally
|
||||||
|
confusing.
|
||||||
|
5. **Anti-enumeration response shape**: no "you have N hidden
|
||||||
|
matches" anywhere. The count, the cursor, and the snippet
|
||||||
|
list all reflect the filtered set and nothing else.
|
||||||
|
6. **Treat the reindex as a blocking step of D0**, not a D4/D5-
|
||||||
|
era polish item — otherwise the index is the silent leak
|
||||||
|
path during the dual-write window. The reindex re-runs the
|
||||||
|
existing `content_index_worker::drain_once()` loop after
|
||||||
|
`TantivyContentIndex::open_or_rebuild()` detects the schema
|
||||||
|
version bump; no separate CLI command needed (per the audit).
|
||||||
|
|
||||||
|
Hurl coverage in D0 includes a concrete anti-enumeration test:
|
||||||
|
user A indexes a file in a drive user B can't see, user B searches
|
||||||
|
the indexed term, response is empty + no hidden-count leak.
|
||||||
|
|
||||||
Filtering on a low-cardinality `drive_id` is something Tantivy
|
Filtering on a low-cardinality `drive_id` is something Tantivy
|
||||||
handles natively; this is bookkeeping, not a query-plan risk.
|
handles natively; this is bookkeeping, not a query-plan risk.
|
||||||
@@ -809,6 +845,72 @@ ancestry but **does NOT** propagate `updated_by` — ETags are
|
|||||||
fingerprints of structure, not authorship. Only the direct
|
fingerprints of structure, not authorship. Only the direct
|
||||||
mutation site updates `updated_by`.
|
mutation site updates `updated_by`.
|
||||||
|
|
||||||
|
### 15. Global sections — scope mapping
|
||||||
|
|
||||||
|
"Global sections" are the user-facing views that aggregate across
|
||||||
|
the filesystem rather than browsing a single folder: Photos, Music
|
||||||
|
library, Favorites, Recent items, Search, Trash. With drives
|
||||||
|
landing, each of these needs an explicit scope decision. The
|
||||||
|
table below locks the choices; the rationale is **noise risk by
|
||||||
|
file type**, not a uniform rule.
|
||||||
|
|
||||||
|
| Section | Scope | Capability flag (per-drive policy) | Why |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **Photos** (`/api/photos`) | Default Personal Drive only | `policies.include_in_photo_index = true` to opt a non-default drive in | Shared drives often carry images that aren't "photos" (screenshots, scans, charts-as-PNGs). Defaulting cross-drive pollutes the personal timeline. Opt-in for shared drives where the owner explicitly wants them indexed (e.g. "Family Photos" shared drive). |
|
||||||
|
| **Music** — library view (future) + playlists | Cross-drive (all accessible drives) | `policies.forbid_music_index = true` to opt a drive out | Audio files in shared drives are almost always intentional content (band collaboration, family music, podcast archive). Defaulting cross-drive matches user intent. Owner opts a drive out for the rare case it shouldn't be indexed. The Music section today is *only* playlists; a `/api/music/tracks` library view added later inherits this scope. |
|
||||||
|
| **Music playlists** (`audio.playlists`) | User-scoped, cross-drive curation | n/a | Playlists are a curation tool. `owner_id` stays on `auth.users(id)`; tracks reference files via `playlist_items.file_id` and may live in any drive the user has access to. At list time, `list_playlist_tracks` filters out tracks in drives the caller can no longer reach (see §11's defense-in-depth pattern). |
|
||||||
|
| **Favorites** (`/api/favorites/resources`) | Cross-drive (all accessible drives) | n/a | Personal organisation tool. Star a PDF from the work drive AND a photo from Personal — the whole point is cross-drive curation. ReBAC visibility check at list time drops rows the user can no longer reach. |
|
||||||
|
| **Recent items** (`/api/recent/*`) | Cross-drive (all accessible drives) | n/a | Personal history. Same shape as Favorites — you touched files across drives; the timeline reflects that. ReBAC visibility check at list time. |
|
||||||
|
| **Search** (`/api/search`) | Cross-drive (all accessible drives) | n/a | Discovery tool. See §11 for the Must-clause filter + handler-side ReBAC re-verification + anti-enum response shape. |
|
||||||
|
| **Trash** (`/api/trash/resources`) | Per-drive (owner-actioned) | n/a | Already specified in §12 — trash listing filters by drive(s) the caller can read; mutations require the owner role on the drive. |
|
||||||
|
|
||||||
|
#### Capability flag mechanism
|
||||||
|
|
||||||
|
Both `policies.include_in_photo_index` and
|
||||||
|
`policies.forbid_music_index` live under the same JSONB
|
||||||
|
`policies` column on `storage.drives` (see §8) — no new schema.
|
||||||
|
The default values reflect the table above: omitted = "off" for
|
||||||
|
photos (so non-default drives don't show photos unless the owner
|
||||||
|
opts in), omitted = "off" for music (so all accessible drives
|
||||||
|
*are* indexed unless the owner opts out).
|
||||||
|
|
||||||
|
The owner-only UI in the drive settings panel toggles these
|
||||||
|
flags. The query layer reads them at request time; flipping
|
||||||
|
either flag is instant — no reindex required because the filter
|
||||||
|
applies in the query Must-clause, the index itself is unchanged.
|
||||||
|
|
||||||
|
#### The Photos/Music asymmetry — defensible, not a smell
|
||||||
|
|
||||||
|
Photos defaulting to "default-drive only" while Music defaults to
|
||||||
|
"cross-drive" is the one case where two similar surfaces have
|
||||||
|
different defaults. The justification is the noise-risk argument
|
||||||
|
above: image content in shared drives is heterogeneous (often
|
||||||
|
not "photos" in the gallery sense), audio content in shared
|
||||||
|
drives is usually intentional. The capability flags let owners
|
||||||
|
fix either case, but the defaults match what the typical user
|
||||||
|
will want without configuration.
|
||||||
|
|
||||||
|
If a uniform rule is ever preferred, the cheapest move is to
|
||||||
|
flip Photos to cross-drive with `forbid_photo_index` as the
|
||||||
|
opt-out (mirroring Music). That can land later without a schema
|
||||||
|
change — just a behaviour change.
|
||||||
|
|
||||||
|
#### Verification sketch
|
||||||
|
|
||||||
|
The D0 Hurl suite (`tests/api/drives_foundation.hurl`) covers
|
||||||
|
the scope decisions concretely:
|
||||||
|
|
||||||
|
- Photos: file uploaded in Personal appears in `/api/photos`; same
|
||||||
|
file uploaded into a secondary personal drive does NOT appear
|
||||||
|
unless `include_in_photo_index` is set on that drive.
|
||||||
|
- Music: track uploaded in any accessible drive appears in the
|
||||||
|
library / sweeper output; setting `forbid_music_index` on a
|
||||||
|
drive removes its tracks from the next library response.
|
||||||
|
- Favorites: star a file in drive A and a file in drive B (both
|
||||||
|
accessible to caller); list returns both. Lose access to drive
|
||||||
|
B → next list omits the B file (no error, just absent).
|
||||||
|
- Search: see §11 anti-enum test.
|
||||||
|
|
||||||
## Migration strategy
|
## Migration strategy
|
||||||
|
|
||||||
A drive-id column on every resource is a database surgery touching
|
A drive-id column on every resource is a database surgery touching
|
||||||
|
|||||||
Reference in New Issue
Block a user