fix(name duplicate): fixed via NFC normalisation

TL;DR:

    fix duplicate filename via:

    ```
    docker exec <container> migrate-nfc-filenames --dry-run    # preview
    docker exec <container> migrate-nfc-filenames              # execute
    ```

 == issue ==

  Last week I uploaded Capture d'écran 2026-06-03 à 20.04.24.png from the web. It synced down to Nextcloud on my Mac. Two minutes later, the Web UI was showing the file twice.

  Both rows had:
  - the same name
  - the same size
  - the same content hash

  So why two rows? Because to PostgreSQL, the names weren't the same.

  Web upload (browser → Postgres):
    "é" stored as 1 codepoint  (U+00E9)        bytes: c3 a9     ← NFC

  NiextCloud client (macOS → Postgres):
    "é" stored as 2 codepoints (e + U+0301)    bytes: 65 cc 81  ← NFD

  macOS's APFS keeps filenames in NFD (decomposed); browsers send NFC (composed). Visually é and é are identical. To WHERE name = $1 they're two different keys. Our UNIQUE index on (folder_id, name, user_id) never fired — and the row count quietly drifted every time a Mac user touched an accented
  filename.

 == The fix is two halves ==

  1. No new duplicates — every name-receiving boundary (file upload, NC PUT, rename, MOVE, path lookup) now NFC-normalizes before touching the database. The storage invariant becomes "every stored name is NFC".
  2. Clean up existing data — one-shot migrate-nfc-filenames binary walks storage.files, NFC-normalizes any non-NFC row, and resolves the collisions we've accumulated. Same-content duplicates go to trash (recoverable); different-content collisions get renamed with a .duplicate suffix.

 == use of the clean up ==

    example of use (do not forget to define env **DATABASE_URL**)

    either
        `cargo run --bin migrate-nfc-filenames -- --dry-run`
    or
        `cargo build --bin migrate-nfc-filenames`
        `./target/debug/migrate-nfc-filenames --dry-run`

    example:
```
    % ./target/debug/migrate-nfc-filenames --dry-run
    === NFC filename migration (DRY RUN — no writes) ===

    Loaded 543 non-trashed file rows

    NORMALIZE  163451b5-5e6c-404b-9b1e-f4b01a2b7269  user=42433185-4717-416d-9a15-4580fff171ec  'Capture d’écran 2026-03-20 à 14.44.50.png' → 'Capture d’écran 2026-03-20 à 14.44.50.png'
    NORMALIZE  827dddec-4dd5-48c2-a120-dec5289f7d29  user=969deca6-7935-4f12-a430-4d636b62fa3e  'Capture d’écran 2026-04-03 à 15.43.38.png' → 'Capture d’écran 2026-04-03 à 15.43.38.png'
    NORMALIZE  09559934-a620-472d-9ba8-fc3cfeb6dc6f  user=a0643a21-0092-4a84-9dde-7ac4e76bc1a5  'Capture d’écran 2026-06-03 à 20.05.38.png' → 'Capture d’écran 2026-06-03 à 20.05.38.png'
    NORMALIZE  5ce6dbf9-0562-4758-8783-671aa9069590  user=a0643a21-0092-4a84-9dde-7ac4e76bc1a5  'Capture d’écran 2026-06-05 à 11.07.25.png' → 'Capture d’écran 2026-06-05 à 11.07.25.png'
    DEDUP      newer=26bcf82b-99cc-45c8-9d69-dd7e5c4484ff (trash, same blob)  older=df3adc67-a778-424d-a817-b930c75f3b06  user=a0643a21-0092-4a84-9dde-7ac4e76bc1a5  hash=0d2cc7b0ffce2850

    === Summary ===
      scanned                            : 543
      already in NFC                     : 538
      normalized in place (no collision) : 4
      dedup-trashed (same content)       : 1
      renamed to .duplicate              : 0

    DRY RUN — no rows were written. Re-run without --dry-run to apply.
```

    once valid remove --dry-run
This commit is contained in:
Edouard Vanbelle
2026-06-06 17:47:59 +02:00
parent 5a18ec4bac
commit 7db547a669
8 changed files with 433 additions and 8 deletions
+12 -4
View File
@@ -1,6 +1,8 @@
use uuid::Uuid;
use crate::domain::services::path_service::{StoragePath, validate_storage_name};
use crate::domain::services::path_service::{
StoragePath, normalize_storage_name, validate_storage_name,
};
// Re-export entity errors from the centralized module
pub use super::entity_errors::{FolderError, FolderResult};
@@ -80,6 +82,7 @@ impl Folder {
parent_id: Option<String>,
owner_id: Option<Uuid>,
) -> FolderResult<Self> {
let name = normalize_storage_name(&name);
// Validate folder name
if let Err(reason) = validate_storage_name(&name) {
return Err(FolderError::InvalidFolderName(format!("{name}: {reason}")));
@@ -171,6 +174,7 @@ impl Folder {
modified_at: u64,
tree_modified_at: u64,
) -> FolderResult<Self> {
let name = normalize_storage_name(&name);
if let Err(reason) = validate_storage_name(&name) {
return Err(FolderError::InvalidFolderName(format!("{name}: {reason}")));
}
@@ -272,10 +276,13 @@ impl Folder {
let storage_path = StoragePath::from_string(&path);
// Create directly without validation to avoid errors in DTO
// conversions. `tree_modified_at` defaults to `modified_at`:
// DTO round-trips lose the real rollup signal, so callers
// that need a freshly-rolled-up etag must reload from the
// conversions. Still NFC-normalize so DTO-reconstructed
// entities maintain the storage invariant.
// `tree_modified_at` defaults to `modified_at`: DTO
// round-trips lose the real rollup signal, so callers that
// need a freshly-rolled-up etag must reload from the
// repository.
let name = normalize_storage_name(&name);
Self {
id,
name,
@@ -293,6 +300,7 @@ impl Folder {
/// Creates a new version of the folder with updated name
pub fn with_name(&self, new_name: String) -> FolderResult<Self> {
let new_name = normalize_storage_name(&new_name);
if let Err(reason) = validate_storage_name(&new_name) {
return Err(FolderError::InvalidFolderName(format!(
"{new_name}: {reason}"