Edouard Vanbelle
0e09cb81ff
The twin of thumb_derived_import, with the difference that shapes the
whole job: the legacy tree is keyed by FILE (`{file_id}.webp`) while the
destination is keyed by CONTENT. Thumbnail sidecars were already named
by blob hash, so importing them was a move; every entry here has to be
resolved through storage.files first.
That re-keying is the point rather than bookkeeping. A sandbox with five
.skip markers had three of them naming the same image, so the file-keyed
tree stored one verdict three times. After the import it is one row, and
any future upload of those bytes inherits it instead of paying for the
decision again.
Both artifact kinds are claimed by one walk: `{id}.webp` becomes a
derived Blob, `{id}.webp.skip` becomes a negative row. They share a
source file and a cursor, so splitting them into two passes would be two
chances for the pair to disagree about what had been handled. `.skip` is
matched BEFORE `.webp` — the shorter suffix matches a marker too, and
getting that backwards would read a zero-byte file and store it as the
transcode of its source, then serve it to clients. There is a test.
Entries whose file is gone cannot be re-keyed at all, so they are
reported and, under repair, deleted: unimportable by definition, and a
run that keeps rediscovering them never reports zero, so the gate for
removing the directory never opens.
Deletion reuses verify_and_unlink, so a cached transcode is removed only
after its stored replacement reads back byte-identical. Directory removal
follows the same rule as .thumbnails/ — delete, and only rename aside if
a non-cache file is in the way.
The batch checkpoint is a shared helper rather than inline at both exits
of the loop body. The first draft duplicated it and dropped the future on
the skip path without awaiting: the entry counted toward the batch, the
cursor never advanced, and a resumed run would have rewalked everything
it had already handled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
74 lines
2.1 KiB
Rust
74 lines
2.1 KiB
Rust
pub mod audio_metadata_service;
|
|
pub mod azure_blob_backend;
|
|
pub mod backend_consistency_service;
|
|
pub mod backend_migration_service;
|
|
pub mod backend_rotate_service;
|
|
pub mod blob_diagnostics;
|
|
pub mod blobs_consistency_service;
|
|
pub mod cached_blob_backend;
|
|
pub mod chunked_upload_service;
|
|
pub mod compression_service;
|
|
pub mod consistency_batch_service;
|
|
pub mod db_pool_monitor;
|
|
pub mod dedup_service;
|
|
pub mod dpop_nonce_service;
|
|
pub mod dpop_replay_cache;
|
|
pub mod dpop_verifier;
|
|
pub mod drives_consistency_service;
|
|
pub mod encrypted_blob_backend;
|
|
pub mod entry_backend;
|
|
pub mod exif_service;
|
|
pub mod face_geometry;
|
|
pub mod face_indexing_service;
|
|
pub mod ffmpeg_video_frame_service;
|
|
pub mod file_content_cache;
|
|
pub mod file_system_i18n_service;
|
|
pub mod files_consistency_service;
|
|
pub mod folders_consistency_service;
|
|
pub mod grant_cleanup_service;
|
|
pub mod image_transcode_service;
|
|
pub mod jwt_service;
|
|
pub mod last_seen_tracker;
|
|
pub mod local_blob_backend;
|
|
pub mod local_fs_mount_provider;
|
|
pub mod login_lockout_service;
|
|
pub mod manifests_consistency_service;
|
|
pub mod media_metadata_service;
|
|
pub mod mock_email_sender;
|
|
pub mod mount_provider_factory;
|
|
pub mod nextcloud_chunked_upload_service;
|
|
pub mod noop_face_analyzer;
|
|
pub mod oidc_service;
|
|
#[cfg(feature = "faces-onnx")]
|
|
pub mod onnx_face_analyzer;
|
|
pub mod opaque_login_exchange;
|
|
pub mod opaque_service;
|
|
pub mod password_hasher;
|
|
pub mod path_resolver_service;
|
|
pub mod path_service;
|
|
pub mod pg_acl_engine;
|
|
#[cfg(feature = "plugins")]
|
|
pub mod plugins;
|
|
pub mod recent_recording_hook;
|
|
pub mod retry_blob_backend;
|
|
pub mod s3_blob_backend;
|
|
pub mod satellites_consistency_service;
|
|
pub mod search_index;
|
|
pub mod session_cleanup_service;
|
|
pub mod session_liveness_gauges;
|
|
pub mod share_unlock_cookie;
|
|
pub mod smtp_email_sender;
|
|
pub mod swappable_blob_backend;
|
|
pub mod thumb_attached_import_service;
|
|
pub mod thumb_derived_import_service;
|
|
pub mod thumbnail_service;
|
|
#[cfg(test)]
|
|
mod thumbnail_service_test;
|
|
pub mod transcode_import_service;
|
|
pub mod trash_cleanup_service;
|
|
pub mod tree_etag_flush_service;
|
|
pub mod webdav_dead_property_store;
|
|
pub mod webdav_lock_service;
|
|
pub mod wopi_discovery_service;
|
|
pub mod zip_service;
|