DioCrafts
168 lines
5.3 KiB
Markdown
168 lines
5.3 KiB
Markdown
# File System Safety in OxiCloud
|
|
|
|
This document describes the implementation of file system safety mechanisms in OxiCloud to ensure data integrity and durability during file operations.
|
|
|
|
## Introduction
|
|
|
|
Data integrity is critical in a file storage system like OxiCloud. When files are written to disk, it's important to ensure that:
|
|
|
|
1. Writes are atomic - they either complete fully or not at all
|
|
2. Data is properly synchronized to persistent storage
|
|
3. Directory entries are properly updated and persisted
|
|
4. The system can recover from unexpected crashes or power failures
|
|
|
|
OxiCloud implements several mechanisms to achieve these goals.
|
|
|
|
## The Problem: Buffered I/O and Data Loss
|
|
|
|
Standard file system operations in many programming languages and operating systems use buffered I/O by default:
|
|
|
|
```rust
|
|
// This operation may not immediately persist to disk
|
|
fs::write(path, content)
|
|
```
|
|
|
|
When an application writes data, the operating system typically:
|
|
|
|
1. Accepts the write into memory buffers
|
|
2. Acknowledges completion to the application
|
|
3. Schedules the actual disk write for later
|
|
|
|
This creates a window where a system crash or power failure can result in data loss, as the data may exist only in memory buffers that haven't been flushed to disk.
|
|
|
|
## OxiCloud's Solution
|
|
|
|
OxiCloud implements a comprehensive approach to file system safety through the `FileSystemUtils` service, which provides:
|
|
|
|
### 1. Atomic Write Pattern
|
|
|
|
Files are written using a safe atomic pattern:
|
|
|
|
```rust
|
|
/// Writes data to a file with fsync to ensure durability
|
|
/// Uses a safe atomic write pattern: write to temp file, fsync, rename
|
|
pub async fn atomic_write<P: AsRef<Path>>(path: P, contents: &[u8]) -> Result<(), IoError>
|
|
```
|
|
|
|
This implements a write-then-rename pattern:
|
|
1. Write to a temporary file in the same directory
|
|
2. Call `fsync` to ensure data is on disk
|
|
3. Atomically rename the temp file to the target file
|
|
4. Sync the parent directory to ensure the rename is persisted
|
|
|
|
### 2. Directory Synchronization
|
|
|
|
Directory operations are also synchronized:
|
|
|
|
```rust
|
|
/// Creates directories with fsync
|
|
pub async fn create_dir_with_sync<P: AsRef<Path>>(path: P) -> Result<(), IoError>
|
|
```
|
|
|
|
This ensures that:
|
|
1. Directories are properly created
|
|
2. Directory entries are persisted to disk
|
|
3. Parent directories are also synchronized
|
|
|
|
### 3. Rename and Delete Operations
|
|
|
|
Renames and delete operations follow the same pattern:
|
|
|
|
```rust
|
|
/// Renames a file or directory with proper syncing
|
|
pub async fn rename_with_sync<P: AsRef<Path>, Q: AsRef<Path>>(from: P, to: Q) -> Result<(), IoError>
|
|
|
|
/// Removes a file with directory syncing
|
|
pub async fn remove_file_with_sync<P: AsRef<Path>>(path: P) -> Result<(), IoError>
|
|
```
|
|
|
|
These operations ensure that:
|
|
1. The operation itself is completed
|
|
2. The parent directory entry is updated and synchronized
|
|
|
|
## Implementation Details
|
|
|
|
### Implementing fsync on Files
|
|
|
|
```rust
|
|
// Write file content
|
|
file.write_all(contents).await?;
|
|
|
|
// Ensure data is synced to disk
|
|
file.flush().await?;
|
|
file.sync_all().await?;
|
|
```
|
|
|
|
The `sync_all()` call is critical as it instructs the operating system to flush data and metadata to the physical storage device.
|
|
|
|
### Implementing fsync on Directories
|
|
|
|
```rust
|
|
// Sync a directory to ensure its contents (entries) are durable
|
|
async fn sync_directory<P: AsRef<Path>>(path: P) -> Result<(), IoError> {
|
|
let dir_file = OpenOptions::new().read(true).open(path).await?;
|
|
dir_file.sync_all().await
|
|
}
|
|
```
|
|
|
|
This is essential after operations that modify directory entries, such as creating, renaming, or deleting files.
|
|
|
|
## Usage in the Codebase
|
|
|
|
The `FileSystemUtils` service is integrated throughout OxiCloud's file operations:
|
|
|
|
### In File Write Repository
|
|
|
|
```rust
|
|
// Write the file to disk using atomic write with fsync
|
|
tokio::time::timeout(
|
|
self.config.timeouts.file_write_timeout(),
|
|
FileSystemUtils::atomic_write(&abs_path, &content)
|
|
).await
|
|
```
|
|
|
|
### In File Move Operations
|
|
|
|
```rust
|
|
// Move the file physically with fsync
|
|
time::timeout(
|
|
self.config.timeouts.file_timeout(),
|
|
FileSystemUtils::rename_with_sync(&old_abs_path, &new_abs_path)
|
|
).await
|
|
```
|
|
|
|
### For Directory Creation
|
|
|
|
```rust
|
|
// Ensure the parent directory exists with proper syncing
|
|
self.ensure_parent_directory(&abs_path).await?;
|
|
|
|
// Implementation uses FileSystemUtils
|
|
async fn ensure_parent_directory(&self, abs_path: &PathBuf) -> FileRepositoryResult<()> {
|
|
if let Some(parent) = abs_path.parent() {
|
|
time::timeout(
|
|
self.config.timeouts.dir_timeout(),
|
|
FileSystemUtils::create_dir_with_sync(parent)
|
|
).await
|
|
}
|
|
}
|
|
```
|
|
|
|
## Benefits
|
|
|
|
By implementing these safety measures, OxiCloud provides:
|
|
|
|
1. **Data Durability**: Critical data is properly synchronized to persistent storage
|
|
2. **Crash Resilience**: The system can recover from unexpected failures without data loss
|
|
3. **Consistency**: File operations maintain a consistent file system state
|
|
4. **Atomic Operations**: File writes appear as all-or-nothing operations
|
|
|
|
## Performance Considerations
|
|
|
|
These safety measures do have some performance impact, as synchronizing to disk is more expensive than buffered writes. However, OxiCloud:
|
|
|
|
1. Applies these measures only to critical operations
|
|
2. Uses timeouts to prevent operations from blocking indefinitely
|
|
3. Implements parallel processing for large files
|
|
|
|
The safety-performance tradeoff favors safety for critical data while still maintaining good performance for most operations. |