A comprehensive backup and synchronization tool for .NET 8 that provides both one-way backup and bidirectional file synchronization with advanced conflict resolution.
- Simple Backup: One-way synchronization (Source → Target)
- Bidirectional Sync: Two-way synchronization with conflict detection
- File Filtering: Include/exclude specific file types and folders using wildcard patterns
- Logging: Detailed logging with configurable log levels
- CLI Interface: Command-line interface with help text
- Configuration: JSON configuration file support with CLI override
- Conflict Detection: Automatically detects when both files have been modified
- Conflict Resolution: Creates timestamped backups of conflicting files
- Dry-Run Mode: Preview changes without making any modifications
- File Comparison: Uses both timestamp and hash comparison for accurate sync decisions
- Deletion Propagation: Optional deletion sync (experimental)
- File History Tracking: Complete version history with SQLite database
- Restore Functionality: Restore deleted files from history
- Automatic Cleanup: Configurable retention period for history files
- ZIP Archiving: Create compressed archives with folder structure preservation
- Split Archives: Create multi-part archives for large datasets
- Archive Extraction: Extract and list archive contents
- Scheduled Operations: Automatically create archives, backups, and syncs on schedule
- OS Scheduler Integration: Generate Windows Task Scheduler XML and cron expressions
- Parallel File Operations: Multi-threaded file copying for improved performance
- Progress Tracking: Real-time progress display with configurable update intervals
- Ensure you have .NET 8 SDK installed
- Clone or download this project
- Build the project:
dotnet build
# Simple one-way backup
BackupSynchronizer --source "C:\MyFiles" --target "D:\Backup" --mode simple
# Bidirectional sync
BackupSynchronizer --source "C:\FolderA" --target "C:\FolderB" --mode sync
# Dry-run preview
BackupSynchronizer --source "C:\FolderA" --target "C:\FolderB" --mode sync --dry-run-s, --source <path>: Source directory-t, --target <path>: Target directory-m, --mode <mode>: Mode: "simple" (one-way) or "sync" (bidirectional)-l, --log-level <level>: Log level (info, warning, error)-c, --config <file>: Configuration file path--delete-sync <bool>: Enable deletion propagation (true/false)--dry-run: Preview changes without making them--restore <filepath>: Restore a deleted file from history--list-deleted: List all deleted files--list-history <path>: List history for a specific file--history-keep-days <n>: Set history retention period (default: 30)--cleanup-history: Clean up expired history files--archive <output>: Create ZIP archive from source directory--split-size <size>: Split archive into parts (e.g., 100MB, 1GB)--compression-level <level>: Compression level (NoCompression, Fastest, Optimal, SmallestSize)--extract: Extract archive to target directory--list-archive: List contents of an archive--schedule <type>: Schedule type (daily, weekly, monthly, custom)--schedule-name <name>: Name for the schedule--create-schedule: Create a new scheduled operation--delete-schedule: Delete an existing schedule--list-schedules: List all scheduled operations--execute-schedule: Execute a scheduled operation--generate-task-scheduler: Generate Windows Task Scheduler XML--generate-cron: Generate cron expression for Linux/macOS--cron-expression <expr>: Custom cron expression for advanced scheduling--delete-source-after-archive: Delete source after successful archive--no-timestamped-archives: Don't add timestamps to archive names--max-threads <n>: Set maximum number of parallel threads (default: CPU count)--disable-parallel: Disable parallel file operations--no-progress: Disable progress display--progress-interval <n>: Update progress every N files (default: 10)-h, --help: Show help message
Create a config.json file in the same directory as the executable:
{
"SourcePath": "C:\\MyFiles",
"TargetPath": "D:\\Backup",
"Mode": "simple",
"SyncMode": "OneWay",
"LogLevel": "info",
"IncludePatterns": [],
"ExcludePatterns": [
"*.tmp",
"*.log",
"node_modules",
".git",
".vs",
"bin",
"obj"
],
"PreserveFolderStructure": true,
"OverwriteExisting": true,
"CreateLogFile": true,
"LogFilePath": "backup.log",
"EnableDeletionSync": false,
"DryRun": false,
"UseHashComparison": true,
"CreateConflictBackups": true,
"ConflictBackupDirectory": "conflicts",
"EnableHistoryTracking": true,
"HistoryRetentionDays": 30,
"HistoryDirectory": ".history",
"AutoCleanup": true,
"DatabasePath": "file_history.db",
"ArchivePath": "",
"SplitSizeBytes": null,
"CompressionLevel": "Optimal",
"ExtractArchive": false,
"ListArchive": false
}The tool supports wildcard patterns for including and excluding files:
*.*: Include all files*.txt: Include only .txt files!*.tmp: Exclude .tmp files (negation with !)!node_modules: Exclude node_modules directory*.{jpg,png,gif}: Include specific file extensions
Priority: Exclude patterns are checked first, then include patterns. If no include patterns are specified, all files (except excluded) are included.
BackupSynchronizer --source "C:\Documents" --target "D:\Backup\Documents" --mode simpleBackupSynchronizer --source "C:\FolderA" --target "C:\FolderB" --mode syncBackupSynchronizer --source "C:\FolderA" --target "C:\FolderB" --mode sync --dry-runBackupSynchronizer --source "C:\FolderA" --target "C:\FolderB" --mode sync --delete-sync trueBackupSynchronizer --restore "C:\MyFiles\document.txt"BackupSynchronizer --list-deletedBackupSynchronizer --list-history "C:\MyFiles\document.txt"BackupSynchronizer --cleanup-history --history-keep-days 7BackupSynchronizer --source C:\MyFiles --archive C:\Backup\archive.zipBackupSynchronizer --source C:\LargeFolder --archive backup.zip --split-size 100MBBackupSynchronizer --source archive.zip --target C:\Extracted --extractBackupSynchronizer --source archive.zip --list-archiveBackupSynchronizer --schedule daily --schedule-name "Daily Backup" --source C:\MyFiles --target D:\Backup --create-scheduleBackupSynchronizer --schedule weekly --schedule-name "Weekly Archive" --source C:\Data --archive C:\Backup\weekly.zip --create-scheduleBackupSynchronizer --list-schedulesBackupSynchronizer --schedule-name "Daily Backup" --execute-scheduleBackupSynchronizer --schedule-name "Daily Backup" --generate-task-schedulerBackupSynchronizer --schedule-name "Daily Backup" --generate-cronBackupSynchronizer --source C:\MyFiles --target D:\Backup --mode simple --max-threads 8BackupSynchronizer --source C:\MyFiles --target D:\Backup --mode simple --disable-parallelBackupSynchronizer --source C:\MyFiles --target D:\Backup --mode simple --progress-interval 25# Create config.json with your settings, then run:
BackupSynchronizer- Copies files from source to target
- Maintains folder structure
- Supports file filtering
- Overwrites existing files by default
- Synchronizes files in both directions
- Compares files using timestamps and hashes
- Detects conflicts when both files have been modified
- Creates timestamped backups of conflicting files
- Skips identical files to improve performance
When conflicts are detected:
- Both versions are backed up with timestamps
- Conflict is logged with details
- User can manually resolve conflicts later
- Sync continues with other files
The tool provides comprehensive logging:
- Console Output: Real-time feedback showing operations
- Log File: Detailed log file (default:
backup.log) with timestamps - Conflict Logs: Special logging for conflict detection and resolution
Log levels:
info: All operations and informationwarning: Warnings and errors onlyerror: Errors only
The tool handles various error scenarios:
- Missing source/target directories
- Permission issues
- File access conflicts
- Invalid configuration
- Network issues (for network paths)
Errors are logged and the sync process continues with other files.
The tool automatically tracks all file changes in a SQLite database (file_history.db):
- File Modifications: When files are overwritten during backup/sync
- File Deletions: When files are removed (if deletion sync is enabled)
- Metadata Storage: File path, hash, change type, timestamp, and original location
- History Structure: Files are moved to
.history/<relative_path>/<timestamp>_filename.ext
- Restore Files:
--restore <filepath>restores the latest version of a file - List Deleted:
--list-deletedshows all deleted files with timestamps - List History:
--list-history <path>shows complete history for a specific file - Restore to Date: Restore files to a specific point in time
- Retention Period: Configurable via
--history-keep-days(default: 30 days) - Auto Cleanup: Automatically removes expired history during sync operations
- Manual Cleanup:
--cleanup-historymanually removes old history files - Database Maintenance: Removes database entries for expired files
The SQLite database tracks:
- File path and original path
- SHA256 file hash for integrity verification
- Change type (Modified, Deleted, Created, etc.)
- Timestamp of the change
- File size and reason for the change
- Physical location of the history file
The tool can create compressed ZIP archives with comprehensive features:
- Folder Structure Preservation: Maintains complete directory hierarchy
- File Filtering: Uses same include/exclude patterns as backup/sync operations
- Compression Levels: NoCompression, Fastest, Optimal, SmallestSize
- Progress Tracking: Real-time progress with file counts and compression ratios
For large datasets, archives can be split into multiple parts:
- Size-Based Splitting: Specify maximum size per part (KB, MB, GB)
- Automatic Naming: Parts named as
archive.part001.zip,archive.part002.zip, etc. - Transparent Handling: Each part is a complete, valid ZIP archive
- Create:
--archive <output>creates ZIP from source directory - Extract:
--extractextracts archive to target directory - List:
--list-archiveshows all files and folders in archive - Compression:
--compression-levelcontrols compression algorithm
- Metadata Preservation: File timestamps and attributes maintained
- Error Handling: Graceful handling of file access issues
- Logging: Detailed logging of archive operations
- Size Reporting: Original vs compressed size with compression ratios
The tool supports multiple schedule types for automated operations:
- Daily: Execute every day at a specified time (default: 2:00 AM)
- Weekly: Execute on a specific day of the week
- Monthly: Execute on a specific day of the month
- Custom: Use custom cron expressions for advanced scheduling
- Create Schedules: Define automated backup, sync, or archive operations
- List Schedules: View all configured schedules with next run times
- Execute Schedules: Run schedules manually for testing
- Delete Schedules: Remove unwanted scheduled operations
- Schedule History: Track execution results and performance
The tool generates integration files for operating system schedulers:
- Windows Task Scheduler: Generate XML files for automatic import
- Linux/macOS Cron: Generate cron expressions for crontab
- Cross-Platform: Works on Windows, Linux, and macOS
- Timestamped Archives: Automatic timestamping of archive files
- Source Cleanup: Optional deletion of source files after archiving
- Error Handling: Robust error handling with retry mechanisms
- Logging: Detailed logging of all scheduled operations
- Validation: Comprehensive validation of schedule configurations
Schedules are stored in schedules.json with the following structure:
{
"scheduleName": "Daily Backup",
"scheduleType": "daily",
"sourcePath": "C:\\MyFiles",
"archivePath": "C:\\Backup\\daily.zip",
"enabled": true,
"nextRun": "2025-09-15T02:00:00",
"compressionLevel": "Optimal",
"includePatterns": [],
"excludePatterns": ["*.tmp", "*.log"]
}The tool now supports parallel file operations to significantly improve backup and sync performance:
- Multi-threaded Copying: Uses
Parallel.ForEachwith configurable thread counts - Thread-safe Operations: All operations are thread-safe with proper locking mechanisms
- Progress Tracking: Real-time progress display with file and byte counts
- Configurable Threading: Adjust thread count based on system capabilities
- Fallback Support: Automatic fallback to sequential operations when needed
- Default Threads: Automatically uses CPU core count for optimal performance
- Custom Thread Count: Override with
--max-threadsparameter - Thread Safety: All file operations use thread-safe collections and locking
- Error Isolation: File copy failures don't affect other parallel operations
Progress: 150/500 files (30.0%) | 45.2/150.8 MB (30.0%)
Features:
- File Count Progress: Shows files copied vs total files
- Byte Progress: Shows data copied vs total data size
- Percentage Display: Both file and byte completion percentages
- Configurable Updates: Set update frequency with
--progress-interval - Thread-safe Counters: Progress tracking works correctly with multiple threads
- Large File Sets: Significant speedup for directories with many files
- Network Drives: Improved performance when copying to/from network locations
- SSD Optimization: Better utilization of high-speed storage devices
- CPU Utilization: Makes better use of multi-core processors
{
"MaxThreads": 4,
"EnableParallelCopy": true,
"ShowProgress": true,
"ProgressUpdateInterval": 10
}- Thread Count: Use 2-4 threads for most scenarios, more for SSD-to-SSD copies
- Network Copies: Reduce thread count for network operations to avoid overwhelming
- Progress Updates: Use intervals of 10-50 files depending on operation size
- Error Handling: Monitor logs for any file-specific copy failures
# Clone the repository
git clone <repository-url>
cd BackupSynchronizer
# Restore dependencies
dotnet restore
# Build the project
dotnet build
# Run the application
dotnet run -- --helpThis project is open source and available under the MIT License.