Robust bidirectional file synchronization for Linux and WSL using rsync with manifest-based change tracking, conflict resolution, and safe deletion propagation.
Unlike simple "pull then push" approaches, this tool uses a three-way diff algorithm:
- Manifest tracking - After each sync, a snapshot of all file metadata (path, mtime, size) is saved
- Change detection - On the next run, both local and remote are compared against the previous manifest
- Smart classification - Each file is classified as: new, modified, deleted, or unchanged on each side
- Conflict resolution - Files modified on both sides are handled according to your configured strategy
- Safe deletions - Deletions are only propagated when the file existed in the previous manifest (so new files are never accidentally deleted)
- Per-directory exclusions -
.syncignorefiles (rsync filter syntax) let you exclude files from sync on a per-directory basis, with rules inherited by subdirectories
# 1. Install
curl -fsSL https://raw.githubusercontent.com/INS-JVidal/rsync-bidirectional-sync/main/install-remote.sh | bash
# 2. Configure
nano ~/.config/rsync-sync/config
# 3. (Optional) Create .syncignore
sync-client init-syncignore python
# 4. Set up SSH keys
ssh-copy-id user@remote-host
# 5. Test
sync-client --dry-run
# 6. Sync
sync-clientcurl -fsSL https://raw.githubusercontent.com/INS-JVidal/rsync-bidirectional-sync/main/install-remote.sh | bashgit clone https://github.com/INS-JVidal/rsync-bidirectional-sync.git
cd rsync-bidirectional-sync
./install.shThe installer:
- Copies scripts directly to
~/.local/bin/ - Injects the git version (e.g.,
v1.0.0-3-gabcdef) into installed scripts - Creates configuration directory at
~/.config/rsync-sync/ - Sets up bash completion
- Adds
~/.local/binto PATH if needed
- Bash 4.0+
- rsync (on both local and remote)
- OpenSSH client (with key-based auth recommended)
- Standard Unix tools: find, stat, sort, md5sum
# If installed via curl
curl -fsSL https://raw.githubusercontent.com/INS-JVidal/rsync-bidirectional-sync/main/install-remote.sh | bash -s -- --uninstall
# If installed from source
./install.sh --uninstallFrom a clone inside WSL, setup-wsl.sh installs/updates sync-client, sets up the SSH key, creates a named profile whose local side is a folder in your Windows user profile, and runs the first sync:
./setup-wsl.sh jvidal@192.168.1.7 Sync/KeePass # C:\Users\<you>\Sync\KeePass <-> ~/Sync/KeePass
sync-client -p keepass # later runsThe folder path is relative to the Windows user folder locally and to the remote user's home. Windows programs (e.g. KeePassXC) open the files from the Windows path. Keep the folder outside OneDrive.
Default config: ~/.config/rsync-sync/config
# Remote connection
REMOTE_USER="user"
REMOTE_HOST="192.168.1.100"
REMOTE_PORT=22
# Sync paths
LOCAL_DIR="/home/user/projects"
REMOTE_DIR="/home/user/projects"
# Conflict resolution: newest, skip, backup, local, remote
CONFLICT_STRATEGY="newest"
# Propagate deletions to the other side
PROPAGATE_DELETES=true
# Back up files before overwriting during conflicts
BACKUP_ON_CONFLICT=trueSee config.example for all available options.
Create named profiles for different sync targets:
# Create profile
cp ~/.config/rsync-sync/config ~/.config/rsync-sync/profiles/work.conf
nano ~/.config/rsync-sync/profiles/work.conf
# Use profile
sync-client -p work syncEach profile maintains its own sync state, lock file, and logs.
Exclude files from sync on a per-directory basis using .syncignore files with rsync filter syntax.
Quick start:
# Generate a .syncignore from a built-in template
sync-client init-syncignore python
# Edit to taste
nano .syncignore
# Verify what's excluded
sync-client --show-exclusions
# Sync as usual
sync-clientSyntax:
| Pattern | Meaning |
|---|---|
*.log |
Exclude files matching pattern |
+ important.log |
Force-include (override previous excludes) |
build/ |
Exclude directory only |
# comment |
Comment line |
Note: Unlike
.gitignore, include syntax uses+ patternnot!pattern.
Built-in templates: default, python, java, javascript, php, teaching
Configuration toggles:
USE_SYNCIGNORE=true # Enable .syncignore support (default: true)
SYNCIGNORE_FILENAME=".syncignore" # Custom filename (default: .syncignore).syncignore files are synced to both sides by default so exclusion rules stay consistent.
# Basic sync
sync-client
# Check what would change
sync-client status
# Preview without making changes
sync-client --dry-run
# Verbose output
sync-client --verbose
# Use a profile
sync-client --profile work
# Reset sync state (next sync = first sync)
sync-client reset-state
# Combine options
sync-client -p work -v -n| Command | Description |
|---|---|
sync |
Run bidirectional sync (default) |
status |
Show pending changes without syncing (exit code: 0 = in sync, 1 = changes pending) |
reset-state |
Clear manifest, treat next sync as first sync |
show-exclusions |
Show all active exclusion rules |
init-syncignore [TPL] |
Create .syncignore from template |
| Option | Description |
|---|---|
-p, --profile NAME |
Use named profile |
-n, --dry-run |
No changes, just show what would happen |
-v, --verbose |
DEBUG-level logging |
-f, --force |
Skip confirmation prompts |
-c, --config FILE |
Use specific config file |
--show-exclusions |
Show all active exclusion rules and exit |
--init-syncignore [TPL] |
Create .syncignore (templates: default, python, java, javascript, php, teaching) |
-h, --help |
Show help |
-V, --version |
Show version |
| Strategy | Behavior |
|---|---|
newest |
Keep the version with the most recent mtime (default) |
skip |
Leave both versions untouched, report the conflict |
backup |
Back up both versions, then apply newest |
local |
Always prefer the local version |
remote |
Always prefer the remote version |
- Lock files - Prevents concurrent sync runs using a kernel
flocklock that releases automatically if the process dies (no stale locks); falls back to PID-based stale-lock detection whereflockis unavailable - Signal handling - Clean shutdown on Ctrl+C or SIGTERM
- Manifest-based deletions - Only propagates intentional deletes
- Backup on conflict - Optional backup before overwriting
- Dry-run mode - Preview all changes safely
- Crash-safe transfers - interrupted transfers are quarantined in
.sync-partial(rsync--partial-dir) so a truncated file never overwrites the good copy; resumable on the next run - Atomic state save - the manifest is written via a temp file + rename, so a crash mid-save can't corrupt sync state
- Retry with backoff - transient network failures are retried (
MAX_RETRIES/RETRY_DELAY) .syncignoredelete protection - Files matching.syncignorerules are protected from deletion propagation- State preservation on error - Manifest only saved after full success
- Retry logic - Configurable retries with backoff for network issues
- Version mismatch detection - Pre-flight checks warn if local and remote run different versions (cached daily)
- Change detection is mtime + size (not content hash). An edit that keeps the
same size and restores the original mtime (e.g.
touch -r, some editors/build tools) may go undetected.CHECKSUM_VERIFY=trueonly re-checks files already flagged as conflicts; it does not change initial detection. - Empty directories are not tracked. The manifest records files and symlinks only, so an empty directory is never propagated, and a directory that becomes empty after its files are deleted may remain on the other side.
- "newest" relies on synchronized clocks. If the two hosts' clocks disagree,
CONFLICT_STRATEGY=newestcan keep the older edit. A pre-flight warning fires pastCLOCK_SKEW_WARN_SECONDS; keep both machines on NTP. - One profile per tree. Locks are per-profile; running two profiles against
the same/overlapping
LOCAL_DIRconcurrently is unsupported. - Shared/multi-user trees. If a different, less-trusted user can write into
LOCAL_DIR/REMOTE_DIRwhile a sync runs, a directory-symlink swap race (TOCTOU) is possible. Intended for single-user or same-trust-domain trees.
By default the tool connects with StrictHostKeyChecking=accept-new, which
automatically trusts a host the first time it is seen. This keeps first-time
setup frictionless, but it means a machine-in-the-middle present during that
first connection (e.g. via DNS or ARP spoofing) can have its rogue host key
silently accepted. From then on it controls the remote manifest, the
transferred files, and remote command execution.
To close this window, pin the remote host key before your first sync and switch SSH to strict checking:
# 1. Record the real remote host key (run on a trusted network)
ssh-keyscan -p 22 -H remote-host >> ~/.ssh/known_hosts
# 2. Verify the fingerprint out-of-band against the remote machine:
# on the remote, run: ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
# and confirm it matches what ssh-keyscan recorded.
# 3. Force strict checking for this host in ~/.ssh/config
cat >> ~/.ssh/config <<'EOF'
Host remote-host
StrictHostKeyChecking yes
EOFWith the key pinned and StrictHostKeyChecking yes, any future host-key change
(a real MITM, or a re-imaged server) causes the connection to fail loudly
instead of being trusted silently.
Config and profile files are sourced as Bash, so anything in them runs with
your privileges. Keep them private (chmod 600) and never sync a config file
from an untrusted source. Connection-related values (REMOTE_USER,
REMOTE_HOST, REMOTE_DIR, SSH_IDENTITY, BACKUP_DIR, EXCLUDE_PATTERNS)
are additionally validated at startup and rejected if they contain shell
metacharacters, so a templated or shared config cannot smuggle commands into the
SSH/rsync calls. File names with embedded shell metacharacters are also quoted
safely before reaching any remote command.
~/.config/rsync-sync/
├── config # Default configuration
├── profiles/
│ └── work.conf # Named profile configs
├── state/
│ ├── default.manifest # Sync state for default profile
│ └── default.lock # Lock file
└── logs/
└── sync-default-*.log # Sync logs
# Sync every 30 minutes
*/30 * * * * $HOME/.local/bin/sync-client 2>&1 >> /tmp/sync-cron.logSee docs/USAGE.md for systemd timer setup instructions.
- Usage Guide - Detailed usage examples and automation
- Troubleshooting - Common issues and solutions
MIT