Copying 7TB of Lightroom Photos

For years I’ve been storing all my photos directly on my Synology NAS to save space on my Macbook and editing them through a VPN connection while remote or over ethernet while local. It worked well enough when I was home, but became painful or unusable when away. Lightroom would lag, file browsing was slow, and accessing new imports could take minutes.

I decided to try and flip the model so I have easier access while on the go.

Instead of treating my NAS as the active working directory, I now use two 5TB HDD’s as my portable, always-available photo libraries. I back them up to the NAS periodically, but the drives are the daily drivers.

That meant copying over about 7TB of data from the NAS in a smart, verifiable way. This is easier said than done, if you want to do it in a reasonable amount of time. It took me about two days of trail and error and optimizing my approaches. I think it would have taken much longer and introduced many more bottlenecks had I not followed my final approach. Here’s how it went.

Initial Attempts: Drag & Drop via SMB

At first, I connected one of the HDD’s directly to my Mac and began copying photos using Finder. Simple, right?

Speed started around 100MB/s but dropped dramatically after a few hundred gigabytes — falling below 10MB/s in some cases.

Pros

• Easy to get started
• No setup required

Cons

• Unreliable speeds
• No logging or visibility into what’s copied or failed
• No way to resume or verify

NFS Mounts: Drag & Drop via NFS

Next, I tried mounting the NAS as an NFS share on my Mac and copying files to the drive.

Performance was worse than the USB approach, and the Finder didn’t give much feedback. Speeds hovered around 10–15MB/s, sometimes lower.

Direct Copy via NAS USB: rsync with Restart Logic

Eventually, I connected the HDD directly to the NAS via USB and SSH’d in.

I wrote ran rsync top copy files from the target directories, but found that the NAS itself quickly became the bottleneck. Disk write performance degraded, and metrics like Write Await and KB written per second revealed high disk wait times and reduced throughput.

I modified the script to restart when those metrics declined.

# Pseudo logic
1. Start rsync
2. Every 60s, measure bytes copied
3. If speed < threshold, kill and restart rsync

This got me back to ~100MB/s speeds.

However, as the transfer progressed, the restart script was firing more frequently as I waited for the NAS to become idle, further increasing overhead.

Pros

• Fast (USB direct connection)
• Resilient to slowdowns
• Fully automated

Cons

• Needed SSH access
• Couldn’t close SSH session without killing the job
• NAS became a bottleneck under sustained load

Running rsync from Docker on the NAS

To fix the SSH persistence issue, I built a tiny Docker image that ran the restart script inside a container:

FROM debian:bullseye
RUN apt update && apt install -y rsync iostat
COPY rsync_auto_restart.sh /usr/local/bin/
CMD ["/usr/local/bin/rsync_auto_restart.sh"]

Mounted the photo folders in with volumes and kicked it off in detached mode:

docker run -d \
  -v /volume1/photo:/data/source \
  -v /volumeUSB1/usbshare1-2:/data/dest \
  -v /volume1/homes/myuser/Logs:/logs \
  rsync-runner

Pros

• Fully independent from SSH
• Easy to restart or monitor

Cons

• Slightly lower performance (~80MB/s)
• Harder to interact with logs or pause
• Docker overhead

Final Approach: rsync from Mac over SMB

Ultimately, I moved back to my Mac and ran rsync over an SMB mount from the NAS.

rsync -avh --progress \
  --exclude='@eaDir/' --exclude=... \
  "/Volumes/photo/2023/" "/Volumes/LaCie/2023/"

To improve reliability, I used a modified version of my auto-restart script, but monitored local write speeds via du instead of iostat.

Performance settled around 50–90MB/s, which was plenty fast for unattended syncing.

Comparing All Methods

Verifying File Integrity

After syncing terabytes of data, I wanted to confirm that the file structure and contents were identical between the NAS and the LaCie drives. I used find, sed, and diff to generate relative file lists and compare them.

find /Volumes/2024/2025 -type f \
  ! -name '*.DS_Store' \
  ! -name '*.xmp' \
  ! -name '@eaDir/' \
  ! -name '@SynoResource/' \
  ! -name '@SynoRecycle/' \
  ! -name '.AppleDouble/' \
  ! -name '.DS_Store' \
  ! -name 'Thumbs.db' \
  ! -name '._*' \
  ! -name '.Spotlight-V100/' \
  ! -name '.Trashes/' | sort > drive_2025.txt

find /Volumes/photo/2025 -type f \
  ! -name '*.DS_Store' \
  ! -name '*.xmp' \
  ! -name '@eaDir/' \
  ! -name '@SynoResource/' \
  ! -name '@SynoRecycle/' \
  ! -name '.AppleDouble/' \
  ! -name '.DS_Store' \
  ! -name 'Thumbs.db' \
  ! -name '._*' \
  ! -name '.Spotlight-V100/' \
  ! -name '.Trashes/' | sort > nas_2025.txt

sed 's|.*2025/||' nas_2025.txt | grep -v '@eaDir/' | sort > nas_relative_2025.txt
sed 's|.*2025/||' drive_2025.txt | grep -v '@eaDir/' | sort > drive_relative_2025.txt

diff nas_relative_2025.txt drive_relative_2025.txt

For good measure, I counted the lines in each file too

➜  checks wc -l drive_relative_2025.txt
wc -l nas_relative_2025.txt
    5702 drive_relative_2025.txt
    5702 nas_relative_2025.txt

This process gave me confidence that the files on both drives matched the NAS exactly.

The Copy Script

One of the most important pieces of this project was a custom script to make rsync more robust. I wanted to:

• Monitor transfer speed in real time
• Restart rsync if it slowed down too much
• Let the disk rest between restarts to recover write performance

Here’s how the script is structured:

1. Define Paths and Thresholds

SOURCE="/Volumes/photo/2022/"
DEST="/Volumes/LaCie/2022/"
SPEED_THRESHOLD_MB=30     # Minimum MB/s before restart
INTERVAL_SEC=60           # Time interval between speed checks

The script monitors the copy operation every 60 seconds and expects at least 30MB/s. If performance drops below that, rsync is restarted.

2. Start rsync with Exclusions

rsync -avh --progress \
  --exclude='@eaDir/' \
  --exclude='.DS_Store' \
  --exclude='Thumbs.db' \
  "$SOURCE" "$DEST" \
  --log-file="$RSYNC_LOG" &

This command kicks off the rsync operation in the background and logs output to a file for later analysis. It also skips common hidden files from macOS and Synology.

3. Monitor Disk Write Activity

iostat -d disk2 1 2 | awk '/disk2/ {print $7}' | tail -1

To avoid hammering the disk, the script waits until it has been idle (under 1MB/s write) for at least 2 checks in a row before restarting the rsync process.

4. Wait for Disk to Become Idle

if [ "$write_kbps" -lt "$low_threshold_kbps" ]; then
  idle_checks=$((idle_checks + 1))
else
  idle_checks=0
fi

To avoid hammering the disk, the script waits until it has been idle (under 1MB/s write) for at least 2 checks in a row before restarting the rsync process.

5. Calculate Transfer Speed with du

bytes_before=$(du -sk "$DEST" | awk '{print $1}')
sleep "$INTERVAL_SEC"
bytes_after=$(du -sk "$DEST" | awk '{print $1}')

Every minute, the script calculates how much data was transferred, then computes a MB/s transfer rate.

6. Restart rsync If Performance Drops

if (( mbps < SPEED_THRESHOLD_MB )); then
  kill $RSYNC_PID
  wait_for_disk_idle
  run_rsync
fi

If performance is too slow, the current rsync process is killed, the disk is allowed to rest, and rsync is restarted cleanly.

7. Run in an Infinite Loop

while true; do
  check_speed
done

The script loops forever, restarting rsync when needed, and logging all events to a file.

Wrapping up

My goal was to give myself the ability to edit on the go. I chose HDD’s because of the price, and being unsure whether this new workflow will take root.

Besides doing that, it also gives me another hard copy of the photos stored on my NAS, which might be redundant, but that’s the point.

I’m not an expert with shell scripts, so I relied on ChatGPT to fill the gaps and it did so amazingly. It took me about half the time to iterate through all the possible perumatations to improve speed as it would have otherwise.

I’ll give it a go and see what happens!

🔗 Download the Script →

You can view or download the full version of the script here and modify it to fit your own workflow.