Make a drive mirror on linux
A very brief post to quickly describe what I had to do to have an uncompressed automatic backup of my external hard drive on to another external hard drive of the same capacity.
I have an external USB 1TB hard drive where I store all media files from movies and music to family photos. Once I started having a pretty decent number of family photos I started to think that a backup was definitely in order so I got another 1TB external USB hard drive where I wanted to backup my data to. Here's what I wanted, along with the options that I considered, why they work or not for my application and finally my actual solution for this need:
So, from the table above you can see which of the options was right for me. I ended up using the following command using rsync:
rsync -avh --delete /path/to/localfiles /path/to/externaldrive >>rsyncLog.txt 2>>rsyncErr.txt
This command tells rsync to copy the files in (-a) archive mode, basically keeping all the same as the original in terms of symbolic links, devices, attributes, permissions and ownership. It requests an increse in verbosity (-v), i.e. more information is generated by the program (useful for debugging and detecting potential problems and all numbers appear in human-readable format (-h). I additionally pipe the program output to two files, one is a simple log of actions taken and the other contains any potential error messages generated. The --delete option deletes files from the destination in case they have been deleted on the origin.
I have an external USB 1TB hard drive where I store all media files from movies and music to family photos. Once I started having a pretty decent number of family photos I started to think that a backup was definitely in order so I got another 1TB external USB hard drive where I wanted to backup my data to. Here's what I wanted, along with the options that I considered, why they work or not for my application and finally my actual solution for this need:
Requirements | RAID1 | Simple Backup | rsync |
I do not need the files to be compressed since most of the files on my main HD are media and compressed anyway. I don't want to loose time with compression. | OK | I think it compresses by default | OK |
I want to be able to remove anyone of the disks and the data is still available on the local computer. | OK | OK | OK |
I want to make incremental backups, i.e. data is only sync'ed not entirely copied over each time. | OK | OK | OK |
I'm not really interested in keeping several versions of the same files. What I delete on the main, gets deleted on the backup drive, what gets added to the main disk, gets added to the backup. | OK | OK | OK |
Any of the drives can just be unmounted and used on another machine with no extra software. | NO | OK | OK |
rsync -avh --delete /path/to/localfiles /path/to/externaldrive >>rsyncLog.txt 2>>rsyncErr.txt
This command tells rsync to copy the files in (-a) archive mode, basically keeping all the same as the original in terms of symbolic links, devices, attributes, permissions and ownership. It requests an increse in verbosity (-v), i.e. more information is generated by the program (useful for debugging and detecting potential problems and all numbers appear in human-readable format (-h). I additionally pipe the program output to two files, one is a simple log of actions taken and the other contains any potential error messages generated. The --delete option deletes files from the destination in case they have been deleted on the origin.
As I final step I use cron to run the command above everyday at 04:00am. To simplify I use gnome-schedule to add the command and set the trigger time.
It all works well. Hope this helps someone.
Comments
Great blog. Do you have an email I can contact you on? I have a few questions. My email is rgibson(a)farnell(dot)com
Really keen to hear from you,
Ryan