This is a program that uses rsync and (possibly) ssh to perform very space and bandwidth efficient rotating backups in a push fashion.
The fundamental technique is described at http://www.mikerubel.org/computers/rsync_snapshots/
The major difference of this script is that it is push instead of pull. It appears all the other scripts implementing this technique are pull, which just doesn't work for me.
Here's the Script
Note that this script is currently ALPHA CODE! I'm using it on several production machines and to back up *very* critical data, but I haven't by any means fully exhausted it's failure modes.
I'm *very* interested in fixing the bugs (this is my current project as of Nov 2005) so I'd appreciate any feedback you have for me and particularly any bug reports. Bug reports I'll try to get to quickly. Feature requests will wait until the next time I'm hacking in that direction. Please read the page on bug reports before sending them.
Patches are always welcome.
Oh yeah, license. I'm going to pick some OSS approved license. I really need to review the current set and pick the best. If you care about the exact license let me know and I'll get the lead out...
Here's the man page:
RSYNC-BACKUP(1) User Contributed Perl Documentation RSYNC-BACKUP(1)
NAME
rsync-backup -- Perform a rotating, hard-linked rsync backup to a local
or remote system
SYNOPSIS
Status: Alpha code
rsync-backup [-debug] [-noexec] [-man] [-help] [-verbose] -dest dest
[-source source] [-remote-program-path
path-on-remote-host]
DESCRIPTION
WARNING: This code is alpha. I use it for my own backups right now
but I haven't put a lot of thought/time into error conditions. Please
pay attention to what it's doing and report any surprises.
rsync-backup performs a rotating, hard-linked rsync backup to a local
or remote system per the strategy described at
http://www.mikerubel.org/computers/rsync_snapshots/
The major difference is that in this case the client initiaiates all
connections so this is suitable to make the backup system a (more or
less) passive receiver. In other words, this is a push backup system,
not a pull backup. There are already solutions for the pull case.
The receiver must have an installation of this script on the path under
the program name *or* the location of the path must be specified with
the -remote-program-path option.
As the name suggests, this program relies on rsync to perform the
actual transfer and hardlinking. You must have a version of rsync that
supports --link-dest (2.5.6 or greater) However, both bandwidth and
storage space will be well optimized.
The destination directory will have a series of subdirectories named
with prefix (defaults to "backup-") and a date. Each backup that com-
pleted will contain a file called ".backup.FINISHED". This program
requires no state beyond the datestamp of the .backup.FINISHED file.
Specifically, the names of the backup directories do not matter as long
as they match the appropriate prefix, which can be ''.
Each directory will have a full copy of your backed up files as of that
date. Any files which were unchanged from the previous version will
actually be a hard-link (both Unix and Win32) to the previous version.
Theoretically you could keep 15 years of backups with only constant
space if the file never changes. This is great for storage efficiency,
but you must make sure that the backup doesn't have a bad disk block or
every one of those 15 years of backups will be unrecoverable. In other
words, these backups can protect you on remote systems and from your
own errors but do not contain sufficient redundency to help recover
from disk errors on the backup system.
Right now the rotation schedule is hard coded. I believe this rotation
schedule will work for anything from keeping up to the minute backups
to daily/weekly backups. If you don't want to keep as many revisions
as this program keeps a cron job doing "rm -rf" as appropriate will
work well to prune further.
The backup program always keeps the most recent version plus the oldest
copy from each of the following windows (which begin at the end of the
previoius window and last until):
last minute
last 5 minutes
last 15 minutes
last 30 minutes
last hour
last 2 hours
last 4 hours
last 8 hours
last day
last 2 days
last 3 days
last week
last 2 weeks
last 3 weeks
last 4 weeks
last 90 days
last 6 months
last year
last 1.5 years
last 2 years
last 3 years
last 4 years
last 5 years
last 7 years
last 10 years
last 15 years
In other words, if you've been doing backups every 30 seconds for the
last 20 years, you will have copies of your files from...
30 seconds ago, 1 minute ago, 5 minutes ago, 15 minutes ago, 30 minutes
ago, 1 hour ago... up to 15 years ago.
If, on the other hand, you've been doing *daily* backups for the last
20 years you'll have...
1 day ago, 2 days ago, 3 days ago, something between 4 and 7 days ago,
something between 7 and 14 days ago, ...
OPTIONS
-dest dest
Destination directory. This can be an ordinary path (e.g. /back-
ups/foobackup) or a ssh style remote host/path (e.g. backups.exam-
ple.com:/backup/foobackup)
-source source
The source
-verbose
Be a bit verbose while running. The default is to say nothing
unless there's an error.
-debug
Turn on debugging output
-noexec
Do not execute state changing commands
-man
Show this page
-help
Show a brief summary of options
-remote-program-path path-on-remote-host
This should be a full path to the rsync-backup (including file!) on
the destination server if the server is remote. This is never used
if you're performing local backups.
EXAMPLES
rsync-backup -dest /tmp/lh-backup
Backs up the current directory to /tmp/lh-backup
rsync-backup -dest /tmp/lh-backup -source ~/tmp/testing
Backs up ~/tmp/testing to /tmp/lh-backup
rsync-backup -dest home:/tmp/lh-backup -source ~/tmp/testing
Backs up ~/tmp/testing to /tmp/lh-backup on "home" (as resolved
through ssh)
FILES
Expects directories and files of a certain structure in the destination
area.
Expects the destination area to previously exist.
ENVIRONMENT
Requires PERL and a copy of this script on both client and server sys-
tems.
BUGS AND CAVEATS
I'm quite sure there are bug, mostly in the realm of incorrect assump-
tions I've made or conditions I've failed to consider. Please email me
all bug report. Be as specific as possible.
I'm sure I don't handle error conditions well. If you'd like to put
some thought into how I can simulate error conditions for testing pur-
poses that would be cool, too.
LICENSE
I have not yet decided on a license for this code. I will choose an
OSS license that allows you free use. Contact me if you need this
spelled out right now.
SEE ALSO
http://www.mikerubel.org/computers/rsync_snapshots/
AUTHOR
Dewey Sasser <dewey@sasser.com>
perl v5.8.7 2005-10-16 RSYNC-BACKUP(1)