Category: Development

I maintain several websites and needed a way to regularly backup the site content to my local machine (where time machine provides the next backup layer). Some sites can only be accessed via (S)FTP, for some I have SSH access. So I created a small bash script to back up all servers . This script works with rsync, and lftp.

The script runs on OSX, but it should be easy to adapt it to any Unix platform by replacing the "osascript"-calls with something else.

Please note: This is not a backup solution which works out of the box, but an example script you can modify to fit your own needs. Bash knowledge is required.

How it works:

• Each backup job is defined by a bash function whose name starts with "backup…"
• To run a single backup job: "cw-backup.sh backupSomesite"
• To run all backup jobs: "cw-backup.sh all"
  
• The backups are saved in subfolders of the folder given by $backupDir

Backup via (s)ftp:

The mirror function of lftp will be used. Here is an example method:

backupMyblog()
{
	_lftpBkup \
	srcHost='www.myblog.example' \
	srcDirs='/downloads/,/data/' \
	opts='--exclude=cache/ --exclude-glob=*.zip' \
	targetDir='myblog'
}

Set Hostname (srcHost), the folder name for this backup (targetDir) and the directories which should be backed up (srcDirs). Multiple directories can be separated with a comma.

Additionally you can add specific options (opts).

The username and password for FTP-access are taken from the ~/.netrc file (see in the man page or here https://stackoverflow.com/a/42787829)

Backup via rsync:

Here is an example:

backupMyproject()
{
	_rsyncBkup \
	srcHost='user@myproject.example' \
	srcDirs='/var/www/myproject/data' \
	port='22' \
	opts='--exclude=**/.cache --exclude=*.gz' \
	targetDir='myproject'
}

Set Hostname (srcHost), the rsync-Port (port), the folder name for this backup (targetDir) and the directories which should be backed up (srcDirs). Multiple directories can be separated with a comma.

Additionally you can add specific options (opts).

Configuration

In the "settings" part of the script you should set the location of the backup folder. You can also set the locations for lftp and rsync and default options for those commands.

Periodic execution

I use launchd on OSX to execute the backup periodically.

The script

Find the source code here:
https://bitbucket.org/mariofischer/cw-backup/

Settings

# backup root dir, must end with /
backupDir="$HOME/Projekte/_example-backup/"

# ltfp binary
lftpBin="/usr/local/bin/lftp"

# lftp mirror options
lftpMirrorOptions=" --verbose=1 --skip-noaccess --parallel=5 --delete --only-newer "

# lftp command start
lftpCmdStart="set ssl:verify-certificate false && set cache:enable true && set dns:cache-enable true && set ftp:use-feat off "

# rsync binary
rsyncBin="/usr/bin/rsync"

# rsync options
rsyncOptions=" -a -v --partial "