Backing up VirtualBox VMs

This script below is what I use to make backups of running Windows VMs, they are put into hibernation, then the drive is imaged, then the machine is woken back up. There is downtime with this method, but not much and it is easy to schedule as a low use-time service.


#!/bin/bash
##
## Destination of backup files
## Script won't run if directory doesn't exist, and please note that this runs with a normal
## users privileges, so the $BACKUPDEST must be writable by the user running the VM.
BACKUPDEST="/Backup/VMs/${USER}"

## How many days a compressed version will be left on server
DAYS_TO_KEEP_TAR="+7"

## Exempt VMs - Place each VM with a space seperating them
## Example: ( linux winxp win7 ) or
## ( "Ubuntu 8.04" "Windows XP" ) or
## ( None )
## More here: http://www.cyberciti.biz/faq/bash-for-loop-array/
## Run Command to find list of VMS:
## VBoxManage list vms | grep '"' | cut -d'"' -f2 2>/dev/null
EXEMPTION_ARRAY=( None )

## No need to modify below here
IFS=$'\n'
HOST=`hostname`
DATEFILE=`/bin/date +%Y%m%d`
VMLIST=`VBoxManage list vms | grep '"' | cut -d'"' -f2 2>/dev/null`

#################################################################
## Functions

##
## Notify the starting time of backup
##
function startScript {
echo "-----------------------------------------------------"
echo "START - ${VM}"
echo "Host: ${HOST}"
echo "Date: `date`"
echo "-----------------------------------------------------"
echo
}

##
## Create the backup directories if they do not exist
##
function doCheckDirectories {
## Check to see if BACKUPDEST exist
if [ ! -d ${BACKUPDEST} ]; then
echo "BACKUPDEST does not exist!! Exiting Program."
exit 0
fi
## If the archives directory does not exist, create it
if [ ! -d ${BACKUPDEST}/archives ]; then
echo "${BACKUPDEST}/archives directory does not exist, creating . . ."
mkdir "${BACKUPDEST}/archives"
echo
fi
## If the directories directory does not exist, create it
if [ ! -d ${BACKUPDEST}/directories ]; then
echo "${BACKUPDEST}/directories directory does not exist, creating . . ."
mkdir "${BACKUPDEST}/directories"
echo
fi
}

##
## If this VM is in our exempt array, set VM_EXEMPT to skip entirely.
##
function doCheckExempt {
VM_EXEMPT=false
## array, if we get a match, set VM_EXEMPT to true
for check_vm in "${EXEMPTION_ARRAY[@]}"; do
if [ "${check_vm}" = "${VM}" ]; then
echo "${VM} is on the exception list, skipping."
echo
VM_EXEMPT=true
fi
done
}

##
## Suspend VM if its running, skip it if not
##
function suspendVM {
## Check state of VM
VMSTATE=`VBoxManage showvminfo "${VM}" --machinereadable | grep "^\(VMState=\)" | cut -d'"' -f2 2>/dev/null`

echo "${VM} state is currently: ${VMSTATE}"

## If VM is running, suspend it, otherwise, move on
if [ "${VMSTATE}" = "running" ]; then
echo "Suspending ${VM} . . ."
## Stop vm by saving current state (pause|resume|reset|poweroff|savestate)
VBoxManage controlvm ${VM} savestate 2>/dev/null
[ $? ] && echo Success || echo Failure
echo "${VM} Suspended on `date`"
echo
else
echo "${VM} was not running, not suspending - `date`"
echo
fi
}

##
## Backup VM
##
function doBackup {
## Display location of XML file
XMLFILE=`VBoxManage showvminfo "${VM}" --machinereadable \
| grep "^\(CfgFile=\)" \
| cut -d'"' -f2 2>/dev/null`
## Display location of vdi file
## LOCATION=`VBoxManage showvminfo "${VM}" --machinereadable | grep ".vdi" | cut -d'"' -f4 2>/dev/null`
## Display location of vdi or vmdk file
LOCATION=`VBoxManage showvminfo "${VM}" --machinereadable \
| grep -e ".vdi" -e ".vmdk" \
| cut -d'"' -f4 2>/dev/null`

## If the directories directory does not exist, create it
if [ ! -d ${BACKUPDEST}/directories/${VM} ]; then
echo "${BACKUPDEST}/directories/${VM} does not exist, creating . . ."
mkdir "${BACKUPDEST}/directories/${VM}"
echo
fi
## Backup VM (clonehd)
echo "Backing up "${VM}" to ${BACKUPDEST}/directories/${VM}/"
rsync --inplace -a --stats "${XMLFILE}" "${BACKUPDEST}/directories/${VM}/"
[ $? ] && echo Success || echo Failure
rsync --inplace -a --stats "${LOCATION}" "${BACKUPDEST}/directories/${VM}/"
[ $? ] && echo Success || echo Failure
echo
}

##
## Start VM if suspened
##
function doStart {
if [ "${VMSTATE}" = "running" ]; then
echo "Starting ${VM} . . ."
## Resume VMs which were running [--type gui|sdl|vrdp|headless]
VBoxManage startvm ${VM} --type headless
echo "${VM} Resumed on `date`"
[ $? ] && echo Success || echo Failure
else
echo "${VM} was not running, not resuming - `date`"
fi
echo
}

##
## Making a compressed and mobile backup
##
function doTar {
fileName="backup_${VM}-${DATEFILE}.tgz"
echo "taring up ${VM} to ${BACKUPDEST}/archives/${fileName}"
tar -czf "${BACKUPDEST}/archives/${fileName}" "${BACKUPDEST}/directories/${VM}" 2>/dev/null
[ $? ] && echo Success || echo Failure
echo
}

##
## Clean up any tars or logs that are older than DAYS_TO_KEEP_TAR
##
function doCleanTar {
echo "Cleaning up tars older than ${DAYS_TO_KEEP_TAR} day(s) old"
find "${BACKUPDEST}/archives" -name "backup_${VM}*.tgz" -mtime ${DAYS_TO_KEEP_TAR} -exec rm -vf {} \;
[ $? ] && echo Success || echo Failure
echo "Cleaning up logs older than ${DAYS_TO_KEEP_TAR} day(s) old"
find "${BACKUPDEST}/" -name "*-log" -mtime ${DAYS_TO_KEEP_TAR} -exec rm -vf {} \;
[ $? ] && echo Success || echo Failure
}

##
## Notify the finishing time of backup
##
function finishScript {
echo
echo "-----------------------------------------------------"
echo "FINISH - ${VM}"
echo "Host: ${HOST}"
echo "Date: `date`"
echo "-----------------------------------------------------"
}

#################################################################
## Script

## Make sure we have the appropriate directories for backups
doCheckDirectories
## Start loop
for VM in ${VMLIST}; do
sleep 1
## Check exempt list
doCheckExempt
if [ "$VM_EXEMPT" = "false" ]; then
startScript
## Suspend VM
suspendVM
sleep 3
## Do Backup
doBackup
## Start if suspended
doStart
## Compressing backup
doTar
sleep 3
## Clean old backups
doCleanTar
sleep 3
finishScript
fi
## Reset exemption
shift
done >> ${BACKUPDEST}/${DATEFILE}-log
################################################################

Rsync to Fat32 drives

I regularly provide one of my clients with a backup of his data in the form of an external hard drive. Since his server runs CentOS Linux and his computer is a Windows machine, I need to provide a drive formatted Fat32 so he can plug it into his computer and access the data without problem.

# mount -t vfat /dev/sdc1 /mnt/usb -o shortname=mixed -o utf8

The “shortname=mixed” keeps the case preserved, as otherwise vfat will convert any filename that’s 8 characters or less to lower case (default behavior is “shortname=lowercase”) and cause problems for rsync. UTF8 is what Windows uses when mounting filesystems, so we specify that to ensure that we’re mounting it the same way (default is to mount iso-8859-1, even though the underlying vfat filesystem will store filenames in UTF8 format).

My normal mirror command, “rsync -az /home /mnt/usb“, doesn’t work because -a is a shortcut for the following options:

-a, –archive archive mode; same as -rlptgoD (no -H)
-r, –recursive recurse into directories
-l, –links copy symlinks as symlinks
-p, –perms preserve permissions
-t, –times preserve times
-o, –owner preserve owner (super-user only)
-D same as –devices –specials
-g, –group preserve group

Using -o will cause errors, as rsync will copy the file and then chown (change owner) the file. Fat32 doesn’t support Unix owership or permissions, so rsycn will error on every file that is copied. Ditto for -p and -g. Symlinks aren’t supported either, and we don’t want -L to copy the destination file of the symlink (that will produce multiple copies of a file/directory, not desirable in this particular instance). The -D option is irrelevant because we are only copying website data, so we don’t need special devices (/dev/*).

That leaves -r (recursive), -t (preserve times) for our vfat options. There’s no need to use compression (-z) since we’re not syncing across the network.

So the best command to copy form ext3 to Fat32 drive is something like this:

rsync -rtv /home /mnt/usb

I like using -v for verbosity, unless I’m running this within a shell script.

A good reference for further reading on Fat32 with Linux:
http://www.osnews.com/story/9681/The_vfat_file_system_and_Linux/