filesystem snapshots with Linux (was: missing files from incremental backups)

* Joerg Schilling writes:
 
> In any case, there is no backup method that is able to deal with active 
> filesystems. This is why snapshots have been implemented.
> 
> 
> > I will use filesystem snapshots then. I am new to the snapshots business and 
> > noticed that Linux offers several techniques. Do you have any recommendations
> > which one to use?
> 
> I am not sure whether Linux suppots them. The last time I tried to find a 
> matching technoligy in Linux, there was none.
> 
> I know that 
> 
> FreeBSD
> NetApps
> Solaris
> 
> have snapshots.

Linux can do snapshots of logical volumes. They are part of a concept called 
LVM (logical volume management). This is the first time a came across this, 
so my knowledge about it is rather basic. One has to create volume groups (at 
least one) and inside those volume groups, one can create logical volumes.  
Logical volumes are available under /dev/mapper/<volgroup>-<logvol> and also 
under /dev/<volgroup>/<logvol>. There are several tutorials out there on how 
to create these, and I will not replicate them here.

For snapshots, it is important that the volume group in question still has 
some free space. This space will be used for the snapshot, obviously to 
compensate for changes made to the volume during the existence of the 
snapshot. Doing an incremental dump of a logical volume goes as follows:


1. Take a timestamp.

  rm -f dumpdate
  touch dumpdate

Many tutorials out there tell you to *freeze* the filesystem at this time. 
With recent kernels, this is no longer necessary. On the contrary - it will 
make the following steps fail! So, do not freeze the filesystem manually.


2. Create the snapshot.

  lvcreate -L "${size?}" -s -n snap /dev/"${volgroup?}"/"${volume?}"

Environment variables used:
size - the size used for the snapshot. The volume group must have at least 
       that much of free space.
volgroup - the name of the volume group.
volume - the name of the logical volume inside of the volume group.

The snapshot itself will be a logical volume called `snap'. If this is not 
desired, change `snap' to something else in the above command line.


3. Mount the snapshot.

  mount -o ro,noatime,nosuid /dev/"${volgroup?}"/snap /mnt/snap

This mounts the snapshot to `/mnt/snap'.


4. Do the dump with star.

  star -c -xdev -sparse -acl -link-dirs level="${level?}" dumpdate=dumpdate \
  -wtardumps f=level-"${level?}".star -C /mnt/snap . 

This writes a dump with level `${level}' to the file `level-${level}'.


5. Unmount and remove the snapshot.

  umount /mnt/snap
  lvremove -f /dev/"${volgroup?}"/snap



That's it. Attached you find a script that puts it all together with some 
extras. I used this script for testing.

During my tests, I noticed that disk access can become very slow, sometimes 
close to unresponsive, during the procedure if there is much activity on the 
logical volume which is dumped. My impression was that this effect is 
stronger with ext3 than with reiserfs, but this might be a false impression.


Lasse

_______________________________________________
Star-users mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/star-users

#!/bin/sh

#
#
# usage: snapstar <mount_point> <volume_group> <volume> <snap_size> <level>
#
# E.g.: to do a level 2 backup of `/dev/vg1/vol1' mounted on `/fs/main/mount'
# using 500m for the snapshot:
#
# snapstar /fs/main/mount vg1 vol1 500m 2
#
# There must not be a volume called `snap' in the volume group already!
#
# The backups are put in files `level-<level>.star' in the current directory.
# A file `dumpdate' in the current directory is used.
# A directory `snap' in the current directory is used to mount the snapshot.
#
#


: "${5?}" &&
mount="${1?}" &&
volgroup="${2?}" &&
volume="${3?}" &&
size="${4?}" &&
level="${5?}" &&

case "${level?}" in
	[0123456789][0123456789]) paddedlevel="${level?}" ;;
	[0123456789]) paddedlevel=0"${level?}" ;;
	*) echo "invalid level: ${level}" ; exit 100 ;;
esac &&

rm -f dumpdate &&
touch dumpdate &&
day=`date +%F` &&
time=`date +%R` &&
host=`hostname` &&

lvcreate -L "${size?}" -s -n snap /dev/"${volgroup?}"/"${volume?}" &&
mkdir -p snap &&
mount -o ro,noatime,nosuid /dev/"${volgroup?}"/snap snap &&

star -c -xdev -sparse -acl -link-dirs level="${level?}" dumpdate=dumpdate -wtardumps \
	VOLHDR="level ${level?} of ${mount?} on ${host?} of snapshot taken at approx. ${day?} at ${time?}" \
	-bz \
	f=level-"${paddedlevel?}".star fs-name="${mount?}" -C `pwd`/snap . &&

umount snap &&
lvremove -f /dev/"${volgroup?}"/snap

case "$?" in
	0) : ;;
	*)
		echo
		echo "!!! an error occurred; cleaning up !!!"
		umount snap
		lvremove -f /dev/"${volgroup?}"/snap
		exit 111
		;;
esac

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFGn0UG1gObwed86AkRAvKjAJ0Yg4JHgdX+5suh3dtgdOr2vKzbGgCeI+y5
o+YbJAa480LT7kMTwwMPVsU=
=GSBe
-----END PGP SIGNATURE-----