Bittorrent block devices as "poor mans multicast" disk cloning

Caveat: This has been proven in principle by cloning a single 100MB 
partition from one machine to another on an 'airgapped' network only. 
There may be issues (that I hope to have worked out tomorrow) expanding 
this to multiple partitions, specifically a Windows 7 install on 2 NTFS 
partitions, and there may be issues with doing this on the live Otago 
Uni network (I'll have to talk to the ITS networking and security teams 
in a couple of days).

I had a bit of time last week to work on a little project that I'd been 
meaning to do: get bittorrent based disk cloning via a USB Live RAM 
installation of Ubuntu (13.10). The main motivation for getting this 
working is that standard multicast techniques are 'contra-indicated' by 
the Otago Uni network (unregulated multicast traffic gets delivered to 
all machines on the LAN by definition and some machines *really* don't 
like multicast traffic and die a horrible death). If this doesn't make 
it onto the Uni network and stays just on a separate network then its 
only advantage over a multicast solution on the same separate network is 
the option to be a bit more ad-hoc as you can add new machines to the 
torrent swarm at any time (compared to multicasting where you get all 
your ducks in a row before you start).

First, the streamlined boot process:
=====================

This is kicked off by the following syslinux config option:

LABEL Reece
     Menu label ^0) Run Reece's Menu from 64-bit Ubuntu 13.10 Live terminal
     TEXT HELP
         home-rw contains .bash_profile login script to run /cdrom/rmenu 
which unmounts it
     ENDTEXT
     KERNEL /ubuntu/casper/vmlinuz.efi
#Copied from syslinux "Run Ubuntu 13.10 (64-bit)" and added casper 
option 'text' + home-rw loop filesystem in the root of the USB
     append  persistent file=/cdrom/ubuntu/preseed/ubuntu.seed 
boot=casper text initrd=/ubuntu/casper/initrd.lz ignore_uuid 
live-media-path=/ubuntu/casper locale=en_US.UTF-8 --

There is a 1MB loop filesystem in the root of the USB that is called 
home-rw that if it exists is mounted as /home. This contains one file of 
importance, the one line /home/linux/.bash_profile:
. /cdrom/rmenu

The USB is mounted as /cdrom so the rmenu mentioned is a script in the 
root of the cd that just copies a series of scripts from the USB and 
displays a menu so that the scripts can be easily changed.:
#!/bin/bash
#In this case the USB drive is mounted as /cdrom by the Ubuntu Live boot 
process as it is the same one the squashfs is on.
mntpoint=/cdrom
#This is where we copy the scripts off the USB stick to (assumed to be 
in the path)
pth=/usr/local/bin
sudo cp $mntpoint/scripts/* $pth
#Note that the r and u scripts must be run as in the same environment as 
the standard command line so add an alias
alias r='. r'
alias u='. u'
echo "Copied scripts from $mntpoint/scripts to $pth"
menu

The u script ("Unmount 1MB persistent home folder") is used to unmount 
the /home loop filesystem when it isn't needed:
#!/bin/bash
# Note for this to work correctly it needs to be called as '. u' rather 
than just 'u'
# The rmenu script sets up an alias for this reason.
cd /
sudo umount -l /home
echo Sleeping for 5 seconds after lazy unmount of persistent home loop 
file system
sleep 5
sudo mkdir /home/linux
sudo chown linux:linux /home/linux
cd ~

The r script ("Reset persistent home folder") is used to reset the 
persistent home folder to the pristine copy, also stored in the root of 
the USB:
u
sudo rm /cdrom/home-rw
sudo cp /cdrom/home-rw-copy /cdrom/home-rw

Now, the actual bittorrent stuff:
===================
Initially I thought about trying to mount the file system and create a 
standard torrent for all the files on the partition. This worked fine 
for files and folders but not for other types such as symlinks and FIFO 
pipes. When a Windows 7 NTFS partition is mounted, <JUNCTION> folders 
such as Application Data (linked to AppData) are shown as symlinks and 
some files under a users CryptnetUrlCache folder are shown as FIFO 
pipes. There is also the added disadvantage that NTFS formatted 
partitions contain ACLs for each file/folder and so they would also have 
to be recorded and applied separately.

Instead I went for the option of creating a torrent for the block device 
(so far only tested on a simple 100MB /dev/sda1). Most, if not all, 
torrent programs deal only with files and folders, not block devices, 
but I found a diff from 2003 that added the capability to BitTornado, a 
python bittorrent suite: 
http://osdir.com/ml/network.bit-torrent.general/2003-12/msg00356.html

In the case of python at least, the main difference between standard 
files and block devices is the way to get the size. From this I found a 
fork of bittornado on GitHub: https://github.com/effigies/BitTornado (a 
couple of commits have happened in the last few days so the exact link 
is 
https://github.com/effigies/BitTornado/tree/73abe68ac13bda31db5307db8c2d0ba9e58bbfac) 
and changed 2 files:

1)To create a torrent of a block device I had to change the BTTree.py file

BTTree.py changes, in context:
==================
under def __init__

if os.path.isfile(loc):
             self.size = os.path.getsize(loc)
         #This elif added by Reece
         elif S_ISBLK(os.stat(loc).st_mode):
             f = open(loc, 'rb');
             f.seek(0,2)
             self.size= f.tell()
             print("Block device detected. Size is %s" % self.size)
             f.close()
         # We'll need to know the size of all subfiles
         elif os.path.isdir(loc):
             for sub in sorted(os.listdir(self.loc)):
                 # Ignore .* (glob, not regex)
                 if sub[0] == '.':
                     continue
                 sloc = os.path.join(loc, sub)
                 spath = self.path + [sub]
                 try:
                     self.subs.append(BTTree(sloc, spath))

                 # Notify, but ignore entries that are neither
                 # files nor directories
                 except IOError as problem:
                     print problem

             # For bittorrent's purposes, size(dir) = size(subs)
             self.size = sum(sub.size for sub in self.subs)
         else:
             raise IOError("Entry is neither file nor directory: " + loc)


Also, just for cosmetic purposes:

     def addFileToInfos(self, infos):
         """Add file information and data hash to a sequence of Info
         structures"""
         with open(self.loc, 'rb') as fhandle:
             pos = 0L
             piece_length = 0
             for info in infos:
                 piece_length = max(piece_length, info.hasher.pieceLength)
                 info.add_file_info(self.size, self.path)

             while pos < self.size:
                 nbytes = min(piece_length, self.size - pos)
                 buf = fhandle.read(nbytes)
                 pos += nbytes
                 #Added by Reece so can tell where in the file we are as 
the ordinary complete percentage only updates after each file is completed
                 #Note that there are four tabs here so the standard % 
complete still shows
                 print("                Current file is %.1f%% complete. 
%s out of %s bytes\r" % ((float(pos)*100)/float(self.size),pos,self.size)),
                 for info in infos:
                     info.add_data(buf)


2) To allow for checking when running the bittorrent client I changed 
the BT1/Storage.py file:

Storage.py change in context:
==================
Under def __init__
                     if exists(file):
                         #Added by Reece
                         f=open(file,'rb')
                         f.seek(0,2)
                         l=f.tell()
                         f.close()
                         #Reece commented the below out
                         #l = getsize(file)
                         if l > length:
                             with open(file, 'rb+') as h:
                                 h.truncate(length)
                                 h.flush()
                             l = length

The actual scripts for bittorrenting:
======================
For the first master/seed machine, set it up as the tracker as well, so 
run b+e+g+h
For all but the first run b+g+h
(script names will change once I get it working so I can delete the 
non-used scripts and consolidate the working ones but I needed some 
names to put here)

------ b ----

# Copy the bittorrent folder with all the files from the USB drive to 
the /tmp folder
cp -R /cdrom/bittorrent /tmp

------ e -----
#Should have been able to get BitTornado tracker working but instead 
went with
# the one in the Ubuntu repos (downloaded by a 13.04 desktop, copied the 
.deb files from /var/cache/apt)
sudo dpkg -i /tmp/bittorrent/python-bittorrent_3.4.2-11.4ubuntu2_all.deb
sudo dpkg -i /tmp/bittorrent/bittorrent_3.4.2-11.4ubuntu2_all.deb

ipaddy=`ifconfig eth0 | grep "inet addr:" | cut -f 2 -d ":" | cut -f 1 
-d " "`

echo Creating torrent with tracker $ipaddy
cd /tmp/bittorrent/BitTornado-master/
sudo ./btmakemetafile.py http://$ipaddy:80/announce /dev/sda1 --target 
/tmp/currentimage.torrent

echo Starting Tracker on $ipaddy
sudo bttrack --bind $ipaddy --port 80 --dfile /tmp/dstate --logfile 
/tmp/tracker.log &

echo Copying torrent file to USB and RAM cache
sudo cp /tmp/currentimage.torrent /cdrom/bittorrent/currentimage.torrent
sudo cp /tmp/currentimage.torrent /tmp/bittorrent/currentimage.torrent
sudo rm /tmp/currentimage.torrent
sudo sync

------ g ------
# Unmount the USB so can take it to a different machine
sudo sync
sudo umount -l /cdrom
u

------ h ------
cd /tmp/bittorrent/BitTornado-master/
#For some reason the Python script looks for the icons in this folder 
rather than in the icons subfolder
# even though we aren't actually using any of them anyway
cp icons/* .
sudo ./btdownloadheadless.py /tmp/bittorrent/currentimage.torrent 
--saveas /dev/sda1


Other things I will be testing tomorrow:
========================
These are mainly specific to my context of cloning 2 NTFS partitions 
that are a Windows Sysprep'd install...
For the master/seed machine:
- Copy the Windows 7 Autounattend.xml file and $OEM$ additional files to 
the C: drive
- resize /dev/sda2 to minimum size using ntfsresize and fdisk script
- Copy sfdisk info and mbr to files for later restore (under 
/cdrom/bittorrent folder)
------------------------------------
For the rest:
- Restore sfdisk partitions + mbr from /tmp/bittorrent folder after 
wiping current partition table
------------------------------------
And the final thing to do for all machines:
- Resize /dev/sda2 to maximum size + shutdown

(For future-travellers looking at this on an archived forum, you will be 
able to find the followup email by putting in the current email title + 
followup into a search engine)

-- 
"Believing men would act in their own interest was not cynicism, it turned out, but sheerest optimism; in reality men do not meet so high a standard."
-- Harry Potter and the Methods of Rationality (Chapter 84)
http://hpmor.com/

Reece Arnott
Dunedin
New Zealand


_______________________________________________
DunLUG mailing list
[email protected]
http://lists.ethernal.org/listinfo/dunlug
DunLUG Wiki - http://dunlug.kallisti.net.nz/