RE: svnsync large files

Julian Foad <[email protected]> Tue, 01 Dec 2009 13:37:31 +0000

Newsgroups	gmane.comp.version-control.subversion.devel,gmane.comp.version-control.subversion.rapidsvn.devel
Message-ID	<1259674651.26091.271.camel@edith>

On Mon, 2009-11-30, Bitts, Daniel wrote:
> Julian,
> 
> Here's a simple, reproducible test case:

Thank you very much. That will help. I have scripted it, but only with
"file:" URLs. Attached.

My script runs the sync fine in svn 1.6.5 with "file:" URLs. (I didn't
try 1.6.6.) Perhaps somebody else is willing to extend the script to use
HTTP, and then we will see the failure. I can't spare the time to do
that now.

- Julian

> 1. download a file 150Mb or larger.  I used a copy of the latest Ubuntu ISO for this test.
> 2. create an svn 1.6.6 repository under Apache 2.2.3 using the following subversion.conf file (make sure apache owns everything and you've configured your AuthUserFile correctly):
> LoadModule dav_svn_module     modules/mod_dav_svn.so
> LoadModule authz_svn_module   modules/mod_authz_svn.so
> 
> <Location /svn>
>    DAV svn
>    SVNPath /var/www/svn
>    SSLRequireSSL
>    AuthType Basic
>    AuthName "SVN"
>    AuthUserFile /var/www/svn_auth.dat
>    SVNPathAuthz off
>    Require valid-user
> </Location>

[...]

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2425873

Please start new threads on the <[email protected]> mailing list.
To subscribe to the new list, send an empty e-mail to <[email protected]>.

svnsync-large-files-1.sh (application/x-shellscript, 1.4 KB)

#!/bin/sh
# Test svnsync with a large file.
# From <http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2425682>.

SVN=/usr/bin/svn
SVNADMIN=/usr/bin/svnadmin
SVNSYNC=/usr/bin/svnsync

HUGE_FILE=huge.bin

set -ex

$SVN --version --quiet

# 1. Get a file 150Mb or larger
if ! [ -e $HUGE_FILE ]; then
  dd if=/dev/urandom of=huge.bin bs=1M count=150
fi

# 2. Create an svn 1.6.6 repository under Apache 2.2.3
$SVNADMIN create master-repo
MASTER_URL=file://`pwd`/master-repo
echo > subversion.conf '
LoadModule dav_svn_module     modules/mod_dav_svn.so
LoadModule authz_svn_module   modules/mod_authz_svn.so

<Location /svn>
   DAV svn
   SVNPath /var/www/svn
   SSLRequireSSL
   AuthType Basic
   AuthName "SVN"
   AuthUserFile /var/www/svn_auth.dat
   SVNPathAuthz off
   Require valid-user
</Location>
'

# Create a WC
$SVN checkout $MASTER_URL wc

# 3. Check in a simple one line text file
echo "Before" > wc/foo.txt
$SVN add wc/foo.txt
$SVN commit -m "" wc/

# 4. Check in the large file
cp $HUGE_FILE wc/
$SVN add wc/huge.bin
$SVN commit -m "" wc/

# 5. Check in another one line text file
echo "After" > wc/bar.txt
$SVN add wc/bar.txt
$SVN commit -m "" wc/

# Create a slave repo
$SVNADMIN create slave-repo
ln -s /bin/true slave-repo/hooks/pre-revprop-change
SLAVE_URL=file://`pwd`/slave-repo
# add "SSLProxyEngine on" and an SVNMasterURI

# Do the sync
$SVNSYNC init $SLAVE_URL $MASTER_URL
$SVNSYNC sync $SLAVE_URL