Re: Dar architecture questions

On 10/16/2012 02:42 PM, Denis Corbin wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 16/10/2012 17:07, Kevin Wormington wrote:
>>> Le 16/10/2012 00:30, Kevin Wormington a =E9crit :
>>>> I strace'd the small backup (14k files) both with and without
>>>> --disable-special-alloc and it appears the difference is in the huge
>>>> number of rt_sigprocmask calls made by special-alloc.
>>>
>>> interesting... I have tests currently running on my side...
>
> The so far ran tests also reported huge number of rt_sigprocmask calls
> but they did not have a big impact (see below). I do not know yet
> whether this is caused by the --empty argument or by the 64 bits flavor
> of dar used here. I will do additional tests on Saturday. The used
> system is a Debian stable one with a slightly tuned kernel (kernel with
> less modules and many unused features disabled, but I do not see how
> this may have any concern here).
>
My systems are stock ubuntu server 10.04 LTS and debian 6 all on 64bit =

versions of the OS.  My configure arguments are the same as yours.  I =

did a "make install" between rounds of testing vs running it from the =

source directory.  This is probably a dumb question -- but I don't know =

your dev environment -- are you doing something to cause the rebuilt =

library in the source directory to be used vs the one installed in the =

system ld path (assuming there is one)?


[snip ]

>> I have also included a patch below that prints some status info before
>> the memory frees/on-fly-catalog since these can take several minutes to
>> complete after the archive stats print.
>>
>> diff -c dar.cpp.orig dar.cpp
>> *** dar.cpp.orig	2012-10-16 09:51:37.157093322 -0500
>> --- dar.cpp	2012-10-16 09:55:24.467092154 -0500
>> ***************
>> *** 270,275 ****
>> --- 270,277 ----
>>
>>     		    // making some room in memory
>>
>> + 		if(!param.quiet)
>> + 		    dialog.warning(gettext("Making room in memory..."));
>
> Here I would display this message only if -v option has been set
> (param.info_details is true)

I had thought of that as well...however in a use case like mine =

(ultimately control of dar by another script/program) I don't want to =

output the display of 1.7 million filenames.  I also didn't think =

another command line switch would be justified just for this case.

>
>>     		if(arch !=3D NULL)
>>     		{
>>     		    delete arch;
>> ***************
>> *** 295,301 ****
>>     			throw SRC_BUG;
>>     		    if(param.op =3D=3D create)
>>     		    {
>> ! 			if(param.info_details)
>>     			    dialog.warning(gettext("Now performing on-fly isolation..."));
>>     			if(cur =3D=3D NULL)
>>     			    throw SRC_BUG;
>> --- 297,303 ----
>>     			throw SRC_BUG;
>>     		    if(param.op =3D=3D create)
>>     		    {
>> ! 		        if(!param.quiet)
>>     			    dialog.warning(gettext("Now performing on-fly isolation..."));
>
> I completely agree with this change, I take it as is.
>
>>     			if(cur =3D=3D NULL)
>>     			    throw SRC_BUG;
>> ***************
>> *** 609,614 ****
>> --- 611,618 ----
>>     	}
>>     	catch(...)
>>     	{
>> + 	    if(!param.quiet)
>> + 	        dialog.warning(gettext("Final memory cleanup..."));
>
> I would not report here anything, because this statement is reached is
> only reached in case of major problem. I would not add the risk to hide
> a problem by another one.
>
oops...I obviously didn't catch() that!  I agree no reason to report it =

there especially if it might mask another problem.

>>     	    if(arch !=3D NULL)
>>     	    {
>>     		delete arch;
>> ***************
>> *** 626,631 ****
>> --- 630,637 ----
>>     	    }
>>     	    throw;
>>     	}
>> + 	if(!param.quiet)
>> + 	    dialog.warning(gettext("Final memory cleanup..."));
>
> Here I agree but only when param.info_details is true
>
Again, my only thought was not wanting to get filenames listed just to =

have status update.

>>     	if(arch !=3D NULL)
>>     	{
>>     	    delete arch;
>>
>
> Your patch (adapted) is now in GIT on the branch_2.4.x :)

Thanks.
>
>>
>>> However, I must document the result of your tests (and mine) about this
>>> subject. This might be of interest for the ones that are not concerned
>>> by memory limitation and want to speed up their backup.
>>>
>>>    Perhaps someone else on the list that
>>>> makes large backups either in size or number of files could confirm if
>>>> they see the same behavior.
>>>>
>>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQIVAwUBUH24uwgxsL0D2LGCAQLCHxAAtOubNT2HF9L47bdQrZtCZ4BxOoM7VVx1
> denZGca1ZokPDxJGeEweVdAOXumsGVoVRnnERPuBTa+Fd9jezISe13yc14cJ3iKk
> jbgiii6mXDKi4AU2vh8QnY1VRUIIzkZZr7gDQ6Stcnim3nb061+yx2o/sEMj07L7
> uwybe8klnnrA/hKZkXfl1oZqCNpdk4Trp7BogWxtwZhjPoGWnVQe0joNkrn7gdYU
> bPH5rjdl60lwDNtjthUgtVHgFGH7zCgwMVXwIyk/U7xa1nnTlZW3bcjnBzEUQbzn
> eRlMTg2sQHzpkW9cQM+EkgpNNjwsXot2Nx0E0ov2Ee2N24WGGFo9mI1GSj/4OMZs
> hvj4giiakZt3t0Fn+Ij8MbFYE0hXq8CBvcTu9wh4cpIqapiIg0m9b+3xgi38ZZp5
> 8zFTUgB+r/d+VzucfXjOkZDx00Hw7s9aoDp8dsRVZ0YwS5Hm3632jwJO3L/F8l0O
> C9IJ3Ut7nnTtBg3e8UtWI7D6Y1EB2IfAzwYbZzDctFVmWhdIzyo8n26CHf0pMFDj
> PdDriuprlT400t4pgYsodUu6KYwIFYNGmFOAyJJsL0X2O6/NJRpJWgmsd7TZDwQB
> fGKqUrPtyCefbZWmbE+TaOh/tb3N4lZrULpnaUSlp9r8z6EabM0BTivGFjmjShlj
> je2+2GzoHc0=3D
> =3DdMdR
> -----END PGP SIGNATURE-----
>
> -------------------------------------------------------------------------=
-----
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Dar-discussions mailing list
> Dar-discussions-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/dar-discussions
>

---------------------------------------------------------------------------=
---
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev