Re: Dar architecture questions

On 10/17/2012 01:42 AM, Denis Corbin wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> [...]
>>>>>
>>>>>> My systems are stock ubuntu server 10.04 LTS and debian 6 all on 64bit
>>>>>> versions of the OS.
>>
>> Are they multi-core or multiprocessors host? Mine in an not very recent
>> 64bits mono processor host. Linux kernel used had SMP disabled at
>> compilation time. This might have an impact.
>>
>>
>>> They are a mix of dual processor single-core (hyperthreading enabled) 
>>> older Intel Xeons and newer AMD Opteron quad-core dual processor.  So if 
>>> you have SMP disabled in the kernel that could definitely be a 
>>> difference.  Perhaps it's some sort of race condition with the 
>>> rt_sigprocmask or the mutex lock/unlock that surrounds it.
> 
> Reviewing library and system calls documentation, I've noted an error in
> the way libdar handles signals. By default, libdar is thread safe and
> should not call sigprocmask but pthread_sigmask instead, well dar itself
> is single-threaded so this is not a problem here (it is for kdar
> graphical multi-threaded application that relies on libdar). On my
> system, I see no significant difference in execution time nor in system
> call used (both ways still result in rt_sigprocmask as show by strace).
> 
> Could you apply the following patch and see if you get any difference on
> one of your multi-core host?

Performance is the same with or without the patch.  Just for kicks I defined
CRITICAL_START and CRITICAL_END in special_alloc.cpp to be just '//' and the performance is only slightly better than using just special-alloc.  This pretty much removes the sigprocmask and mutex locks from the picture.  

It is looking like the system libraries are going to be the fastest at least on my systems.
> 
> diff --git a/src/libdar/tools.cpp b/src/libdar/tools.cpp
> index 2ce62a3..e54e4e0 100644
> - --- a/src/libdar/tools.cpp
> +++ b/src/libdar/tools.cpp
> @@ -1635,13 +1635,21 @@ namespace libdar
>         sigset_t all;
> 
>         sigfillset(&all);
> +#if HAVE_LIBPTHREAD
> +       if(pthread_sigmask(SIG_BLOCK, &all, &old_mask) != 0)
> +#else
>         if(sigprocmask(SIG_BLOCK, &all, &old_mask) != 0)
> +#endif
>             throw Erange("tools_block_all_signals",
> string(dar_gettext("Cannot block signals: "))+strerror(errno));
>      }
> 
>      void tools_set_back_blocked_signals(sigset_t old_mask)
>      {
> +#if HAVE_LIBPTHREAD
> +       if(pthread_sigmask(SIG_SETMASK, &old_mask, NULL))
> +#else
>         if(sigprocmask(SIG_SETMASK, &old_mask, NULL))
> +#endif
>             throw Erange("tools_set_back_block_all_signals",
> string(dar_gettext("Cannot unblock signals: "))+strerror(errno));
>      }
> 
> 
> 
>>
>>> Did you see my comment in the previous post about the patch?  I was 
>>> thinking that with -v it would display all 1.7 million files instead of 
>>> just the status messages which is why I had checked the quiet flag 
>>> instead or adding a new command line flag to just get verbose program 
>>> status and not file listing.
> 
> Yes, I've seen it. By default, dar only report errors met and a summary
> at the end of execution. --quiet option suppresses the summary and dar
> then report nothing excepted the let errors. --verbose option adds step
> by step information about what's going on, including which file is being
> processed (not the one skipped, which is activated by -vs). So by
> coherence with that, I cannot add the report that are useful for you in
> that situation.
> 
> I will add a switch that let dar show the different steps it is inside
> of, without showing processed files. However this would not take place
> in 2.4.x but in 2.5.x. Sorry for that.
> 
> [...]
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iQIVAwUBUH5TQggxsL0D2LGCAQJNchAAqaYPz59wZIEM+kiavcH8S5aOWyQsl9rJ
> ZyzuCwreuXKsn30RGRO5RalCXE3Ni4sPePifTwGxkmwqMII60lV5OxyDBdBXF94Q
> HyHrdlGF9gmMi3xDISnkkF2gKFNtKiER0XawVLlyF/sb0E9cnUSZszl+TFAo2mVZ
> AOxHhq+E+ynDemADatXQX8syd7M7O34Bn0zm5LoSvZHLKkAubE9O87wsc8brc5My
> WOdYLD+myw9/4JXm5acyPHZmvGeBLxKXznkARntLOQ7d4dQZKzDt9pFQxAj1cffz
> YAO/bVRJO66mgWFPjaM1X8mEqMg7cQtFpV5H6oZ4NywZMDqSEy8wn0OVwTH8GIOM
> ni6LNsgHxakz0NYfX6Qj8WEF2xyaLnyAsu9F+HTBoKc3kezK1vEdLisUAT8vxvSo
> OsezEbLe3qOH96D9ToefdU2p4Z7E9XzeyzQ0s10Nz39JPsU0I4vYuAr1f9VYf6cL
> OWIIicjm7QRfL7YQGpl+3fZR0YZ2z4aEGnuxVSbOP9WVW4gugGB/NhCKaR2MYgfa
> jQKI4dpr77CBHztLexaCtoeraNRPzQY+3066gdrXiEfR1Zrxw9thBVelv/UkAXmp
> wjLM9DJoJ/z1yWg4/+RSpaoidDg06vfbkNpZV3+HOlWilI6FQDFnZ3epAyo1gKxP
> /N67nVrkBbo=
> =QsD+
> -----END PGP SIGNATURE-----
> 
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_sfd2d_oct
> _______________________________________________
> Dar-discussions mailing list
> Dar-discussions-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/dar-discussions
> 

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct