Re: Dar architecture questions
Denis Corbin <[email protected]> Tue, 16 Oct 2012 09:22:12 +0200
| Newsgroups | gmane.comp.sysutils.backup.dar.general |
|---|---|
| Message-ID | <[email protected]> |
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 16/10/2012 00:30, Kevin Wormington a =E9crit : > I strace'd the small backup (14k files) both with and without = > --disable-special-alloc and it appears the difference is in the huge = > number of rt_sigprocmask calls made by special-alloc. interesting... I have tests currently running on my side... > Am I correct that = > the amount of memory used by dar is mostly related to the number of = > files backed up and not the sizes of the files backed up? = yes this is correct. > Ie, I wonder = > if a different workload of small number of large files would be effected = > by the use of special-malloc. If it could be confirmed that it's a win = > to disable special-malloc I would suggest that you consider disabling it = > by default. I only have recent ubuntu and debian servers to test with = > so I'm not sure what the result would be on the other flavors of linux = > and/or different os platforms. = Yes, that's a point to take into account. And you also reported that with special-alloc uses dar uses less virtual memory, right? Sometimes I get feedback about lack of memory (huge number of files under backup). So I guess its better to have by default a slower binary less fond of memory than a faster one using more memory. However, I must document the result of your tests (and mine) about this subject. This might be of interest for the ones that are not concerned by memory limitation and want to speed up their backup. Perhaps someone else on the list that > makes large backups either in size or number of files could confirm if = > they see the same behavior. > = > run with --disable-special-alloc: > = > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 71.72 0.000208 0 14067 lstat > 19.31 0.000056 0 2679 read > 8.97 0.000026 0 1053 lseek > 0.00 0.000000 0 216 write > 0.00 0.000000 0 440 28 open > 0.00 0.000000 0 412 close > 0.00 0.000000 0 409 fstat > 0.00 0.000000 0 48 mmap > 0.00 0.000000 0 26 mprotect > 0.00 0.000000 0 9 munmap > 0.00 0.000000 0 208 brk > 0.00 0.000000 0 9 rt_sigaction > 0.00 0.000000 0 47 rt_sigprocmask > 0.00 0.000000 0 10 ioctl > 0.00 0.000000 0 15 15 access > 0.00 0.000000 0 1 nanosleep > 0.00 0.000000 0 1 execve > 0.00 0.000000 0 775 fcntl > 0.00 0.000000 0 800 getdents > 0.00 0.000000 0 8 getcwd > 0.00 0.000000 0 1 getrlimit > 0.00 0.000000 0 5 getuid > 0.00 0.000000 0 1 getppid > 0.00 0.000000 0 1 mlock > 0.00 0.000000 0 1 arch_prctl > 0.00 0.000000 0 5 1 futex > 0.00 0.000000 0 1 set_tid_address > 0.00 0.000000 0 1 set_robust_list > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.000290 21249 44 total > = > Here is the same set using special-alloc: > = > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 96.79 0.032938 0 1622329 rt_sigprocmask > 2.80 0.000953 0 14066 lstat > 0.20 0.000067 0 2679 read > 0.15 0.000051 0 800 getdents > 0.06 0.000021 0 1053 lseek > 0.00 0.000000 0 216 write > 0.00 0.000000 0 440 28 open > 0.00 0.000000 0 412 close > 0.00 0.000000 0 409 fstat > 0.00 0.000000 0 48 mmap > 0.00 0.000000 0 26 mprotect > 0.00 0.000000 0 9 munmap > 0.00 0.000000 0 140 brk > 0.00 0.000000 0 9 rt_sigaction > 0.00 0.000000 0 10 ioctl > 0.00 0.000000 0 15 15 access > 0.00 0.000000 0 1 nanosleep > 0.00 0.000000 0 1 execve > 0.00 0.000000 0 775 fcntl > 0.00 0.000000 0 8 getcwd > 0.00 0.000000 0 1 getrlimit > 0.00 0.000000 0 5 getuid > 0.00 0.000000 0 1 getppid > 0.00 0.000000 0 1 mlock > 0.00 0.000000 0 1 arch_prctl > 0.00 0.000000 0 5 1 futex > 0.00 0.000000 0 1 set_tid_address > 0.00 0.000000 0 1 set_robust_list > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.034030 1643462 44 total > = -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIVAwUBUH0LGwgxsL0D2LGCAQLooRAAwrZCMTTgdUJema7Gg/eVqtMHgEbx/Zfv F3mWcwibE5b/zLnDUcTO9oJb4r3/fNaTEz+flfO944RRmWjcRArUiCAZxL4Rfmpr EQiCvkCFkyQr98X6A1W7+pZTTBCp70zqB1VSnmMbvIBKwPcX26OggW1Uo2GNS6pn rAlUBcHW2sESH9/KlwX7UIeXcO53a/XHOlHIBQES7itd0UYjhOHV0a4r1eA9bq4E 5l9D6J9+7AbtPHMoi2Dv2WstMHxhiQ0/mbd4GXJAA12OvcV9OIY8fFURXfKkZYNk 1TIi3LcKVwglMCP0zLl3OGuY8Q65lT5jWG5srDgO1+EGoV9cI6TkRdsqsjrNqRNp 1vL+A5eJIjTDRQToUZrvTCNh45irNJJKRBFOB0m9bk+IugS90bTZRqY1NWow+Jy+ k1CdjBqrlgzlymNJwgQUi+lb1GYTDMyuT7AhFXt1a3vZ+BC6Iobw6yATYxIkInpv J/k3AUpaDF3SQ+wAztRRQvm278IH9594MtU1ZAueO5I3zQaQB/JSRvijBAGSqb7P p99w0QqEDxQvRVDSzSi2TkFGcxnz+blV6T1zfZuZDK+2bngf7vhyym4e0z3If0My rjU9ymlSUJAeMwS+FnpTZQ2MiSa09f1gs1Cufuv+lTyMRSOqh0XEkCE2jhw4OPWe JwztNy+xZU8=3D =3DntfL -----END PGP SIGNATURE----- ---------------------------------------------------------------------------= --- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev