Re: Dar inconsistency on differential backup based on catalogue file

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

T o n g a wrote:
> Hi Denis, 

Hi Tong,

> 
> I decided to still follow up here because it is more a bug report now.

Well, this is not a bug mailing-list, you could have followed to
dar-support or open a bug in the bug tracker for dar at sourceforge.

Anyway, I have open a bug report at
https://sourceforge.net/tracker/?func=detail&aid=3064839&group_id=65612&atid=511612
I ask you to follow up there. But also read below.

> 
> To recap all related info first:
> 
>  $ dar -l /path/to/da-name_2009-12-11-cat | head -15
>   [ . . . ]
>  [InRef]       [-----]   -rwxrwxr-x   tong       tong    8510    Fri Jan  4 13:47:05 2008        inet/web/larve.2006.add-to-social-bookmarking-services.htm
>  [InRef]       [-----]   -rwxrwxr-x   tong       tong    12385   Fri Jan  4 00:00:00 2008        inet/web/sb.zip
>  [InRef]       [-----]   -rwxrwxr-x   tong       tong    199     Fri Jan  4 13:47:17 2008        inet/web/11getlog.lst
>   [ . . . ]
> 
>  $ ls -l inet/web/larve.*.htm inet/web/sb.zip inet/web/11getlog.lst
>  -rwxrwxr-x 1 tong tong   199 2008-01-04 09:47 inet/web/11getlog.lst
>  -rwxrwxr-x 1 tong tong  8510 2008-01-04 09:47 inet/web/larve.2006.add-to-social-bookmarking-services.htm
>  -rwxrwxr-x 1 tong tong 12385 2008-01-03 20:00 inet/web/sb.zip
> 
> I.e., all files are time-shifted by exactly 4 hours. Here's what happened:
> 
>   $ dar -e -v -R /path/to/d_arch -c /path/to/da-name_2010-09-05-diff -A /path/to/da-name_2009-12-11-cat -s 10M
>   Extracting contents of the archive...
>   Adding file to archive: /path/to/pwd/db
>   Adding file to archive: /path/to/pwd/db/dict
>   Adding file to archive: /path/to/pwd/dev
>   Adding file to archive: /path/to/pwd/fun
>   Adding file to archive: /path/to/pwd/fun/games
>   Adding file to archive: /path/to/pwd/inet
>   Adding file to archive: /path/to/pwd/inet/lan
>   Adding file to archive: /path/to/pwd/inet/web
>   Adding file to archive: /path/to/pwd/inet/web/larve.2006.add-to-social-bookmarking-services.htm
>   Adding file to archive: /path/to/pwd/inet/web/sb.zip
>   Adding file to archive: /path/to/pwd/inet/web/11getlog.lst
>   Adding file to archive: /path/to/pwd/inet/agent
>   Adding file to archive: /path/to/pwd/inet/email
>   [ . . . ]
> 
> I.e., all files are put again into the differential backup.
> 
>   $ dar -e -v -R /path/to/d_arch -c /path/to/da-name_2010-09-05-diff -A /path/to/da-name_2009-12-11-cat -s 10M -H5
>   Extracting contents of the archive...
>   [ . . . ]
>   Adding file to archive: /path/to/pwd/inet/web
>   Adding file to archive: /path/to/pwd/inet/web/larve.2006.add-to-social-bookmarking-services.htm
>   Adding file to archive: /path/to/pwd/inet/web/11getlog.lst
>   Adding file to archive: /path/to/pwd/inet/agent
>   Adding file to archive: /path/to/pwd/inet/email
>   [ . . . ]
> 
> I.e., having add '-H5'/'-H6' (ideally -H4 should be enough) to the dar
> command line, dar only considers the 'sb.zip' file to be backed
> up before, and still add the other 2 files.
> 
>> What's happening? How can I do further investigation?
> 
> ,-----
> | there is an easy way to know why dar saved a file for differential
> | backup, it is to use the archive of reference and make a diff (dar -d)
> | against the fileystem you want to backup. For each file that differs
> | between archive and fileystem, dar will output what causes the difference.
> `-----
> 
> OK, here it goes:
> 
>   $ /usr/bin/dar -v -R /path/to/d_arch -d /path/to/da-name_2009-12-11-cat
>   [ . . . ]
>   OK   /path/to/d_arch/inet/lan
>   OK   /path/to/d_arch/inet/web
>   OK   /path/to/d_arch/inet/web/larve.2006.add-to-social-bookmarking-services.htm
>   OK   /path/to/d_arch/inet/web/sb.zip
>   OK   /path/to/d_arch/inet/web/11getlog.lst
>   OK   /path/to/d_arch/inet/agent
>   OK   /path/to/d_arch/inet/email
>   [ . . . ]
> 
> I.e., 'dar -d' thinks and reports that all 3 files are OK, and
> shouldn't be put into the differential backup, even the -Hx
> parameter is NOT given. There is sure some inconsistency here.

Well, I was a bit wrong, the archive difference does not show all the
differences that dar can see at differential backup time. After having
review the code, -d option, do compare file type, user and group
ownership, permission, file size when applicable and date of EA if
present. But it does not compare last access time nor last modification
time. This is also for that reason that -H option is useless with -d
option as you might have noticed the following warning:

	-H is only useful with -c or -x

where from the bug I have openned. It should let us better see what is
the reason that lead dar to save the files during the differential backup.

Regards,
Denis.


> 
> FYI, I've also executed the actual backup command without the '-e'
> parameter, and the result is same. I.e., the behaviour is
> consistent w/ or w/o '-e'.
> 
> Please investigate. 
> 
> PS. I don't know how easily for you to duplicate a time-zone
> shifting scenario (ie, whether changing the time-zone will have immediate
> affect without rebooting), but this is how it happened to me.
> http://thread.gmane.org/gmane.linux.debian.user/386125/focus=386199
> Long story short, the files are on vfat partitions, you get
> different file time when flipping BIOS CMOS clock between UTC and
> localtime. Again, hope there is an easier way for you to
> duplicate a time-zone shifting scenario. 
> 
> Thanks
> 
> On Mon, 06 Sep 2010 03:21:34 +0000, T o n g wrote:
> 
>> Ok, the time-zone shifting might be the cause:
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFMjPuSpC5CI8gYGlIRAifpAJ0dZSaISTcNCmXBPdKqgPYpMcBzzwCfb4fz
TozwfLogFFOOJ9UrOAw9q8o=
=hdsJ
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing
http://p.sf.net/sfu/novell-sfdev2dev