Re: Why does createrepo -C use ctime, not mtime?

Replying to myself, for reference if someone should be interested in
the full answer to the initial question.

On Wed, Apr 07, 2010 at 07:16:00PM +0200, Robert Vogelgesang wrote:
> On Wed, Apr 07, 2010 at 12:42:28PM -0400, Seth Vidal wrote:
> > 
> > 
> > On Wed, 7 Apr 2010, Seth Vidal wrote:
> > 
> > >
> > >
> > >On Wed, 7 Apr 2010, Robert Vogelgesang wrote:
> > >
> > >>Hello developers,
> > >>
> > >>currently, I'm investigating an issue with "createrepo --update -C"
> > >>when run by "cobbler reposync", and found in createrepo's source
> > >>that the -C option is based on Python's os.path.getctime(), i. e. the
> > >>time of the last status change on *nix hosts (ctime).
> > >>
> > >>As of version 2.0.3.1, "cobbler reposync" chown's and chmod's all
> > >>files in a repo on each run, changing the ctime of all files in the
> > >>repo; I think this is a bug in itself, but nonetheless I'd expect
> > >>that mtime is used by createrepo when checking for changed files.
> > >>
> > >>createrepo's variable which holds the reference timestamp,
> > >>self.mdtimestamp in class MetaDataConfig, seems to indicate that mtime
> > >>should be, or even was historically used.  So, is there any special
> > >>reason why this is based on ctime and not mtime?  I've searched
> > >>createrepo's source, and http://createrepo.baseurl.org/, but found
> > >>no answer.
> > >>
> > >>Can you shed some light on this, please?
> > >
> > >You have to search pretty far back.
> > >
> > >here is where the original -C behavior comes from:
> > >
> > >http://lists.baseurl.org/pipermail/rpm-metadata/2006-July/000663.html
> > >
> 
> ahh, thank you.
> 
> > 
> > And looking back further - the original behavior was getmtime() but Pete 
> > found some problems with the reliability of mtime() for the check.
> > 
> > I don't remember the _why_ there, though.
> 
> As far as I can see in the mailing list archive, the timestamp check
> initially used the timestamp of the repo directory instead of the
> repomd.xml file, as now in the current version.  Could that be part
> of the reason?  I. e., maybe the mtime issues are fixed by the
> change to check repomd.xml instead of the repo directory?

After adding some debugging code to createrepo, I've learned
the following:

The external tools used by cobbler for mirroring repositories,
i. e. reposync, rsync, and wget, update the mtime of the mirrored
files to match the mtime of their "original" counterparts.

But the ctime timestamps of the files in the local repositories
managed by cobbler are not altered, they represent the time of the
download (with a small offset that is caused by the timespan these
tools need to update permissions and the mtime, plus a bigger offset
caused by cobbler running "chown -R" and "chmod -R").

Now, if createrepo wants to check if some files were added to the
local repo after the current repomd.xml was created, it _has_ to compare
the ctime timestamps, which give the time of the update of the local
repository, and not the mtime timestamps of the files, because the mtime
represents the last update at the "original" site.

	Robert

PS.: Feel free to add this (or something similar) to createrepo's
source or docs.

> 
> 	Robert
> 
> > 
> > -sv
> > 
> > _______________________________________________
> > Rpm-metadata mailing list
> > [email protected]
> > http://lists.baseurl.org/mailman/listinfo/rpm-metadata
> _______________________________________________
> Rpm-metadata mailing list
> [email protected]
> http://lists.baseurl.org/mailman/listinfo/rpm-metadata