Re: Fwd: Hdup feature suggestion

On Tue, 2004-10-26 at 20:08 +0200, Miek Gieben wrote:
> couldn't resist, so I started on making some sort of directory crawl
> tool, which is going to be folded in hdup. 

Of course I can't resist throwing in my two cents.

> And maybe released as a
> seperate tool - dunno...

Have you checked for the existence of such libraries already?  It seems
like a problem that must have been solved before.

> 1.
> The include/exclude stuff is going to be regexp based. Not shell
> globbing.  Should these patterns only apply to files, or also to
> directories?

It might be clearest to have separate include and exclude lists for
files and directories.

> 1.5
> So 'exclude = .*\.html' will exclude all *.html files, should it
> also skip dirs with the same name?

There's no particular reason to think that a .html directory should be
treated the same way as a .html file.

> 3.
> include/exclude clashes. One could conceive conflicting configuration
> options. What to do? Make an explicit include weigh heavier?

This is where you need to put a great deal of forethought -- these
schemes can quickly get very confusing, especially if your patterns are
allowed to cover context as well as the current filename (for example, I
might want to exclude all ".netscape/cache" directories, but not other
directories named "cache"; or I might want to exclude all "src"
directories under "/opt"...).

A couple of existing examples (besides tar) are apache configuration
(the part where you specify access rules to directories and get to
explicitly state whether you want "allow,deny" order or "deny,allow"
order of interpretation) and rsync
(http://samba.anu.edu.au/ftp/rsync/rsync.html -- search for 'EXCLUDE
PATTERNS').

I'm not saying you want to follow either of these examples -- the rsync
one in particular seems to be one of the primary sources of user
frustration and queries to the mailing list.  But it may give you some
ideas (and if you like it, the code is open source...)


> Globbing is ancient and not consistent. Regexps are the way to go
> and more powerfull - but maybe also slower. I opt for regexps.

Regexps are also not that consistent, since there are multiple
'flavors'. But the power is useful, and the speed should not be an issue
(insignificant next to the cost of actually reading and archiving the
files, except perhaps in the case where you are only backing up a small
percentage of the files on your disk).


> PS the .nobackup files should be named .hdup_nobackup (and empty??)

Perhaps the name should be configurable; some people may also want to
turn this feature off (and avoid the overhead of looking for the file in
every directory).

Here's an idea: rather than cancelling directories entirely,
the .nobackup file could be like .cvsignore, and list the names of files
and subdirectories within the current directory that should not be
backed up.  This would allow easy local control over large files that
should not be backed up as well as directories.  Putting "." in the file
could cancel backing up the directory.  Or you could really go wild and
allow the file to contain exlcude= and include= directives that would
apply only to that directory and its descendants.

Bng
-- 
Boris Goldowsky <boris-FrUbXkNCsVf2fBVCVOL8/[email protected]>