representation of tricky filenames in index

[*many* apologies for any duplication - I'm sure I've sent this twice,
already, but it has never appeared in the list archive or had any replies,
even though a different message I sent *later* has appeared in the archive.
trying again...]


Hi,

The wiki for Application_API/Operations has this to say about
index entries:

  (One filename per line similar to the output of tar -t)
  each name must start with a '/' and must be relative to the device.
  A directory must also terminate with a '/'.
  the simplest entry is just one '/'.

I remembered that I have a directory from years ago that I put
some very tricky file names in (to test how well a different
program handled them), so I looked in a recent Amanda index file
to see how they were recorded there.  I see them as:

/home/test/Carriage\rReturn

/home/test/Hello\r

/home/test/Line\nFeed

/home/test/The O'Leary & Company "cow" (<moo>)

/home/test/escape\\me\a!

From these examples, I can see that the rules are like C string
escapes (for \\ \r \n \a etc., anyway) but with no quote marks
around the entire name, and no special treatment *of* quote marks
or spaces. (That's as produced by amgtar anyway.)

Those seem to be different quoting rules than what
Amanda::Util::quote_string implements.  Interestingly, when
I use amrecover to browse the contents, it displays them with
what looks like an extra layer of Amanda::Util::quote_string
wrapped on top:

2016-04-05-23-30-03 "escape\\\\me\\a!"
2016-04-05-23-30-03 "The O'Leary & Company \"cow\" (<moo>)"
2016-04-05-23-30-03 "Line\\nFeed"
2016-04-05-23-30-03 "Hello\\r"
2016-04-05-23-30-03 "Carriage\\rReturn"

I can successfully 'add' the cow one by typing it exactly as shown,
but the only way I found to 'add' the other four in amrecover
was by typing wildcards where the funny stuff is.

amrecover does, in fact, then recover all five files successfully
under their original names. Again, at least all of this is true
for amgtar, which was used for this DLE; I have not compared other
programs/applications.

Anyway, the moral of the story seems to be that the brief note
"similar to the output of tar -t" on the wiki page is really
quite important, because the "output of tar -t" (for GNU tar)
has its own quoting rules, and without them my examples would
have failed. (At least the file with the line feed in the name
would certainly cause trouble, because without the quoting it
would appear as two names in the index.) Also, not all tar
programs use these rules. I tried the plain Solaris tar and
it does no quoting, the file name with the line feed comes out
on two lines.

So it seems the index file format would benefit from being more
precisely specified ... I can think of two options though.

1. Perhaps the cleanest, most precise option: document that the
   index file format *uses exactly* the quoting rules of GNU tar
   for its default --quoting-style=escape, documented here:

   https://www.gnu.org/software/tar/manual/html_chapter/tar_6.html#SEC113

   Then it would be quite useful to have utility functions
   in perl to apply and unapply those exact rules, so it would be
   easy to write new applications with consistent behavior.
   (But note there is one error even in that GNU document: it
   says the rules include a \ before a space, which we see tar
   is not producing ... even in the example on the page. I've
   just reported that on bug-tar...)

2. But maybe the situation already is not that good, and
   existing programs/applications that do not use GNU tar are
   already writing index lines according to different rules?

   In that case perhaps a more lax rule is needed, one that says
   every program/application may have its own rules for index
   lines, safe as long as the same program/application will be
   used for restoring the backup as for creating it. There only
   needs to be one hard rule, which is that any contained newline
   has to be quoted somehow, because otherwise the one-filename-per-
   line property breaks, which I'm sure amindexd needs.

To me, (1) seems more ideal ... make the specification explicit.
Maybe a compromise would be to say there might be *existing*
programs/applications using different rules, as long as they are
getting newlines right, but any *new* ones should use the same
GNU tar --quoting-style=escape rules.

But I don't know what the veteran Amanda devs would prefer.
What do you think?

Regards,
-Chap