Re: [Sbcl-commits] master: pathnames: fix dot escaping with multiple preceding escapes

Hi Stas,

This one and its parent fix a couple bugs (namestring bracket notation
didn't allow a character set to contain a right bracket or a dot), but
IMO these aren't the best ways to do it.

1. The parent is a minor incompatible change: (parse-namestring "[a\\]")
   used parse to #<SB-IMPL::PATTERN (:CHARACTER-SET . "a\\")>. Now it's
   a parsing error.

2. It appears that the new parse of "[a\\]]" is
   #<SB-IMPL::PATTERN (:CHARACTER-SET . "a\\]")>, and of "[a\\.b]"
   #<SB-IMPL::PATTERN (:CHARACTER-SET . "a\\.b")>. That is, the escape
   character is included in the character set. I think this would be a
   bug, right?

3. "[a\\]" used to match the same filenames in Lisp as it would at a
   Unix shell. Although, SBCL's wildcards don't do everything POSIX
   fnmatch() or glob() can, I've found it useful that SBCL's wildcard
   syntax has been somewhat compatible with Unix shell's, since that
   allows copy-n-pasting sometimes. Using the Lisp namestring escape
   character between right brackets increases incompatibility a bit.

So I'd propose:

- make the namestring notation for a character set containing right
  bracket and "abc" be "[]abc]" (i.e., put the right bracket first).

- treat both a dot and the namestring escape character as literals
  between brackets.

The differences between the notations would be

  your notation              my proposed notation (same on Unix/Windows)
  "[\\]abc]" or "[^]abc]"    "[]abc]"
  "[\\.abc]" or "[^.abc]"    "[.abc]" ; but see the Note below
  "[\\\\abc]" (Unix)         "[\\abc]"
  "[^^abc]" (Windows)        "[^abc]"

The easiest way to do the second thing is to have physical namestring
parsing go strictly left-to-right, rather than deciding where the type
separator is using a right-to-left pre-scan.

Here's a commit that does these things; it passes all GH CI runs:

https://github.com/sbcl/sbcl/commit/3455ecbc8b699d161e0dd88a3cdaba7d5afc2940

(That commit also normalizes the stuff between the brackets, e.g., so
that e.g., "[aab]" and "[ba]" both parse to #P"[ab]", which also makes
(pathname-match-p "[ab]" "[ba]") return true. I think that's a small
improvement, but I guess it's orthogonal to the issue at hand.)

What do you think?

Thanks,
Richard

Note: Your notation and mine both allow a dot to appear inside a
character set. But so long as PATHNAME-MATCH-P operates component-wise
on physical pathnames either of

  (pathname-match-p "x.y" "x[\\.]y") ; your notation
  (pathname-match-p "x.y" "x[.]y")   ; my proposed notation

will return nil, just as (pathname-match-p "x.y.z" "x.y\\.z") does. I
consider this orthogonal to choice of bracket notation.