Re: encountered a sed POSIX gotcha (GNU vs. BSD)

2018-05-06 07:31:30 -0500, Tim Chase [email protected] [sed-users]:
> Encountered this one yesterday and thought I'd bring it to light.
> 
> On a GNU system this works:
> 
>  sed -n '/pattern/{s/foo/bar/p}'
> 
> but fails on BSD sed (tested on OpenBSD & FreeBSD)
> 
>  sed: 1: "/pattern/{s/foo/bar/p}": bad flag in substitute command: '}'
> 
> This is actually POSIX which requires a newline or semicolon before a
> close-brace:
> 
> "The <right-brace> shall be preceded by a <newline> or <semicolon>
> (before any optional <blank> characters preceding the <right-brace>)."
> 
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03
> 
> Thus on a BSD system it needs to have the extra semi-colon:
> 
>  sed -n '/pattern/{s/foo/bar/p;}'
> 
> Unfortunately, I suspect that a lot of scripts out there depend on
> the GNU-specific relaxed (but POSIX-violating) rules, so enforcing
> the POSIX will likely break them.  However, I also know that a number
> of GNU tools respect a POSIXLY_CORRECT environment variable which
> might be useful in this situation.
[...]

GNU sed is not violating POSIX here, a script that relies on the
specific behaviour of GNU sed (of treating it as if there was
a ; before the }) or for that matters on the BSD behaviour (of
failing with an error, though it would be very unusual for a
script to rely on a utility returning with an error) would be
the ones being non-conformant.

POSIX doesn't specify the behaviour for {s/x/y/} so either
failing with an error, or the GNU behaviour (or any other
behaviour) are valid behaviours. The GNU behaviour is a more
useful one. The only problem with it is that it doesn't help you
realise that your script is not portable.

But it's not GNU sed's job to do that.

What you're looking for is a linting tool that can spot non
portable constructs in your code. You could make a "sedlint"
fork of GNU sed that does that, like posh forked pdksh to make a
shell that helps identify non-POSIX shell constructs, or one
could write a shellcheck-like static code analysis tool to
identify non-standard/portable code for sed.

POSIXLY_CORRECT is to have tools align with POSIX when they
don't by default.

For instance, sed 's/[\t]//' is required to remove every
instance of backslash and t characters per POSIX, which GNU sed
doesn't do by default (and in that would not be compliant). GNU
sed only does that under POSIXLY_CORRECT (otherwise it removes
TAB characters instead).

Here, a more sensible thing to do would be to ask FreeBSD folks
to have their sed align with GNU sed, as it's a useful feature.

And then, if enough implementations align with GNU sed, ask
POSIX to make that into law.

Here however, there's a problem in that for instance:

{w file}

is currently meant to write to a file called "file}" (and that's
also what GNU sed does), and changing that could break backward
compatibility, so it's possible that POSIX would not bother
specifying a "}" that appears even without a preceding
";"/newline. They did however relax the requirement that "b foo}"
should branch to a "foo}" label (which GNU sed didn't even under
POSIXLY_CORRECT), so you never know.

-- 
Stephane