Re: Is it fair to say, "sed supports # comments starting any column"?

Hi Stephane,

Thanks for your thoughtful reply.

- The POSIX spec, at the link I sent, does not say "# is a command". If 
anyone has a POSIX link with that wording, please post it. Instead, 
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html says, 
"The comment character is treated as a command, and it has the same 
properties in terms of being accepted with leading <blank> characters". 
The point is that # can have leading blanks.

The POSIX text is confusingly worded. What else is new? Even the GNU sed 
manual sloppily lumps in # with other "commands". But a comment is not a 
command. It's a comment. A command must be separated from other 
commands. Neither POSIX nor the GNU sed manual say anything about having 
to separate # from other commands on the same line. But I am open to 
other information anyone might provide.

- The POSIX spec, at the link I sent, does not say "it (#) must be 
separated from another command with ; or newline". If anyone has a 
reference that says what you quoted, please post it. I think the 
standard allows "s/x/y/ # comment" format.

- The POSIX spec, at the link I sent, does not mention ; syntax. If 
anyone has a POSIX link that mentions ; syntax, please post it. I'm not 
saying you are wrong. I would very much hope that POSIX would mention 
the very useful ; syntax! If it does not, could it please by added? I 
just don't see the mention, so the point about "s/x/y/; # comment" being 
required does not make sense.

For some of the examples you post, probably best for me not to respond 
unless someone can post the apparently different POSIX citation that 
backs up the wordings you quote. You obviously have expertise with POSIX 
specs. I hope you can provide a better link.

Could you clarify what you concluded about the solaris sed versions? I'm 
not sure what "command -p" does.

Thanks for posting to the POSIX discussion group. My understanding of 
your post and the replies is that solaris will at least be updated to 
allow blanks before the # character, and that the intention is that it 
be POSIX compliant, whatever that means. I think "s/x/y/ # comment" is 
POSIX compliant. Does anyone disagree, and if so, on what basis?

Is my understanding correct, based on "SVID which only allows a comment 
as the first line of the script", and what Rakesh said, that System V 
only allows the line 1 of the script to be a comment? If so, isn't that 
incredibly pitiful, and no basis for any kind of argument? It seems 
extremely primitive, which is saying a lot coming from a seder. :)

AFAIK, the examples with the w command have nothing to do with this 
comment issue, as exactly the same problem occurs with other sed syntax. 
In other words, the w command has to end a script line, regardless of 
what follows. Even ; after the w command will cause an error (assuming 
the file name does not contain ; character).

You are right that # is not an s flag. It never will be, that would 
break backward compatibility, not to speak of being an awful design 
choice. Of course, sed 's/x/y/#comment' is asking for trouble, a kind of 
"coding horror". But it is perfectly legal, and in accord with the POSIX 
standard, to my understanding. And from your answer, it sounds like 
there is no other standard that overrides POSIX.

The GNU sed online manual does not list "comments can start in any 
column" as an GNU extension. Apparently, the maintainers consider GNU 
sed # behavior to be POSIX compliant. To me, combined with the POSIX 
wording I have seen so far, that indicates the answer to my question, 
"is it fair to say sed supports # comments starting any column?" is 
"YES". Would anyone argue "NO", and on what basis?

To my knowledge, 99% to 100% of languages do not require a command 
separator before a comment. To me, the idea is ridiculous. The POSIX 
standard is confusingly worded. But I doubt 100% that the POSIX 
designers intended to require "s/x/y/; # comment". What would be the 
point? I've used sed for decades, and the idea of "s/x/y/; # comment" is 
laughable, even if POSIX mentioned ; syntax. As you point out, "fixing" 
a sed version to allow "s/x/y/ # comment" would not break existing 
scripts. In contrast, requiring "s/x/y/; # comment" syntax would break 
scripts. :( Thus, my answer to "must sed # comments be separated from 
commands on the same line by ; syntax?" is "NO". Would anyone argue 
"YES", and on what basis?

Thanks,
Daniel

On 8/11/2015 2:23 PM, Stephane Chazelas wrote:
> It says # is a command though and that it must be separated from
> another command with ; or newline (optionaly followed/preceded
> by blanks) like other commands, and that some commands can't be
> followed by other commands (other than on a separate line or
> with a new -e expression).
>
> s/x/y/ # comment
>
> is wrong. It's attempting to use the (unspecified) # flag to the
> s command.
>
> s/x/y/; # comment
>
> would be POSIX AFAICT.
>
> s/x/y/w file; # comment
>
> Writes the pattern space after successful substitution to the
> file called "file; # comment" even with GNU sed.
>
> /x/{ =; } # comment
>
> is not POSIX.
>
> b foo;#x
>
> jumps to the "foo;#x" label in a POSIX sed (will change in the
> next POSIX spec where the behaviour will be unspecified
> instead and GNU sed will no longer be non-compliant in that
> regard).
>
>>   http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html
>>
>>
>>   So here are my questions for the experts on the egroup:
>>
>>
>>   1) Which actively maintained sed versions require comments to start in column 1?
>
> /usr/xpg4/bin/sed on Solaris 10 and Solaris 11 at least do:
>
> s11:~$ echo x | sed ' #'
> x
> s11:~$ echo x | command -p sed ' #'
> sed:  # is an unrecognized command.
> s11:(2)~$ echo x | command -p sed '#'
> x
>
>
>>   2) I have nothing against these other versions. The more
>>   versions, the better. But if POSIX says comments can start in
>>   any column, why aren't these other versions "fixed up" to be
>>   compliant, or even to be adequate? It shouldn't be all that
>>   difficult. If they aren't "fixed up", are they really
>>   actively maintained?
>
> What's strange is that /usr/bin/sed on Solaris doesn't have the
> issue. While /usr/xpg4/bin/sed (aka command -p sed) is meant to
> be POSIX compliant.
>
> I've raised the question on the Austin Group mailing list:
> http://thread.gmane.org/gmane.comp.standards.posix.austin.general/11287
>
> and it appears that's how it was specified in SVID before the
> POSIX days.
>
> It looks like Solaris forgot to update that part and that the
> conformance tests do not have a test case for that.
>
> Now that it's been noted by some Solaris guys on that mailing
> list, it may be fixed in the future.
>
>>   3) Given POSIX, is it fair to say, "sed supports # comments starting any column"?
>
>
> Whether it can start on a given column depends on what is on the
> columns left to it on the same line/expression as said above.
>
>>   4) What other standard overrides POSIX, and justifies other seds not getting "fixed up"?
>
> None that I'm aware. I can imagine a standard leaving it
> unspecified if # is not in first position, but not one requiring
> sed to fail there. IOW,  I can't imagine a standard that would
> prevent a sed implementation from being fixed.
>
> I can't see what could break if a sed
> implementation changed to stop outputting an error on
> sed ' # comment'
>
> On the other hand,
>
> sed 'w file # comment'
>
> Can't be changed to allow "# comment" to be treated as a comment
> as that would break scripts that expect that to write in the
> "file # comment" file.
>
> sed 's/x/y/#comment'
>
> could break sed implementations that have # as a s flag as an
> extension (not that I know of any).
>