Re: Stow doesn't use the "ignore" option given in the rc file

On Wed, Aug 24, 2016 at 03:23:02PM +0000, Kristoffer Haugsbakk wrote:
>This is a reply to [Adam Spiers' answer] to my question/bug report on 
>Unix & Linux StackExchange.  The title of this email is the name of the 
>question that I posted. 
>
>[Adam Spiers' answer]: http://unix.stackexchange.com/a/305225/185805

Firstly thanks a lot for this email, and sorry for the terribly slow 
reply! 

>>Thanks for the excellent bug report!  I can answer your 
>>questions, and as the Stow maintainer I can also fix the issues, 
>>but I'd appreciate your feedback from a UX perspective so we can 
>>figure out the best fix. 
>>
>>Firstly, it's worth noting that `--verbose=5` will give you much 
>>more detail about the internals of the ignore mechanism, although 
>>in this case it would not be sufficient to explain why things are 
>>not behaving the way you expect. 
>
>The [online manual] says that verbosity goes up to 3.   The info file in 
>Emacs says the same.   My man page says that it goes up to 4 (maybe 
>that's correct for version 2.2.0).  Maybe the manuals need to be 
>synchronized? 

Yes thanks for reporting this - I've done a fix which I'll push to git 
(and hopefully a release) soon. 

>[online manual]: https://www.gnu.org/software/stow/manual/stow.html#Invoking-Stow
>
>>There are two reasons why neither of your `.stowrc` files worked: 
>>
>>1. The `.stowrc` parser splits (option, value) pairs based on the
>>      space character, not on `=`.  So a line for an ignore option
>>      in that file should start `--ignore ` not `--ignore=`.
>>2. The `.stowrc` parser doesn't automatically strip quotes.  When
>>      the `--ignore` option (or any other option, for that matter)
>>      is passed via the CLI, the shell will strip the quotes before
>>      Stow sees them.  That's why it works there.
>
>When making my `--ignore` options I used the examples in the online 
>manual (see section 11 "Resource Files") as a guideline.  There `=` and 
>single quotes are used, e.g.: 
>
>   --ignore='~'
>   --ignore='^CVS'
>
>The same goes for the info file.   I assumed that the quotes were 
>supposed to be stripped, or else the CVS example wouldn't make much 
>sense to me.  I also prefer to use single quotes since then I won't have 
>to fret about globbing in the shell.  (In hindsight I guess that doesn't 
>make sense for what is just a string that will be copied around in a 
>script.  I'm not skilled with shells so I try to play it safe.) 
>
>So my assumption was that the options given in the file would be the 
>same as passing them on the command line.   The online manual says that 
>this is "effectively" what happens: 
>
>   [Default command line options] are parsed in that order, and 
>   effectively prepended to you command line. 
>
>Though perhaps "effectively" is supposed to be interpreted more 
>conservatively; again, I am not proficient with shells and their 
>limitations. 

"Effectively" refers to the fact that CLI arguments are first 
processed by the shell before reaching Stow, whereas the contents of 
.stowrc are not.  The shell will strip quotes.  Quotes in arguments 
in .stowrc will not be stripped, but then you wouldn't need them there 
anyway because there is no expansion to defend against.  The only 
possible exception to this is if values contain spaces: 

    https://github.com/aspiers/stow/issues/15

>I don't know if higher verbose levels do this (my version doesn't seem 
>to), but it might be good if a certain verbose level and higher printed 
>the `stow` command that was executed once the resource file(s) were 
>taken into account.  Maybe there also could be an option that prints 
>what the `stow` command that is sent looks like, in addition to or 
>instead of including it in the `verbose` output. 

I'm not sure that's the most useful thing to do, because how the 
command looks after prepending .stowrc doesn't make it clear how shell 
expansion is done.  A more useful thing is to say what values are 
being used for each option, and actually that's basically already 
done, since with -v2 or higher it tells you what stow dir and target 
dir it is using. 

>This could perhaps be 
>combined with the `--simulate` option to see what command would run when 
>taking into account the resource files.   This could make any differences 
>between the format of the resource files and the final output more 
>obvious, and simpler than (1) figuring out which resource file is used 
>(maybe you happen to be in the wrong directory), and (2) reading it and 
>combining the options given in that file with the ones you are invoking 
>directly in your mind. 

I think -v2 --simulate should already give you the information you 
need.  Am I missing something? 

>Since I have no idea about how hard certain things are to implement, 
>I'll be talking strictly about the UX side of things.   Just keep that in 
>mind. 

OK, that's still valuable feedback :-) 

>I think that, ideally, the format of the options in the stowrc files 
>should be as close to how they would be when passed directly on the 
>terminal as reasonably possible.   As to the stripping of the quotes, I 
>think it would be better to strip the quotes, since that would be what 
>would happen in the terminal and shell scripts.   If someone wants the 
>quotes, maybe they could escape them.  But again, I have no idea how 
>hard this is to implement. 

Hmm, I'm not sure about this.  It might be more intuitive, but that 
sets the expectation that the contents of .stowrc and the CLI are 
treated identically.  That could be attempted by either reimplementing 
shell expansion inside Stow (which would be a gross duplication of 
effort), or passing the contents of .stowrc to a new shell for 
expansion.  However neither of these can reliably reproduce the exact 
identical expansion which the CLI args are subject to.  For example, 
if you set a non-exported shell variable: 

    foo=bar

then it would be simply impossible for 

    --target=baz$foo

to be expanded identically from both the CLI and .stowrc, because Stow 
has no knowledge of the value of the $foo variable in order to expand 
it.  Another corner-case would be the expansion of $$ and $$PID, for 
which expansion by a new shell would yield totally different values 
to when expanded by the original shell from which stow was invoked. 

Additionally, shell expansion is already a complicated thing to 
understand, and the results of invoking it in a non-standard way from 
values stored in a file would be even harder to define and implement. 

So instead I'd prefer to keep things simple: 

1. clarify the status quo in the manual
2. partially implement the most useful aspects of shell expansion,
   for example expansion of ~ and environment variables.  Fortunately
   this work is already underway, thanks to excellent contributions
   from Charles LeDoux which I shamefully still haven't managed to
   review:

     https://github.com/aspiers/stow/issues/13
     https://github.com/aspiers/stow/issues/14

>Maybe there is some precedent when it comes to how resource files tend 
>to handle options? 

Well, not a precedent as such, but they simply won't undergo any type 
of shell expansion (yet). 

>With regards to the linked issue: I think having one option per line 
>sounds good.   This is what I guessed when I read the manual, based on 
>the examples.   But just to be clear about it, maybe the manual (online 
>and info file) could specify that explicitly?  A sentence like this 
>could be added to the first paragraph: 
>
>   Each line contains one option only.

Yes, more clarity is certainly needed. 

>>Finally, I believe your `.stow-local-ignore` did not work because 
>>you placed it in the stow directory rather than in the `a/` 
>>package directory.  The documentation about this seems clear to 
>>me, so I think it's fair to write this one off as pilot error. 
>>However if you have any suggestions for how to make the docs 
>>clearer than I'm all ears. 
>
>Yes, the manual is clear about that.   The only possible change would be 
>to explicitly mention that resource files and ignore files are put in 
>different places.   But I don't know if that would be an improvement or 
>just minor noise. 

Hmm, AFAICS the manual already explicitly explains the locations of 
these files.  Am I missing something? 

>Thanks again for your help. 

Welcome, and sorry again it took so long! 

>PS Sorry if my webclient ruins the formatting. 

It didn't ;-)