Re: more splitting

As Bart pointed out, the two hex versions are not interchangeable. For
clarity, let me call hex-doors my version, which shows exactly what's at
the front and back doors, and hex-var the version below, which shows
exactly what's inside the variable passed by name.

function hex-var()
{
  echo "--- ${(r:25::-:: :)1}"
  if [[ -v $1 ]]; then
    set -- "${(@P)1}"
    print -rC1 -- ${(q+)@}
    echo "\n-----------------------------\n"
    for element ("$@") print -rn -- $element | od -vAx -tx1 -tc
  else
    echo "No such variable: ${(q+)1}"
  fi
}

And, all else equal it's probably more othodox to pass the value of the
> variable than the name.


It all depends on what you want to do. For instance, if you want to see
exactly what's in a variable, you better pass its name to a function like
hex-var than (try to) pass its value to a function like hex-doors because
with Zsh chances are high that if you try to attempt the latter you will
fail.

Here is a first example of failure:

% var=""; hex-var var; hex-doors $var
--- var ---------------------
''

-----------------------------


hex-var correctly shows that var contains the empty string but hex-doors
see nothing at the front door. The "problem" is that Zsh entirely drops
$var if it expands to an empty string. Thus, in this case, hex-doors is
called with no arguments instead of with a single empty string.

You can fix the "problem" by quoting $var, then hex-doors also sees the
empty string:

% var=""; hex-var var; hex-doors "$var"

--- var ---------------------
''

-----------------------------

--- Front door --------------
''

-----------------------------


Here is another example of where $var fails to pass the right value(s) to
hex-doors:

% var=(aa "" bb cc); hex-var var; hex-doors $var
--- var ---------------------
aa
''
bb
cc

-----------------------------

0000000    61  61
           a   a
0000002
0000000    62  62
           b   b
0000002
0000000    63  63
           c   c
0000002
--- Front door --------------
aa
bb
cc

-----------------------------

0000000    61  61
           a   a
0000002
0000000    62  62
           b   b
0000002
0000000    63  63
           c   c
0000002

Again, hex-doors doesn't see the empty string. The "problem" is again that
when Zsh epands $var and replaces it by its values, it drops any empty
string. Thus, it calls hex-doors with 3 strings: "aa", "bb", and "cc",
instead of calling it with the 4 values stored inside var. If we apply the
same trick as above and quote $var, we still don't get the right result:

% var=(aa "" bb cc); hex-var var; hex-doors "$var"
--- var ---------------------
aa
''
bb
cc

-----------------------------

0000000    61  61
           a   a
0000002
0000000    62  62
           b   b
0000002
0000000    63  63
           c   c
0000002
--- Front door --------------
'aa  bb cc'

-----------------------------

0000000    61  61  20  20  62  62  20  63  63
           a   a           b   b       c   c
0000009

Here you can see that the empty string was taken into account because there
are two spaces between aa and bb, while there is only one between bb and
cc. Unfortunately, all the values were jammed together. That's because the
default behavior of quotes on array values is to join their values with
spaces between them. To prevent this joining, we can use the (@) expansion
flag, which tells Zsh to keep array values separated:

% var=(aa "" bb cc); hex-var var; hex-doors "${(@)var}"
--- var ---------------------
aa
''
bb
cc

-----------------------------

0000000    61  61
           a   a
0000002
0000000    62  62
           b   b
0000002
0000000    63  63
           c   c
0000002
--- Front door --------------
aa
''
bb
cc

-----------------------------

0000000    61  61
           a   a
0000002
0000000    62  62
           b   b
0000002
0000000    63  63
           c   c
0000002

Now, hex-doors finally see the same as hex-var. And the good thing is that
the same expression also works for the string variable:

% var=""; hex-var var; hex-doors "${(@)var}"
--- var ---------------------
''

-----------------------------

--- Front door --------------
''

-----------------------------


What this shows is that you can indeed use hex-doors to see what's inside a
variable but it requires being very careful on how you pass that variable's
value to hex-doors. If your aim is to see what's inside a variable, it's
much more reliable to pass its name to hax-var.

Now, to see whether you better understand what's going on, you can try to
guess what's the result of the following three calls to hex-doors:

% var=(aa "" bb cc)
% hex-doors foo${var}bar
% hex-doors "foo${var}bar"
% hex-doors "foo${(@)var}bar"

If it's at the front door the splitting is retained.


It all depends on how you refer to the variable. With $var, array values
remain separate values but empty strings are lost, with "$var" array values
are joined together, it's only with "${(@)var}" (or "${var[@]}") that all
array values, including empty strings, are kept and remain separate values.

It really is impossible to know from the pipe version where the input
> array was split, the information no longer exists.


Indeed, in a similar way, I can't tell whether you typed your reply with
your hands, or with your feet, whether you used some voice-to-text tool, or
whether it was your cat that randomly jumped on your keyboard. The only
thing I see is the resulting mail.


Still, there is an ambiguity:
>


% var=("a b" c$'\n''d e f'' ''g h' ij)
> % var2=( ${(f)var} )
>


It seems that the splitting char is always removed, thus one can't look
> at $var2 and determine that it was created via split on newlines,


Obviously not. What you have to understand is that var2 doesn't store the
literal expression ${(f)var}. *It stores the result of its expansion. *

one
> might just as well conclude that it was split on spaces, with a space
> between 'c' and 'd' instead of a newline.


Indeed, there are plenty of ways to generate the same result. Here 6
different ways of initializing var2 to the same values, namely "aa", "bb",
and "cc"

var2=( aa bb cc )                         # Explicit values
var1=( aa bb cc ); var2=( $var1 )         # Expansion of array variable
var1=$'aa\nbb\ncc'; var2=( ${(f)var1} )   # Expansion of string
variable split by newlines
var1="aa bb cc"; var2=( ${(s: :)var1} )   # Expansion of string
variable String split by spaces
var2=( $(echo "aa bb cc") )               # String printed and split by
spaces (and by all other characters present in IFS)
var2=( aa $(echo "bb cc") )               # Mix of explicit values and
split of printed string


> This seems to me an
> imperfection, tho obviously one would have to go back to the 70s to have
> done anything about it.  In the interests of perfect information, I
> wonder if there's any way that even the 'od' output could inform us that
> the output of $var2 is an array split on newlines?  I suspect that 'od'
> has no way of knowing.


How should that work? How could var2 remember in which of the 6 ways listed
above it was initialized? Where should that information be stored? And what
for?

I suspect that some of your confusion comes from C. In C, when you call a
function with a variable like in foo(var), the function foo receives
exactly one argument which contains the value of var. In Zsh, the
equivalent foo $var works very differently. In Zsh, the expression $var
gets first expanded *before foo is called* and foo is then called with the
values resulting from that expansion. As we have seen above, depending on
the nature and content of the variable var, the expansion of $var may lead
to zero, one, or more values, which will translate into as many arguments
for foo. When foo is finally called, foo only sees the result of the
expansion of $var. It can't know whether its arguments were provided as
explicit values, are the result of the expansion of a variable, come from a
command substitution, or were produced in some other fashion.

Philippe