Re: (SPAM?) space-separated tokens (FAQ?)

Scott wrote:
> On Tue, Jun 28, 2005 at 09:58:56AM -0700, Ron Smith wrote:
> 
> Well, here are my results (where test.pl was the file I cut and pasted
> into the original email):
> 
> [0 ~/string/spl]$ perl ./test.pl
>   thumb pick up  far  little finger  string  
>   thumb pick up  top far outer  little finger  string  
>   thumb move under near  forefinger  string, pick up  center near inner  little finger  string  
>   thumb pick up  near middle  forefinger  string  
>   thumb move under near  forefinger  string, pick up  center near inner  little finger  string  (Yucch!)

Oops, my bad.  I didn't copy "package SFNParse;".  Makes all the 
difference...


>>>>Fundamentally you need to decide if white-space is part of your
>>>>grammar. 
>>>
>>>
>>>As is evident from my question, it is.
>>
>>No, see that is the point, it is not "evident" as there is more than one 
>>way to do it, and one of those ways may not really require white
>>space.
> 
> 
> We WANT white space. This is the way we want to do it. How do we do
> it? That was the question.

NO, we *don't* want the white space, if we can avoid it.


> Hmmm. Is it 'together' or 'to get her'? Who is she? Who's on first?

Touché.


> Isthatreallyhowyoureadtext?IfsothenIcanreallysaveawholelotofwearandtearonmythumbsbynotbotheringtoeverpressthespacebaronthiskeyboard!Thankyouverymuchforthishelp,Iwilltreasureitalways.Wasthata'spacebar'ora'spacebaron'?Whocares,asthereisnospace.Wewantspacescanyoutellushoworisitjustnotapossibility?

Originally, I debated whether or not to respond to this in the above, 
without any white space.

> I've proven it to myself. Above run _was_ done in emacs. (Is there any
> other editor?) Sorry it doesn't seem to work out on your setup....

No, there is no other editor.  At least you belong to the True Religion.

> 
> If anyone on this list can address the question of how best to attack
> input as a series of space-separated tokens
> insteadofasteadystreamofcharacters, please let me know.... 

How ironic that I am the only one responding...

> 
> Thanks,
> Scott.
> 
> 

OK, there is more than one way to do it.

Way number one:

lat:	("i" | "o" | /m\b/) {$SFNParse::abbrevs{$item[1]};}
.....................^.^^^

forces a "break", usually with white space given your grammar.  Doing 
this gives:

   thumb pick up  far  little finger  string
   thumb pick up  top far outer  little finger  string
   thumb move under near  forefinger  string, pick up  center near inner 
  little finger  string
   thumb pick up  near  forefinger  string, move under center near inner 
  little finger  string  (this is now right - maybe?)
   thumb move under near  forefinger  string, pick up  center near inner 
  little finger  string  (Yucch! But it parses....)


Which I think is what you want.  (You never *did* say what you were 
looking *for*.  So I'm guessing here.

Way number two is:

lat:	("i" | "o" | "m" ...!rel_move ) {$SFNParse::abbrevs{$item[1]};}

which is the "lookahead" I mentioned previously.

Now note that it *still* works, 
andevenworksinyournowhitespacecasethatyousodetest:

1 pu 5f
   thumb pick up  far  little finger  string

1 pu 5tfo
   thumb pick up  top far outer  little finger  string

1 mu 2n pu 5cni
   thumb move under near  forefinger  string, pick up  center near inner 
  little finger  string

1 pu 2n mu 5cni # this is now right
   thumb pick up  near  forefinger  string, move under center near inner 
  little finger  string  (this is wrong)

1pu2nmu5cni # this is the no white space case
   thumb pick up  near  forefinger  string, move under center near inner 
  little finger  string  (this is wrong)

1mu2npu5cni     # Yucch! But it parses....
   thumb move under near  forefinger  string, pick up  center near inner 
  little finger  string  (Yucch! But it parses....)


Noticethatwhitespacehasnothingtodowithitsavingwearandtearonboththespacebaraswellasyourthumb.