Re: [perl #133541] Grammer bug <alnum> vs <alpha>

------------=_1538442871-12758-1
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="utf-8"

This issue surfaces because of the token TOP line. If instead of
<rport>|<ruport> only ruport was used the testcase works for both cases. So it is quite
possible that the bug is elsewhere but shows up as a difference between
alpha and alnum.

Regards
Vijay


On Fri, Sep 28, 2018 at 07:18:49AM -0700, Patrick R. Michaud via RT wrote:
> The issue doesn't seem to be the underscore, because I get the same result even when converting the underscore into a letter ('b'):
> 
> $ cat gentb.p6 
> grammar G0 {
> 	token TOP {<rport>|<ruport>.*}
> 	regex rport { <type>}
> 	rule ruport { <type>}
> 	#token type {<alpha>+}
> 	token type {<alnum>+}
> }
> 
> grammar G1 {
> 	token TOP {<rport>|<ruport>.*}
> 	regex rport { <type>}
> 	rule ruport { <type>}
> 	token type {<alpha>+}
> 	#token type {<alnum>+}
> }
> my $str="scbin<foo> bar";
> say "=========== <alnum> Example==============";
> say G0.parse($str);
> say "=========== <alpha> Example==============";
> say G1.parse($str);
> 
> $ perl6 gentb.p6
> =========== <alnum> Example==============
> Nil
> =========== <alpha> Example==============
> ｢scbin<foo> bar｣
>  ruport => ｢scbin｣
>   type => ｢scbin｣
>    alpha => ｢s｣
>    alpha => ｢c｣
>    alpha => ｢b｣
>    alpha => ｢i｣
>    alpha => ｢n｣
> $ 
> 
> 
> On Fri, Sep 28, 2018 at 02:26:41AM -0700, Brent Laabs wrote:
> > Are you sure about that?  Underscore has been part of the specs (synopses)
> > for <alpha> for at least 10 years, probably longer.
> > 
> >  >  "_" ~~ /<alpha>/
> > ｢_｣
> >  alpha => ｢_｣
> > 
> > On Thu, Sep 27, 2018 at 7:52 PM Brandon Allbery <[email protected]> wrote:
> > 
> > > "_" is not an alphabetic character. It's allowed in "alnum" because that
> > > is by intent what is \w in other regex implementations, which includes "_".
> > >
> > > On Thu, Sep 27, 2018 at 10:47 PM Vijayvithal <[email protected]>
> > > wrote:
> > >
> > >> # New Ticket Created by  Vijayvithal
> > >> # Please include the string:  [perl #133541]
> > >> # in the subject line of all future correspondence about this issue.
> > >> # <URL: https://rt.perl.org/Ticket/Display.html?id=133541 >
> > >>
> > >>
> > >> In the attached code, the only difference between the Grammars G0 and G1
> > >> is the defination of token 'type' it is defined as <alpha> in one case
> > >> and as <alnum> in another.
> > >>
> > >> Since the string being matched is 'sc_in' both the alpha and alnum
> > >> tokens should have captured it. But we see the following result on
> > >> execution
> > >>
> > >> =========== <alnum> Example==============
> > >> Nil
> > >> =========== <alpha> Example==============
> > >> ｢sc_in<foo> bar｣
> > >> ruport => ｢sc_in｣
> > >> type => ｢sc_in｣
> > >> alpha => ｢s｣
> > >> alpha => ｢c｣
> > >> alpha => ｢_｣
> > >> alpha => ｢i｣
> > >> alpha => ｢n｣
> > >>
> > >>
> > >> Perl Version is
> > >>
> > >> This is Rakudo Star version 2018.06 built on MoarVM version 2018.06
> > >> implementing Perl 6.c.
> > >>
> > >>
> > >>
> > >> --
> > >> Vijayvithal
> > >> Dyumnin Semiconductors
> > >>
> > >
> > >
> > > --
> > > brandon s allbery kf8nh
> > > [email protected]
> > >
> 

-- 
Vijayvithal
Dyumnin Semiconductors


------------=_1538442871-12758-1
Content-Type: application/pgp-signature; name="signature.asc"
Content-Disposition: attachment; filename="signature.asc"
Content-Transfer-Encoding: base64
RT-Attachment: 133541/1584832/827948

LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KVmVyc2lvbjogR251UEcg
djIuMC4yMiAoR05VL0xpbnV4KQoKaVFFY0JBRUJBZ0FHQlFKYnJrUFhBQW9K
RUtBS1o5UTJncFVabTgwSC9pNmZQazJPRHJJME9MN1gvOVVUVHM3MAppTUU3
dVN5c0M1UnJ3SmN2ZGxZNWo2TlNQNzJTWTk4OExtV2hncThVZ0VGMkQ2aWxw
SHdWME54clNqVVFkbVBKCjNJTXgyMTlmZWtmY3NJZHpyeW1kN3NWRS9MSDRF
Nko0SmJIcmx6cnlxRnBOeHRvUEprWHJUMkEzNW5nMi8wTjQKYUNLb3RsVFhT
MDdGUHg2UzdIeE5BNlJ3NnpOU3V6bnAzMWFZZWRkUVdxbFlHdEJtY3NQRVNZ
dGtJSXExOUxkdQoyK2FlSjZaVW9pclR5VHJrbC9QNjM0UTZlWlY1VjNmWE44
MTJyampzZUt5R2dGa0d3aDEyWTRzVTRwaFhseFFZCnQ5NlY4bWRBdWRlaEtx
VDJmL3V3N21EdW9mREpwUGlyWTY5V1M0VmRESjJYMEcvT1hTZHJMblgrQWM0
OTNmZz0KPWxvaFUKLS0tLS1FTkQgUEdQIFNJR05BVFVSRS0tLS0tCg==

------------=_1538442871-12758-1--