Re: [perl #133541] Grammer bug <alnum> vs <alpha>

The issue doesn't seem to be the underscore, because I get the same resul=
t even when converting the underscore into a letter ('b'):

$ cat gentb.p6=20
grammar G0 {
	token TOP {<rport>|<ruport>.*}
	regex rport { <type>}
	rule ruport { <type>}
	#token type {<alpha>+}
	token type {<alnum>+}
}

grammar G1 {
	token TOP {<rport>|<ruport>.*}
	regex rport { <type>}
	rule ruport { <type>}
	token type {<alpha>+}
	#token type {<alnum>+}
}
my $str=3D"scbin<foo> bar";
say "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D <alnum> Example=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D";
say G0.parse($str);
say "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D <alpha> Example=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D";
say G1.parse($str);

$ perl6 gentb.p6
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D <alnum> Example=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D
Nil
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D <alpha> Example=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D
=EF=BD=A2scbin<foo> bar=EF=BD=A3
 ruport =3D> =EF=BD=A2scbin=EF=BD=A3
  type =3D> =EF=BD=A2scbin=EF=BD=A3
   alpha =3D> =EF=BD=A2s=EF=BD=A3
   alpha =3D> =EF=BD=A2c=EF=BD=A3
   alpha =3D> =EF=BD=A2b=EF=BD=A3
   alpha =3D> =EF=BD=A2i=EF=BD=A3
   alpha =3D> =EF=BD=A2n=EF=BD=A3
$=20


On Fri, Sep 28, 2018 at 02:26:41AM -0700, Brent Laabs wrote:
> Are you sure about that?  Underscore has been part of the specs (synops=
es)
> for <alpha> for at least 10 years, probably longer.
>=20
>  >  "_" ~~ /<alpha>/
> =EF=BD=A2_=EF=BD=A3
>  alpha =3D> =EF=BD=A2_=EF=BD=A3
>=20
> On Thu, Sep 27, 2018 at 7:52 PM Brandon Allbery <[email protected]> w=
rote:
>=20
> > "_" is not an alphabetic character. It's allowed in "alnum" because t=
hat
> > is by intent what is \w in other regex implementations, which include=
s "_".
> >
> > On Thu, Sep 27, 2018 at 10:47 PM Vijayvithal <perl6-bugs-followup@per=
l.org>
> > wrote:
> >
> >> # New Ticket Created by  Vijayvithal
> >> # Please include the string:  [perl #133541]
> >> # in the subject line of all future correspondence about this issue.
> >> # <URL: https://rt.perl.org/Ticket/Display.html?id=3D133541 >
> >>
> >>
> >> In the attached code, the only difference between the Grammars G0 an=
d G1
> >> is the defination of token 'type' it is defined as <alpha> in one ca=
se
> >> and as <alnum> in another.
> >>
> >> Since the string being matched is 'sc_in' both the alpha and alnum
> >> tokens should have captured it. But we see the following result on
> >> execution
> >>
> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D <alnum> Example=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
> >> Nil
> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D <alpha> Example=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
> >> =EF=BD=A2sc_in<foo> bar=EF=BD=A3
> >> ruport =3D> =EF=BD=A2sc_in=EF=BD=A3
> >> type =3D> =EF=BD=A2sc_in=EF=BD=A3
> >> alpha =3D> =EF=BD=A2s=EF=BD=A3
> >> alpha =3D> =EF=BD=A2c=EF=BD=A3
> >> alpha =3D> =EF=BD=A2_=EF=BD=A3
> >> alpha =3D> =EF=BD=A2i=EF=BD=A3
> >> alpha =3D> =EF=BD=A2n=EF=BD=A3
> >>
> >>
> >> Perl Version is
> >>
> >> This is Rakudo Star version 2018.06 built on MoarVM version 2018.06
> >> implementing Perl 6.c.
> >>
> >>
> >>
> >> --
> >> Vijayvithal
> >> Dyumnin Semiconductors
> >>
> >
> >
> > --
> > brandon s allbery kf8nh
> > [email protected]
> >