[perl #61086] [PATCH] mk_language_shell script generates wrong

# New Ticket Created by  "gabriele renzi" 
# Please include the string:  [perl #61086]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=61086 >


the default grammar for a newly generated language shell contains the
following lines:


rule integer { \d+ {*} }
rule quote {
     [ \' <string_literal: '\'' > \' | \" <string_literal: '"' > \" ]
     {*}
 }

since these are defined as "rule" they skip whitespace, so the second
declaration actually forces parsing of literals like
 " "
into
 ""
The attached patch modifies mk_language_shell to generate a grammar
that declares these as tokens and adds an additional test to the
default t/00-sanity.t generated for a new lanaguage shell that takes
this into account.

All parrot tests still pass, cause there is nothing testing
mk_language_shell output, AFAICT.
The documentation in languages/squaak/doc may be updated to reflect
the change but it seems the only direct reference is

languages/squaak/doc/tutorial_episode_3.pod:Rename the rule C<integer>
as C<integer_constant>, and C<quote>...
whereas it may say "token" but I guess it's pretty fine even like this.

Parrot svn at revision 33548.

Index: tools/dev/mk_language_shell.pl
===================================================================
--- tools/dev/mk_language_shell.pl	(revision 33548)
+++ tools/dev/mk_language_shell.pl	(working copy)
@@ -364,9 +364,9 @@
     | <quote> {*}                                #= quote
 }
 
-rule integer { \d+ {*} }
+token integer { \d+ {*} }
 
-rule quote {
+token quote {
     [ \' <string_literal: '\'' > \' | \" <string_literal: '"' > \" ]
     {*}
 }
@@ -535,10 +535,11 @@
 
 __t/00-sanity.t__
 # This just checks that the basic parsing and call to builtin say() works.
-say '1..3';
+say '1..4';
 say 'ok 1';
 say 'ok ', 2;
 say 'ok ', 2 + 1;
+say 'ok', ' ', 4;
 
 __DATA__