Re: Speeding up recdescent...

"Yves Orton" <[email protected]> writes:

> Over the weekend I started playing with Parse::RecDescent and decided
> to apply it to a real world problem. Now perhaps using a tool this
> powerful for what I want to do is gross overkill, but it seemed that
> using it resulted in less code that is more maintainable than hand
> rolling a parser. Seeing how much advice in the Monastery tends to
> argue that maintainable code is superior to faster code this route
> seemed ideal. Unfortunately using Parse::RecDescent is _much_ slower
> than I would like. So my question is threefold

> - Should I even be using Parse::RecDescent at all?

Yes, if you can wait for the C-optimized version of the module, which
Damian will release sometime in the future.

-or-

Yes, if your data format is likely to change, and you don't need to
parse the data quickly.

-or-

Yes, if you try the grammar without P:RD and decide it's not worth the
performance gains.  In your case, the grammar looks so simple that it
may be overkill to use P:RD.

> - Can I change the grammer to make it more efficient?

Avoid the type_t declaration with an optional regex, you should use

type_t      :   /ADD|ADDRANGE|DELETE|DELETERANGE/

or (even better)

ranged_type_t : /ADDRANGE|DELETERANGE/
simple_type_t : /ADD|DELETE/

and that will make the parsing simpler and faster.  But you're likely
to see small improvements from this and others.  P:RD is slower than a
specialized solution.

> - Are there any recommendations for what is in effect my first attempt
>   at using this very cool tool.

Look at the ways other people have used P:RD.  There's many examples
out there.

> The data (a file) that I need to parse looks like this:
>
> HDRCOMPNAME BIG000OLD111IDENTIFIER1020301WITH1010LOTS1010OF1010CRAP
> ADD,1234567890,,COMPNAME
> ADDRANGE,2468,4680,COMPNAME
> DELETE,987654321,,COMPNAME
> DELETERANGE,13579,13599,COMPNAME
> TLR000004
>
> and the grammer I am using looks like this:
>
> my $Grammar=<<'END_GRAMMAR';
>
> startrule   :   file
>
> file        :   header record(s?) trailer_t
>                 { $return={
>                            header=>$item[1],
>                            records=>$item[2],
>                            count=>$item[3]
>                           }
>                 }
>
> header      :   header_t data_t
>                 {
>                  $return={
>                           company=>$item[1],
>                           code=>$item[2]
>                          }
>                 }
>
> record      :   valid_rec | <error>
>
> valid_rec   :   type_t ',' number_t ',' number_t(?) ',' name_t
>                 { $return=[
>                            $item[1],
>                            $item[3],
>                            @{$item[5]} ? $item[5] : undef,
>                            $item[7]
>                           ]
>                 }
>
> header_t    :   /HDR\w+/ { $return=substr($item[1],3) }
> trailer_t   :   /TLR\d+/ { $return=substr($item[1],3) }
>
> data_t      :   /\w+/
> type_t      :   /ADD(?:RANGE)?|DELETE(?:RANGE)/
> number_t    :   /\d+/
> name_t      :   /\w+/
>
> END_GRAMMAR
>
> If its not obvious I have used the postfix _t for tokens.
> Any wisdom regarding this would be really appreciated. Especially if
> there is some way to modify the grammer to enhance speed. These files
> can contain thousands+ records and the speed hit is seriously making
> me think of hand rolling this (which I really really dont want to do).

-- 
Teodor Zlatanov <[email protected]>
"Brevis oratio penetrat colos, longa potatio evacuat ciphos." -Rabelais