RE: [code-review] list scope

I would have thought that showing any code would be fine. Everyone still
learns from the analysis even if they never have a use for the code.

I've run into something similar to your problem before. If you end up
running various tweaked analyses on the same big, hard to parse files you
may get great benefit from preparsing them into a faster to parse format.
You could use Storable for instance but for maximum speed I think you won't
beat pack/unpack.

It's a bit of work to split your process into a parse and dump phase,
followed by an undump and analyse phase but if you end up reanalysing even
once, it sounds like you'll get that time back pretty quickly.

Of course improving the parsing sounds like it would be worth doing too. In
that same project I made good gains also by switching some method calls
inside tight loops to be function calls. Of course this is not something to
do without careful thought but I had some methods that were never called by
anything outside their own class and so I just turned them into functions.

Finally, putting loops _inside_ methods rather than around them can be a
big help. So convert from

foreach my $line (@lines)
{
  push(@parsed, $parser->parse_line($line));
}

to

push(@parsed, $parser->parse_lines(@lines));

and if you need it you can just reimplement parse_line as a wrapper around
parse_lines.

If @lines is enormous or is coming in from a file then split them into
batches and you'll still see a speed up.

Obviously there's not much benefit if parse_lines needs 5 minutes per line,

F


--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .