Re: Perl XML::Parser failing tests

Hi everyone,

I did some research and benchmarks on processing large XML documents -
specifically I benchmarked all the SAX parsers I could find,
XML::Twig, XML::Records, XML::Rules, and two C based parsers across 5
different languages of the Wikipedia. You can find a graph of the
results here: http://triddle.us/XML_parser_benchmark.png

X is the XML parser name (expat.t and libxml.t are the C based SAX interfaces)
Y is the speed in mebibytes
Z is the language of the Wikipedia

The cv file is about 50 megs and the en file is about 22 gigs; the
other files are somewhere in between.

Tyler

On Mon, Mar 22, 2010 at 9:00 PM, David Kaufman <[email protected]> wrote:
> Hi Jenda,
>
> Jenda Krynicky wrote:
>> From: Robin Berjon<[email protected]>
>>> On Mar 17, 2010, at 19:59 , Dave Wombat wrote:
>>>> Or can I safely ignore the errors, and just force install
>>>> XML::Parser?
>>>
>>> [...] why are you using XML::Parser? As best I know, there isn't
>>> much left on CPAN that depends on that, everything's moved on to
>>> either using SAX (through other parsers, including ExpatXS) or
>>> other processing methods offered by XML::LibXML (or sat on top of
>>> it). If you can avoid XML::Parser, do so.
>>
>> As best I know (I just looked) XML::LibXML::SAX [is not ready] So
>> what's actually the golden SAX parser we are supposed to install to
>> solve all our problems? I looked into XML::SAX and a few more and did
>> not find any pointers to available SAX parser. Though I have to admit
>> I did not spend much time looking.
>
> While I don't pretend to know what is golden or what you are supposed to
> install, I have found XML::SAX, paired with the XML::SAX::ExpatXS parser
> to be a reliable, fast solution that's able to handle arbitrarily huge
> xml files efficiently.
>
>> I also know that XML::Twig, which is a fairly popular module is
>> still based on XML::Parser, XML::Rules still uses
>> XML::Parser::Expat, XML::Simple can use both XML::Parser and some
>> SAX2 parser, ...
>
> XML::Twig really is pure *joy* to use.  It does what I want it to, the
> way I want it to, and IMO results in the most concise, expressive and
> readable code.  I haven't tried it on the aforementioned arbitrarily
> huge xml files though, just on some smaller more digestible xml
> structures that fit nicely into memory, so I'd suspect (and expect)
> that the cost of its elegance may be available RAM.
>
> The official Perl SAX page
> http://perl-xml.sourceforge.net/perl-sax/
>
> Perl Parser Performance : XML.com
> http://www.xml.com/pub/a/2004/09/15/pl-perf.html
> (still relevant, six years later (and counting!)!)
>
> hth,
>
> -dave
> _______________________________________________
> Perl-XML mailing list
> [email protected]
> To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
>



-- 
If you wish to make an apple pie from scratch you must first invent
the universe. -- Carl Sagan
_______________________________________________
Perl-XML mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs