Re: [Fwd: [Bratislava-pm] XML::LibXML performance]

Hi,

The XS <-> Perl interface is expensive.

#!/usr/bin/perl

use XML::LibXML;
use Benchmark;

$e = XML::LibXML::Element->new( "foo" );

timethese( 1000000, {
    'ref' => sub { if( ref($e) eq "XML::LibXML::Element" ) { $i++ } },
    'nodetype' => sub { if( $e->nodeType == XML_ELEMENT_NODE ) { $j++ } },
} );

Using "ref" is about 4x faster.

All the best,
Tim.

Michael Kröll wrote:
> Interesting findings on XML::LibXML performance - English text starting
> after the Slovak version.
>
> Attachments at
> http://mail.pm.org/pipermail/bratislava-pm/2008-September/000107.html
>
> --michael
>
> -------- Original Message --------
> Subject: [Bratislava-pm] XML::LibXML performance
> Date: Tue, 09 Sep 2008 23:41:07 +0200
> From: Emmanuel Rodriguez <[email protected]>
> To: [email protected]
>
> Ahojte,
>
> Jozef mi povedal cez víkend, že mal Perl kód ktorý používal XML::LibXML
> a bol veľmi pomalý. Ak si dobre pamätám, prvá verzia potrebovala asi 20
> sekúnd a potom Jozef urobil druhú verziu, ktorá potrebovala 1.5 sekundy.
> Prvá verzia používala veľa XPath tak nebola taká rýchla, druhá verzia
> používala XPath len jeden krát a bola rýchlejšia a potom urobila všetko
> cez normálny DOM. Keď je XML jednoduché, lepšie je použiť DOM ako XPath
> aj keď kód bude trochu dlhší ale program bude rýchlejší.
>
> Bol som veľmi zvedavý lebo používam veľa XML::LibXML a chcel by som
> vidieť v čom bol problém. Pracoval som trochu s Jozefovým kódom a teraz
> máme verziu, ktorá potrebuje len pol sekundy, ale to je tiež pomalé
> podľa mňa.
>
> Súbor je asi 474K a má 17743 elementy, čo nie je veľa ale čítať tento
> súbor nie je v Perle veľmi rýchle. Meral som rýchlosti a zistil som, že
> chodiť cez celý strom v C je veľmi rýchle a potrebuje len 0.030 sekundy,
> ale v Perle to isté potrebuje 0.450 sekundy, a to je 15 krát pomalšie!
>
> V C chodiť cez DOM je veľmi ľahké a rýchle lebo všetko je cez "pointers"
> napríklad:
> node = node->next;
> if (node->type == XML_ELEMENT_NODE)
> process_element(node->children);
>
> Ale v Perle to isté nemôže byt cez "pointers" a tento kód musí používať
> funkcie:
> $node = $node->nextSibling();
> if ($node->nodeType == XML_ELEMENT_NODE)
> process_element($node->firstChild());
>
> Neviem presne prečo kód je taký pomalý v Perle, myslím že to je lebo
> máme veľa elementy a každý DOM operácia potrebuje jeden funkcia v Perl.
> Možno XS je drahý? Pozeral som XS kód a tieto tri funkcie sú veľmi krátke:
>
> SV*
> nextSibling( self )
> xmlNodePtr self
> ALIAS:
> getNextSibling = 1
> CODE:
> RETVAL = PmmNodeToSv( self->next,
> PmmOWNERPO(PmmPROXYNODE(self)) );
> OUTPUT:
> RETVAL
>
> int
> nodeType( self )
> xmlNodePtr self
> ALIAS:
> XML::LibXML::Node::getType = 1
> CODE:
> RETVAL = self->type;
> OUTPUT:
> RETVAL
>
> SV*
> firstChild( self )
> xmlNodePtr self
> ALIAS:
> getFirstChild = 1
> CODE:
> RETVAL = PmmNodeToSv( self->children,
> PmmOWNERPO( PmmPROXYNODE(self) ) );
> OUTPUT:
> RETVAL
>
> ----
> Hi,
>
> This weekend Jozef told that he had a Perl program that was using
> XML::LibXML which as very slow. If I recall well, the first version of
> his program needed in average 20 seconds. Latter Jozef made a second
> version which needed only 1.5 seconds. His first version was using
> exhaustively XPath while the second version used XPath only once and
> replaced the other calls by simple DOM references. Xpath is very nice
> when dealing with complex documents or when the queries are very narrow,
> but when the XML is simple, the best is to revert to DOM, sure the code
> will be slightly longer but the gain in speed will be considerable.
>
> Never the less, I was puzzled by the problem that plagued Jozef's code,
> specially because I'm using a lot XML::LibXML and I wanted to know that
> was the source of the problem. I managed to work a little bit with
> Jozef's sample code and I managed to remove a second from the execution
> time. But passing from 1.5 seconds to 0.5 it's not too convincing, the
> program is still slow.
>
> It's interesting to notice that the input file used has 474K and
> contains 17743 elements. That's not a lot, but still parsing such an
> input file in Perle is very slow. I benchmarked the program and I found
> out that walking through the DOM tree in C is very fast and can be done
> in 0.030 seconds, but in Perl the same operation needs 0.450 seconds,
> that's 15 times slower!
>
> I think that the reason of the speed difference is due to the way that
> the DOM tree is walked. In C crossing the DOM tree is not expensive and
> is very fast as everything is done through pointers, for instance:
> node = node->next;
> if (node->type == XML_ELEMENT_NODE)
> process_element(node->children);
>
> But in Perl this same operation can't be done through pointers a the
> same program needs to use functions instead:
> $node = $node->nextSibling();
> if ($node->nodeType == XML_ELEMENT_NODE)
> process_element($node->firstChild());
>
> I'm not sure why the same code in Perl is so slow. I think that the
> problem is not due to the size of the input file but to the numbers of
> elements in the XML document. Probably that the number is so high that
> the XS method calls are becoming quite expensive. I took a look a the XS
> code and the functions are as short as they can be:
>
> SV*
> nextSibling( self )
> xmlNodePtr self
> ALIAS:
> getNextSibling = 1
> CODE:
> RETVAL = PmmNodeToSv( self->next,
> PmmOWNERPO(PmmPROXYNODE(self)) );
> OUTPUT:
> RETVAL
>
> int
> nodeType( self )
> xmlNodePtr self
> ALIAS:
> XML::LibXML::Node::getType = 1
> CODE:
> RETVAL = self->type;
> OUTPUT:
> RETVAL
>
> SV*
> firstChild( self )
> xmlNodePtr self
> ALIAS:
> getFirstChild = 1
> CODE:
> RETVAL = PmmNodeToSv( self->children,
> PmmOWNERPO( PmmPROXYNODE(self) ) );
> OUTPUT:
> RETVAL
>
> For the curious I have joined the programs that I've used to parse the
> files. These programs simply perform a walk of the whole DOM tree. These
> benchmarks are interesting because they show that scanning the whole XML
> file in Perl can't be done in less than 0.450 seconds that's why the
> fastest version of Jozef's parser is doomed to work in about half a
> second. No matter the algorithm that I've used, I couldn't go under the
> 0.500 second barrier. This is as fast as it gets with the current
> implementation of XML::LibXML.
>
>
>
>   

_______________________________________________
Perl-XML mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs