[Fwd: [Bratislava-pm] XML::LibXML performance]

Interesting findings on XML::LibXML performance - English text starting
after the Slovak version.

Attachments at
http://mail.pm.org/pipermail/bratislava-pm/2008-September/000107.html

--michael

-------- Original Message --------
Subject: [Bratislava-pm] XML::LibXML performance
Date: Tue, 09 Sep 2008 23:41:07 +0200
From: Emmanuel Rodriguez <[email protected]>
To: [email protected]

Ahojte,

Jozef mi povedal cez víkend, že mal Perl kód ktorý používal XML::LibXML
a bol veľmi pomalý. Ak si dobre pamätám, prvá verzia potrebovala asi 20
sekúnd a potom Jozef urobil druhú verziu, ktorá potrebovala 1.5 sekundy.
Prvá verzia používala veľa XPath tak nebola taká rýchla, druhá verzia
používala XPath len jeden krát a bola rýchlejšia a potom urobila všetko
cez normálny DOM. Keď je XML jednoduché, lepšie je použiť DOM ako XPath
aj keď kód bude trochu dlhší ale program bude rýchlejší.

Bol som veľmi zvedavý lebo používam veľa XML::LibXML a chcel by som
vidieť v čom bol problém. Pracoval som trochu s Jozefovým kódom a teraz
máme verziu, ktorá potrebuje len pol sekundy, ale to je tiež pomalé
podľa mňa.

Súbor je asi 474K a má 17743 elementy, čo nie je veľa ale čítať tento
súbor nie je v Perle veľmi rýchle. Meral som rýchlosti a zistil som, že
chodiť cez celý strom v C je veľmi rýchle a potrebuje len 0.030 sekundy,
ale v Perle to isté potrebuje 0.450 sekundy, a to je 15 krát pomalšie!

V C chodiť cez DOM je veľmi ľahké a rýchle lebo všetko je cez "pointers"
napríklad:
node = node->next;
if (node->type == XML_ELEMENT_NODE)
process_element(node->children);

Ale v Perle to isté nemôže byt cez "pointers" a tento kód musí používať
funkcie:
$node = $node->nextSibling();
if ($node->nodeType == XML_ELEMENT_NODE)
process_element($node->firstChild());

Neviem presne prečo kód je taký pomalý v Perle, myslím že to je lebo
máme veľa elementy a každý DOM operácia potrebuje jeden funkcia v Perl.
Možno XS je drahý? Pozeral som XS kód a tieto tri funkcie sú veľmi krátke:

SV*
nextSibling( self )
xmlNodePtr self
ALIAS:
getNextSibling = 1
CODE:
RETVAL = PmmNodeToSv( self->next,
PmmOWNERPO(PmmPROXYNODE(self)) );
OUTPUT:
RETVAL

int
nodeType( self )
xmlNodePtr self
ALIAS:
XML::LibXML::Node::getType = 1
CODE:
RETVAL = self->type;
OUTPUT:
RETVAL

SV*
firstChild( self )
xmlNodePtr self
ALIAS:
getFirstChild = 1
CODE:
RETVAL = PmmNodeToSv( self->children,
PmmOWNERPO( PmmPROXYNODE(self) ) );
OUTPUT:
RETVAL

----
Hi,

This weekend Jozef told that he had a Perl program that was using
XML::LibXML which as very slow. If I recall well, the first version of
his program needed in average 20 seconds. Latter Jozef made a second
version which needed only 1.5 seconds. His first version was using
exhaustively XPath while the second version used XPath only once and
replaced the other calls by simple DOM references. Xpath is very nice
when dealing with complex documents or when the queries are very narrow,
but when the XML is simple, the best is to revert to DOM, sure the code
will be slightly longer but the gain in speed will be considerable.

Never the less, I was puzzled by the problem that plagued Jozef's code,
specially because I'm using a lot XML::LibXML and I wanted to know that
was the source of the problem. I managed to work a little bit with
Jozef's sample code and I managed to remove a second from the execution
time. But passing from 1.5 seconds to 0.5 it's not too convincing, the
program is still slow.

It's interesting to notice that the input file used has 474K and
contains 17743 elements. That's not a lot, but still parsing such an
input file in Perle is very slow. I benchmarked the program and I found
out that walking through the DOM tree in C is very fast and can be done
in 0.030 seconds, but in Perl the same operation needs 0.450 seconds,
that's 15 times slower!

I think that the reason of the speed difference is due to the way that
the DOM tree is walked. In C crossing the DOM tree is not expensive and
is very fast as everything is done through pointers, for instance:
node = node->next;
if (node->type == XML_ELEMENT_NODE)
process_element(node->children);

But in Perl this same operation can't be done through pointers a the
same program needs to use functions instead:
$node = $node->nextSibling();
if ($node->nodeType == XML_ELEMENT_NODE)
process_element($node->firstChild());

I'm not sure why the same code in Perl is so slow. I think that the
problem is not due to the size of the input file but to the numbers of
elements in the XML document. Probably that the number is so high that
the XS method calls are becoming quite expensive. I took a look a the XS
code and the functions are as short as they can be:

SV*
nextSibling( self )
xmlNodePtr self
ALIAS:
getNextSibling = 1
CODE:
RETVAL = PmmNodeToSv( self->next,
PmmOWNERPO(PmmPROXYNODE(self)) );
OUTPUT:
RETVAL

int
nodeType( self )
xmlNodePtr self
ALIAS:
XML::LibXML::Node::getType = 1
CODE:
RETVAL = self->type;
OUTPUT:
RETVAL

SV*
firstChild( self )
xmlNodePtr self
ALIAS:
getFirstChild = 1
CODE:
RETVAL = PmmNodeToSv( self->children,
PmmOWNERPO( PmmPROXYNODE(self) ) );
OUTPUT:
RETVAL

For the curious I have joined the programs that I've used to parse the
files. These programs simply perform a walk of the whole DOM tree. These
benchmarks are interesting because they show that scanning the whole XML
file in Perl can't be done in less than 0.450 seconds that's why the
fastest version of Jozef's parser is doomed to work in about half a
second. No matter the algorithm that I've used, I couldn't go under the
0.500 second barrier. This is as fast as it gets with the current
implementation of XML::LibXML.



-- 
063A F25E B064 A98F A479  1690 78CD D023 5E2A 6688
http://zis.uibk.ac.at/.m/uibk.ac.at_pgp_pubkey.asc

_______________________________________________
Perl-XML mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs