Re: parsing a huge xml document

From: subs subs <[email protected]>
> >Depends on what do you mean by "parse". I mean, sure you want to
> >extract the tags and their attributes and contents, but what then?
> 
> I have to extract the tags and attributes and insert the values into
> database tables. Any ideas what would be the memory requirements for 4GB xml
> doc if I use XML::twig. I would also like to know the memory requirements
> for XML::SAX

That depends on the size of "twigs". That is the part of XML that you 
can process as a unit. I mean if the XML looks like

<root>
 <record>....</record>
 <record>....</record>
 <record>....</record>
 ... several milions of those
</root>

then both with XML::Twig and XML::Rules you will only have to store 
one record at a time in memory.

The memory footprint of XML::Rules should be (slightly) smaller, 
because it (unless instructed otherwise) stored just plain old Perl 
datastructures, with no sibling and parent references, but that will 
most probably not make a difference.

What might make a difference is if the <record>s are huge and contain 
some repeated tags, then with XML::Rules you might be able to insert 
the data from the subtags into the database as you parse them and 
store only the IDs in memory. But that's probably unnecessary.

Jenda
===== [email protected] === http://Jenda.Krynicky.cz =====
When it comes to wine, women and song, wizards are allowed 
to get drunk and croon as much as they like.
	-- Terry Pratchett in Sourcery

_______________________________________________
Perl-XML mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs