Re: Wikipedia dump file processing shoot out

Tyler Riddle schrieb am 10.12.2009 um 11:46:01 (-0800):

> The dump files are one single document but it gets even worse than the
> English Wikipedia at 22 gigs in one single file - there is a dump file
> for the German Wikipedia that is 1 TB uncompressed as it has every
> single revision for every single article in it and it's still a single
> file.
> 
> The intelligence of the approach of putting that much XML into one
> file is kind of beyond the point as the various language Wikipedias
> are trapped in them. This is a real world situation unfortunately.

Here's another guy trapped in the same kind of monster XML real world
situation:

[MarkLogic Dev General] Avoiding long running transactions
http://www.mail-archive.com/[email protected]/msg03440.html

Monster XML causes pain to man and machine, even to sophisticated ones.
Monster XML should be avoided. Its disadvantages are obvious, its
benefits are nowhere to be seen.

-- 
Michael Ludwig
_______________________________________________
Perl-XML mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs