Re: stress testing Xerces

Steve Mathias <[email protected]> writes:

> I've been meaning to compose an e-mail about this for a few days now,
> but just haven't gotten around to it.  You might not like to hear this,
> but I think there is a *big* problem.

<sigh>
Okey dokey
</sigh>

> When the XML validation is turned on, the script gradually eats memory
> until it crashes.  If validation is off, the script runs fine.  

Well, that is good news (depending on your POV) - this is possibly a
Xerces-C memory leak then, and not something stupid that I've
done. I've writtent the list, and I'll work on a C++ test to see if I
can reproduce it outside XML-Xerces.

> I have tried everything I can think of to get the memory to be
> released, but with no success.
> 
> I am *definitely* creating a new parser every time.  

Yeah, there isn't anything that you can do about this - if it's in the
validation bit, that's deep in the internals of Xerces-C, and it's
nothing that XML-Xerces could possibly affect.

> Here's the sub that does the validation:
> 
> sub validateXML {
>   my $xml = shift ;
> 
>   # Just to make sure there is only one, $Parser is global but it's not used anywhere else:
>   $Parser = XML::Xerces::XMLReaderFactory::createXMLReader() ;
>   $Parser->setFeature("http://xml.org/sax/features/namespaces", 1) ;
>   $Parser->setFeature("http://apache.org/xml/features/validation/schema", 1) ;
>   $Parser->setFeature("http://apache.org/xml/features/validation/schema-full-checking", 1) ;
>   $Parser->setFeature("http://apache.org/xml/features/validation-error-as-fatal", 1) ;
>   $Parser->setFeature("http://xml.org/sax/features/validation", 1) ;
>   $Parser->$Parser->setFeature("http://apache.org/xml/features/validation/dynamic", 0) ;

See the new example for samples/SAX2Count.pl on how to use the unicode
constants defined in Xerces-C, that will keep you from having to
hard-code these strings in your app. All the unicode constants are
enumerated in docs/XMLUni.txt.

>   my $errorHandler = new XML::Xerces::PerlErrorHandler() ;
>   $Parser->setErrorHandler($errorHandler) ;
>   my $contentHandler = new XML::Xerces::PerlContentHandler() ;
>   $Parser->setContentHandler($contentHandler) ;
>
>   eval {
>     $Parser->parse( XML::Xerces::MemBufInputSource->new($xml) ) ;
>   } ;
>   undef $Parser ; # reclaim resources??

It should. In the meantime, I would run without validation. 

Do you really need to validate internally? You could wrap the script
to run an external validator like nsglms if you really need it. I hope
to have this fixed soon.

More as it happens,
jas.