Re: Java Heap Space problems with XSD 1.1 validation, asserts and large files

--0000000000009f4fbd05be7df9cb
Content-Type: text/plain; charset="UTF-8"

On Thu, Mar 25, 2021 at 5:13 PM Zimmel, Daniel <[email protected]>
wrote:

> My XML file is deeply nested and has 440.000 lines when indented.
>

I hope, you mean that, your XML file has 440000 lines.


> Anyhow, when I change the XSD version to 1.1 and insert a sample assertion
> (xsd:assert test="false()") in the content model for my root element, my
> CPU and memory are filling up quite fast, even giving me a Heap Space Error.
>

That's an expected behaviour with Xerces. The Xerces XSD 1.1
implementation, constructs an XML in-memory DOM/XDM tree for (each)
<xsd:assert>, which is rooted at an XML instance element that is validated
by a xsd:complexType that has an <xsd:assert>. This is to say that,
<xsd:assert> implementation is memory hungry for large XML
instance documents that are validated by <xsd:assert> for XML elements
on/near root of the XML instance tree, and also particularly when the
<xsd:assert> XML instance tree is deeply nested.

Some of the measures that I could advise, for issues described by you are
following,
1) If possible, use IDC constraints or CTA, instead of <xsd:assert>. Or,
use any other non <xsd:assert> XSD constructs for validation.
2) Do part of XML instance validation, within your client code that is
invoking Xerces XSD 1.1 validation.
3) Try using the JVM options -Xms and -Xmx, to tune the heap memory to best
extent. If possible (if it's a production and profit making project), use
more RAM on the workstation where XSD 1.1 validation is taking place.

Should I file a JIRA bug issue?
>

Its up to you. From my point of view, this issue won't likely result in
Xerces XSD 1.1 implementation code improvements.



-- 
Regards,
Mukul Gandhi

--0000000000009f4fbd05be7df9cb
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Thu, Mar 25, 2021 at 5:13 PM Zimmel, D=
aniel &lt;<a href=3D"mailto:[email protected]">[email protected]</a=
>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote class=3D"gmail=
_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204=
,204);padding-left:1ex">My XML file is deeply nested and has 440.000 lines =
when indented.<br></blockquote><div><br></div><div>I hope, you mean that, y=
our XML file has 440000 lines.</div><div>=C2=A0</div><blockquote class=3D"g=
mail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204=
,204,204);padding-left:1ex">
Anyhow, when I change the XSD version to 1.1 and insert a sample assertion =
(xsd:assert test=3D&quot;false()&quot;) in the content model for my root el=
ement, my CPU and memory are filling up quite fast, even giving me a Heap S=
pace Error.<br></blockquote><div><br></div><div>That&#39;s an expected beha=
viour with Xerces. The Xerces XSD 1.1 implementation, constructs an XML in-=
memory DOM/XDM tree for (each) &lt;xsd:assert&gt;, which is rooted at an XM=
L instance element that is validated by a xsd:complexType that has an &lt;x=
sd:assert&gt;. This is to say that, &lt;xsd:assert&gt; implementation is me=
mory hungry for large XML instance=C2=A0documents that are validated by &lt=
;xsd:assert&gt; for XML elements on/near root of the XML instance tree, and=
 also particularly when the &lt;xsd:assert&gt; XML instance tree is deeply =
nested.</div><div><br></div><div>Some of the measures that I could advise, =
for issues described by you are following,</div><div>1) If possible, use ID=
C constraints or CTA, instead of &lt;xsd:assert&gt;. Or, use any other non =
&lt;xsd:assert&gt; XSD constructs for validation.</div><div>2) Do part of X=
ML instance validation, within your client code that is invoking Xerces XSD=
 1.1 validation.</div><div>3) Try using the JVM options -Xms and -Xmx, to t=
une the heap memory to best extent. If possible (if it&#39;s a production a=
nd profit=C2=A0making project), use more RAM on the workstation where XSD 1=
.1 validation is taking place.</div><div><br></div><blockquote class=3D"gma=
il_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,2=
04,204);padding-left:1ex">
Should I file a JIRA bug issue?<br></blockquote><div><br></div><div>Its up =
to you. From my point of view, this issue won&#39;t likely result in Xerces=
 XSD 1.1 implementation code improvements.</div><div>=C2=A0</div></div><br =
clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr" class=3D"gmail_signatu=
re" data-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Regards,<br>Mu=
kul Gandhi<br></div></div></div></div>

--0000000000009f4fbd05be7df9cb--