External entity as "#include"

Hi,

Regarding my previous post about an XML parser that allows you to load
non-well-formed documents... perhaps I can clarify the problem and
somebody can help us find the right answer!

The problem in a nutshell is this:

We want to use a single XML "master" document that pulls in fragments
from many sub-documents. The sub-documents are reusable sections used by
different master documents.

Until now our sub-documents have not been well-formed, and we have
included them in the master document as external parsed entities. The
only thing stopping them being well-formed was the absence of a single
root node.

So for example our DTD is something like this (showing only a subset of it):

<!ELEMENT block ( filebytearray | bytearray | intarray | shortarray |
stringarray )* >
<!ATTLIST block id ID #IMPLIED >
<!ATTLIST block type NMTOKEN #REQUIRED >

<!ELEMENT item ( #PCDATA ) >
<!ATTLIST item id ID #IMPLIED >

<!ELEMENT stringarray (item)+ >
<!ATTLIST stringarray id ID #IMPLIED >
<!ATTLIST stringarray nodedupe (true | false) #IMPLIED >

<!ELEMENT wjbinary ( constantdef*, block+ ) >


Our master document might look like this:

<?xml version='1.0' encoding='UTF-8' ?>

<!DOCTYPE wjbinary SYSTEM "wjbinary.dtd"
[
        <!ENTITY strings SYSTEM "strings.xml">
]>

<wjbinary>
     <constantdef id="TESTING">666</constantdef>

     <block type="test">
        &strings;
     </block>
</wjbinary>

...and strings.xml, the fragment in question:

<?xml version='1.0' encoding='UTF-8' ?>

<stringarray id="UISTRINGS_EN">
     <item>HELLO</item>
</stringarray>

<stringarray id="UISTRINGS_FR">
     <item>BONJOUR</item>
</stringarray>


Now this mechanism works, at least using JAXP and Crimson even though
the fragment is -not- well-formed as it doesn't have a single root node,
and we are only explicitly parsing the root document.

We want to be able to load these fragments explicitly (not as entities)
using an XML parser but because they are not well-formed we generally
cannot. We need to work out the root of each included file's DOM.
Perhaps it's possible to do this by allowing the parser to expand
entities, but generally I don't think so, as you lose all info about
where the entity was (and hence what nodes it expanded to).

So we are trying to find a solution for external XML fragments that
results in documents that are well-formed but do not violate the DTD so
that we can retain some validation.

The only solution we can think of at the moment is introducing a new tag
into the DTD like <fragment> to act as a root node for these files, but
to retain the original structure we had that means we have to first load
the document and then remove the <fragment> nodes by moving their
children up to the fragment's parent node and then throw away the fragment:


<!-- A well-formed version of our external entity -->
<?xml version='1.0' encoding='UTF-8' ?>

<fragment>
   <stringarray id="UISTRINGS_EN">
     <item>HELLO</item>
   </stringarray>

   <stringarray id="UISTRINGS_FR">
     <item>BONJOUR</item>
   </stringarray>
</fragment>

Also this will greatly complicate the DTD, and may affect the
reliability of validation. <stringarray> should be able to contain a
<fragment> containing just <item> tags too, but that could result in a
stringarray containing a stringarray as the DTD decl for fragment would
have to permit stringarray as a child.

Needless to say this is an ugly solution. Surely there is a better way?

Eventually we'd like to move to using our own XInclude processor to
replace the use of parsed external entities but we will still be left
with the problem of ensuring these sub-documents are well-formed without
introducing "noise" into the DOM and breaking validation.

XInclude with xpointer -could- be used to pull in just the nodes
underneath <fragment> within the external docs but that is also ugly as
it requires the writer to know the correct node expression.

Perhaps a solution using XML Schema would help, in the sense of improved
validation possibilities?

An even crazier thought... could XML Namespace help? I am not
well-versed in it but how about something like:

<ignoreme:fragment>
   <stringarray id="UISTRINGS_EN">
     <item>HELLO</item>
   </stringarray>

   <stringarray id="UISTRINGS_FR">
     <item>BONJOUR</item>
   </stringarray>
</ignoreme:fragment>

How would these inner nodes (stringarray) be treated when included into
the main document if the main document is not aware of the "ignoreme"
namespace? How would the DTD validate this - would it ignore the
enclosing <ignoreme:fragment> and enforce that the <stringarray> is
within a <block> ?

Thoughts much appreciated.

Sorry for getting into the dark depths. My head is close to exploding,
to be honest.

Kind regards,
--
Marc Palmer
[email protected]

                         w a n g j a m m e r s

                  java and web software design
               experts with an ethical outlook

                    http://www.wangjammers.org

-----------------------------------------------------
xml-interest: A list for discussing XML technologies in the Java Platform.
To post, mailto:[email protected]
Archives at: http://archives.java.sun.com/xml-interest.html
To unsubscribe, mailto:[email protected] the following message;
signoff xml-interest.