Re: Re: XMLPULL in C++

Nick Woolley wrote:

>    > i want eventually to convent MXP1 code from Java to C++ (when
> implementing
>    > i have made code very easy to convert ..) bu ti need first to figure
>    > out the best ways to represent String (UTF-8, UTF-16 other encodings)
>    > that internally keeps wchar_t* or char* and how to abstract IO
> processing
>    > (so streaming works OK).
>
> Have you any references you could give me to help me understand exactly
> what you mean here?  Can you not define the XMLParser as a template
> class to solve the latter. as is done with C++'s  basic_string<> class?

fundamental choice is between char_t*/string (UTF-8 internally)
or wchar_t*/wstring (UTF-16 internally).


>      And what IO streaming do you mean which can't be handled by C++
> streams?

you need to add support decoding different charsets, basically to have
something similar to:
    new InputStreamReader(byte_stream_fd, "UTF-8");

> Incidentally, I was wondering about an isalpha() function or similar
> that was locale specific - my Stroustrup book doesn't say whether these
> functions are or not.

parser should not be locale specific as XML is always UNICODE (that
may be physically represented as UTF-8 or UTF-16 or even UCS4 ...).

>    > what were your decisions about it?
>
> Well, depends what you mean.  I saw there was a need which pull parsing
> seemed to satisfy quite nicely, but there weren't any handy parsers I
> could take "off the shelf".  I tried SAX2 but it seemed like I had to
> bend over backwards to get it to do what I wanted.

i know the feeling :-)

> So I thought, maybe I can write a simple implementation based on this
> which could do what I need to do, and then later if the C++ version
> turns up I can switch to that without much effort.
>
>    > did you have any other problems?
>
> Getting a basic prototype wasn't too hard.  Figuring what to do to make
> the semantics comply to the XMLPULL API isn't always easy, I have lots
> of questions here.
>
> I've written about three versions now, I think they are getting better
> as implementations of the API but I want to stop working on the parser
> and start using it to solve my problem.

sounds excellent!

>    > i think those are the only issues when i was implementing XPP1 (and
>    > learnt hard way how important they are). i think that eventually we
> should
>    > put together C++ binding for XmlPull API ...
>
> I'd be happy to contribute if I can, and I if have time.

ok.

>    > hope that it explains it.
>
> Yes thanks.  I hope you don't mind if I ask some more questions?

absolutely - ask any questions - it helps to understand if API is OK.

> --
>
> Firstly, is there a test suite I could (at some point) adapt to check if
> the compiler parses XML correctly, as per the API spec.? I'm thinking of
> some XML files and lists of the output events and their atttributes
> which should be generated from them.

we have an initial version of XML tests now in CVS, checkout src/xml/tests
(i am attaching example below) - those tests eventually should cover whole API.

> Secondly, does next() skip whitespace, like nextTag()?

next() will always report all events - in this case returning TEXT
(or IGNORABLE_WHITESPACE if parser is validating and element
has unmixed content model)

> Should nextTag() create end tag events for empty tags?

yes! and it must be done by next() as well.

it is required to maintain equivalency "<tag></tag>" == "<tag/>"

> --
>
> Why are some event ids defined as type 'byte', and others as 'int'?

it was a bug - all are now int (it is in CVS and will be in next release of API).

> --
>
> Does the API specify whether text has to be returned as one contiguous
> chunk, or if a sequence of tokens is acceptable?

next() must always returned one TEXT event
nextToken() is free to return as many TEXT/IGNORABLE_WHITESPACE/etc. as it wants

> i.e. given a doc:
>
> <?xml version="1.0"?>
>      stuff
> <roottag>
>      <anothertag>  do be do be do &amp; do  </anothertag>
> </roottag>
>
> When using nextToken(), this is what you might expect to get back:
>
> TEXT = "do be do be do "
> ENTITY_REF = "&amp;"
> TEXT = " do  "

yes

> But might you also get:
>
> TEXT = "do"
> TEXT = " "
> TEXT = "be"
> TEXT = " "
> ....

yes - ultimately depending on implementation of XmlPull API.

currently MXP1 will always return one TEXT from nextToken()
but this will change (performance!) and Xerces2 based implementation
of XmlPull (see CVS in http://sourceforge.net/projects/xni2xmlpull/)
will return multiple TEXT event from nextToken() - this is similar ot SAX2.

> Or even:
>
> TEXT = " do be do be do & do  "

that is what is _always_ returned by next() and nextToken().

getting C++ implementation of XmlPull API is very important (and C#
maybe as well) and i think that i amy have my own rolled up in next month
(basically re-engineered/joined XPP1 and MXP1 to XmlPull API and C++).

thanks,

alek
ps. example XML confromance test:

<tests xmlns="http://xmlpull.org/v1/tests/2002-08.xsd">
 <test-parser name="simple test">
  <create-parser/> <input-inline><![CDATA[ <foo>bar</foo> ]]></input-inline>
  <set-feature>http://xmlpull.org/v1/doc/features.html#process-namespaces</set-feature>
  <expect type="START_DOCUMENT"/>
  <next/><expect type="START_TAG" namespace="" name="foo" empty="false"/>
  <next-text text="bar"/>
  <next/><expect type="END_DOCUMENT"/>
 </test-parser>
</tests>