Re: [xmlpull-user] Can you clone a pull parser? Particularly the xpp3 parser?
Aleksander Slominski <[email protected]> Wed, 04 Feb 2004 01:36:49 -0500
| Newsgroups | gmane.text.xml.xmlpull.devel |
|---|---|
| Message-ID | <[email protected]> |
Nathaniel Hughes wrote:
>Great, thanks. I admittedly (and perhaps obviously) don't have much
>experience implementing clone methods,
>
it is not fun but doeable though ugly - read comments in code and
interview with Ken Arnold to fully appreciate ugliness of cloning in
Java ...
>but I'm reading up right now. I
>look forward to hearing from you.
>
>
here is a full implementation - note that it is *tied* to XPP3
implementation details (and MAY break in future).
as input you MUST use Reader (not InputStream!) that implements
Cloneable (so you need to cook your own CloneableFileReader!!!!) but
toherwise it works like charm :-)
when you run example CloneParser code you should see this:
parser implementation class is class CloneableMXParser
START_TAG foo
START_TAG baz
>>> SPLIT POINT
TEXT 'bar'
END_TAG baz
START_TAG moo
TEXT 'maz'
END_TAG moo
END_TAG foo
END_DOCUMENT
>>> CLONED PARSER
TEXT 'bar'
END_TAG baz
START_TAG moo
TEXT 'maz'
END_TAG moo
END_TAG foo
END_DOCUMENT
hope it is now *exactly* what you need!
best,
alek
/* -*- c-basic-offset: 4; indent-tabs-mode: nil; -*-
//------100-columns-wide------>|*/
// for license please see accompanying LICENSE.txt file (available also
at http://www.xmlpull.org/)
import CloneableMXParser;
import java.io.CharArrayReader;
import java.io.IOException;
import java.io.Reader;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import org.xmlpull.v1.XmlPullParser;
import org.xmlpull.v1.XmlPullParserException;
/**
* An example to demonstrate how cloneable parser can be used.
*
* @author <a href="http://www.extreme.indiana.edu/~aslom/">Aleksander
Slominski</a>
*/
class CloneParser {
private static class CloenableCharArrayReader extends
CharArrayReader implements Cloneable {
public CloenableCharArrayReader(char buf[]) {
super(buf);
}
public Object clone() throws CloneNotSupportedException
{
CloenableCharArrayReader cloned = (CloenableCharArrayReader)
super.clone();
cloned.buf = buf.clone();
//UGLY UGLY UGLY ...
cloned.lock = cloned;
return cloned;
}
}
public static void main(String[] args) throws Exception
{
// XmlPullParserFactory factory =
XmlPullParserFactory.newInstance(
//
System.getProperty(XmlPullParserFactory.PROPERTY_NAME), null);
// //factory.setNamespaceAware(true);
//
factory.setFeature(XmlPullParser.FEATURE_PROCESS_NAMESPACES, true);
//
// XmlPullParser xpp = factory.newPullParser();
final String XML = "<foo><baz>bar</baz><moo>maz</moo></foo>";
CloneableMXParser mpp = new CloneableMXParser();
System.out.println("parser implementation class is
"+mpp.getClass());
Reader cpp = new CloenableCharArrayReader(XML.toCharArray());
mpp.setInput(cpp);
mpp.next(); print(mpp);
mpp.next(); print(mpp);
System.out.println(">>> SPLIT POINT");
CloneableMXParser m2 = (CloneableMXParser) mpp.clone();
mpp.next(); print(mpp);
mpp.next(); print(mpp);
mpp.next(); print(mpp);
while(mpp.getEventType() != XmlPullParser.END_DOCUMENT) {
mpp.next(); print(mpp);
}
System.out.println(">>> CLONED PARSER");
while(m2.getEventType() != XmlPullParser.END_DOCUMENT) {
m2.next(); print(m2);
}
}
private static void print(XmlPullParser pp) throws
XmlPullParserException {
int type = pp.getEventType();
System.out.print(XmlPullParser.TYPES[type]+" ");
if(type == XmlPullParser.START_TAG || type ==
XmlPullParser.END_TAG) {
System.out.print(pp.getName());
} else if(type == XmlPullParser.TEXT) {
System.out.print("'"+pp.getText()+"'");
}
System.out.println();
}
}
------------------------------------------------------------------------------------------------------------
/* -*- c-basic-offset: 4; indent-tabs-mode: nil; -*-
//------100-columns-wide------>|*/
// for license please see accompanying LICENSE.txt file (available also
at http://www.xmlpull.org/)
import java.io.Reader;
import org.xmlpull.mxp1.MXParser;
import org.xmlpull.v1.XmlPullParserException;
import java.lang.reflect.InvocationTargetException;
/**
* When using this class make sure that setInput(Reader) is used
* and that reader passed supports Cloeneable!!!!
*
* @author <a href="http://www.extreme.indiana.edu/~aslom/">Aleksander
Slominski</a>
*/
public class CloneableMXParser extends MXParser implements Cloneable {
public CloneableMXParser() {
}
public void setInput(Reader in) throws XmlPullParserException
{
if(!(in instanceof Cloneable)) {
throw new XmlPullParserException("reader used in parser must
implement Cloneable!");
}
super.setInput(in);
}
public void setInput(java.io.InputStream inputStream, String
inputEncoding)
throws XmlPullParserException
{
throw new XmlPullParserException("for this class only
setInput(reader) can be used");
}
public Object clone() throws CloneNotSupportedException
{
CloneableMXParser cloned = (CloneableMXParser) super.clone();
//protected Reader reader;
if(reader != null) {
if(!(reader instanceof Cloneable)) {
throw new CloneNotSupportedException("reader used in
parser must implement Cloneable!");
}
// why Cloneable as no clone() inside? good question and
needs for stupid CloneableReader ...
//cloned.reader = (Reader) ((Cloneable)reader).clone();
//BING BONG doe snot work ...
//use reflection to call clone() -- this is getting ugly!!!!
// more background on this in
http://www.artima.com/intv/issues3.html "The clone Dilemma"
try {
Object o = reader.getClass().getMethod("clone",
null).invoke(reader, null);
cloned.reader = (Reader) o;
} catch (Exception e) {
throw new CloneNotSupportedException("failed to call
cloen() on reader "+reader+e);
}
}
// NOTE: "A clone of a multidimensional array is shallow, which
is to say that
// "it creates only a single new array. Subarrays are shared, ..."
//
http://java.sun.com/docs/books/jls/second_edition/html/arrays.doc.html#64347
// protected char[] elRawName[];
if(elRawName != null) {
cloned.elRawName = cloneCCArr(elRawName);
}
// protected int elRawNameEnd[];
if(elRawNameEnd != null) {
cloned.elRawNameEnd = elRawNameEnd.clone();
}
// protected int elRawNameLine[];
if(elRawNameLine != null) {
cloned.elRawNameLine = elRawNameLine.clone();
}
// protected String elName[];
if(elName != null) {
cloned.elName = elName.clone();
}
// protected String elPrefix[];
if(elPrefix != null) {
cloned.elPrefix = elPrefix.clone();
}
// protected String elUri[];
if(elUri != null) {
cloned.elUri = elUri.clone();
}
// protected int elNamespaceCount[];
if(elNamespaceCount != null) {
cloned.elNamespaceCount = elNamespaceCount.clone();
}
// protected String attributeName[];
if(attributeName != null) {
cloned.attributeName = attributeName.clone();
}
// protected int attributeNameHash[];
if(attributeNameHash != null) {
cloned.attributeNameHash = attributeNameHash.clone();
}
// protected String attributePrefix[];
if(attributePrefix != null) {
cloned.attributePrefix = attributePrefix.clone();
}
// protected String attributeUri[];
if(attributeUri != null) {
cloned.attributeUri = attributeUri.clone();
}
// protected String attributeValue[];
if(attributeValue != null) {
cloned.attributeValue = attributeValue.clone();
}
// protected String namespacePrefix[];
if(namespacePrefix != null) {
cloned.namespacePrefix = namespacePrefix.clone();
}
// protected int namespacePrefixHash[];
if(namespacePrefixHash != null) {
cloned.namespacePrefixHash = namespacePrefixHash.clone();
}
// protected String namespaceUri[];
if(namespaceUri != null) {
cloned.namespaceUri = namespaceUri.clone();
}
// protected String entityName[];
if(entityName != null) {
cloned.entityName = entityName.clone();
}
// protected char[] entityNameBuf[];
if(entityNameBuf != null) {
cloned.entityNameBuf = cloneCCArr(entityNameBuf);
}
// protected int entityNameHash[];
if(entityNameHash != null) {
cloned.entityNameHash = entityNameHash.clone();
}
// protected char[] entityReplacementBuf[];
if(entityReplacementBuf != null) {
cloned.entityReplacementBuf = cloneCCArr(entityReplacementBuf);
}
// protected String entityReplacement[];
if(entityReplacement != null) {
cloned.entityReplacement = entityReplacement.clone();
}
// protected char buf[];
if(buf != null) {
cloned.buf = buf.clone();
}
// protected char pc[];
if(pc != null) {
cloned.pc = pc.clone();
}
// protected char[] charRefOneCharBuf;
if(charRefOneCharBuf != null) {
cloned.charRefOneCharBuf = charRefOneCharBuf.clone();
}
return cloned;
}
private char[][] cloneCCArr(char[][] ccarr) {
char[][] cca = ccarr.clone();
for (int i = 0; i < cca.length; i++)
{
if(cca[i] != null) {
cca[i] = cca[i].clone();
}
}
return cca;
}
}
>On Tue, 3 Feb 2004, Aleksander Slominski wrote:
>
>
>
>>Nathaniel Hughes wrote:
>>
>>
>>
>>>So if I implemented the clone operation in MXParser, you think I could do
>>>this?
>>>
>>>I would have to call super.clone() and also clone the Reader
>>>(with simply reader.clone() I believe) correct?
>>>
>>>Is there anything else in the MXParser class that requires a deep clone?
>>>
>>>
>>>
>>>
>>it maintains tables of namespaces, attributes etc and those must be deep
>>cloned.
>>
>>i started writing a sample CloneableMXParser and should be ready in few
>>minutes :-)
>>
>>alek
>>
>>
>>
>>>On Tue, 3 Feb 2004, Aleksander Slominski wrote:
>>>
>>>
>>>
>>>
>>>
>>>>Nathaniel Hughes wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>That's too bad that I need to record it to re-parse it, I was hoping I
>>>>>wouldn't have to store any of the data in memory in order to 'rewind' and
>>>>>begin parsing again. The issue that I have is that the files I'm parsing
>>>>>are very large in size and I want to be able to go back to some given
>>>>>point in the file (where I cloned my parser) and begin parsing again, but
>>>>>I can't afford to store everything in between in memory...
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>Oh well, I'll have to look for another solution.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>if you are parsing files you may take a different approach and do a real
>>>>clone of parser object but that will require you looking into actual
>>>>parser implementation so input cloning is handled correctly.
>>>>
>>>>in case of XPP3 take look into org.xmlpull.mxp1.MXParser and add to it
>>>>clone() operation that will also clone actual input stream! this should
>>>>work as long as InputStream or Reader supports seek() (as it is in case
>>>>of files) so you can rewind stream and restart parsing using cloned parser,
>>>>
>>>>HTH,
>>>>
>>>>alek
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>On Mon, 2 Feb 2004, Aleksander Slominski wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>natehughes1003 wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Hey all, I'm new to pull parsers and I had a question. Is it
>>>>>>>possible to clone a pull parser? I was hoping to create a pull
>>>>>>>parser, parse part way through a document, and stop. Then I wanted
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>><> >to clone that parser, and finish parsing the document with the
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>>original parser, and then with the cloned parser.
>>>>>>>
>>>>>>>Is this at all possible?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>yes but not easily.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>If so, can anyone demonstrate for me
>>>>>>>quickly?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>you need to clone not only parser but even more important original input
>>>>>>needs to be saved for reparsing.
>>>>>>
>>>>>>instead i would create a special parser that allows to record events for
>>>>>>replay (for informaiton about XmlPullParserWrapper check
>>>>>>http://www.xmlpull.org/v1/doc/addons.html#wrapper).
>>>>>>
>>>>>>below is full example - to make truly useful you need support for rest
>>>>>>of XmlPull methoids (attributes mostly!) and overwrite nextToken().
>>>>>>
>>>>>>if you run this sample you should get:
>>>>>>
>>>>>>parser implementation class is class org.xmlpull.v1.xni2xmlpull1.X2Parser
>>>>>>START_TAG foo
>>>>>>START_TAG baz
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>START RECORDING
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>TEXT 'bar'
>>>>>>END_TAG baz
>>>>>>START_TAG moo
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>REWIND
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>START_TAG baz
>>>>>>TEXT 'bar'
>>>>>>END_TAG baz
>>>>>>START_TAG moo
>>>>>>TEXT 'maz'
>>>>>>END_TAG moo
>>>>>>END_TAG foo
>>>>>>END_DOCUMENT
>>>>>>
>>>>>>so you could do in your code
>>>>>>mpp.startRecording();
>>>>>>firstFunction(mpp);
>>>>>>mpp.rewind();
>>>>>>secondFunction(mpp);
>>>>>>
>>>>>>if you thin it is very useful i may add itto set of other addons in
>>>>>>http://www.xmlpull.org/v1/doc/addons.html
>>>>>>
>>>>>>good luck!
>>>>>>
>>>>>>alek
>>>>>>
>>>>>>/* -*- c-basic-offset: 4; indent-tabs-mode: nil; -*-
>>>>>>//------100-columns-wide------>|*/
>>>>>>// for license please see accompanying LICENSE.txt file (available also
>>>>>>at http://www.xmlpull.org/)
>>>>>>
>>>>>>import java.io.IOException;
>>>>>>import java.io.StringReader;
>>>>>>import java.util.ArrayList;
>>>>>>import java.util.List;
>>>>>>import org.xmlpull.v1.XmlPullParser;
>>>>>>import org.xmlpull.v1.XmlPullParserException;
>>>>>>import org.xmlpull.v1.XmlPullParserFactory;
>>>>>>
>>>>>>class RecordableXmlPullParser extends
>>>>>>org.xmlpull.v1.wrapper.classic.XmlPullParserDelegate {
>>>>>> private List events = new ArrayList();
>>>>>> private int pos = 0;
>>>>>> private boolean recording = false;
>>>>>> private Event currentEvent;
>>>>>>
>>>>>> public static void main(String[] args) throws Exception
>>>>>> {
>>>>>> XmlPullParserFactory factory = XmlPullParserFactory.newInstance(
>>>>>> System.getProperty(XmlPullParserFactory.PROPERTY_NAME), null);
>>>>>> //factory.setNamespaceAware(true);
>>>>>> factory.setFeature(XmlPullParser.FEATURE_PROCESS_NAMESPACES, true);
>>>>>>
>>>>>> XmlPullParser xpp = factory.newPullParser();
>>>>>> System.out.println("parser implementation class is
>>>>>>"+xpp.getClass());
>>>>>>
>>>>>> final String XML = "<foo><baz>bar</baz><moo>maz</moo></foo>";
>>>>>> RecordableXmlPullParser mpp = new RecordableXmlPullParser(xpp);
>>>>>> mpp.setInput(new StringReader(XML));
>>>>>> mpp.next(); print(mpp);
>>>>>> mpp.next(); print(mpp);
>>>>>> System.out.println(">>> START RECORDING");
>>>>>> mpp.startRecording();
>>>>>> mpp.next(); print(mpp);
>>>>>> mpp.next(); print(mpp);
>>>>>> mpp.next(); print(mpp);
>>>>>> System.out.println(">>> REWIND");
>>>>>> mpp.rewind(); print(mpp);
>>>>>> while(mpp.getEventType() != XmlPullParser.END_DOCUMENT) {
>>>>>> mpp.next(); print(mpp);
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> private static void print(XmlPullParser pp) throws
>>>>>>XmlPullParserException {
>>>>>> int type = pp.getEventType();
>>>>>> System.out.print(XmlPullParser.TYPES[type]+" ");
>>>>>> if(type == XmlPullParser.START_TAG || type ==
>>>>>>XmlPullParser.END_TAG) {
>>>>>> System.out.print(pp.getName());
>>>>>> } else if(type == XmlPullParser.TEXT) {
>>>>>> System.out.print("'"+pp.getText()+"'");
>>>>>> }
>>>>>> System.out.println();
>>>>>> }
>>>>>>
>>>>>> public RecordableXmlPullParser(XmlPullParser pp) {
>>>>>> super(pp);
>>>>>> }
>>>>>>
>>>>>> public void startRecording() throws IOException,
>>>>>>XmlPullParserException {
>>>>>> recording = true;
>>>>>> events.clear();
>>>>>> events.add(new Event());
>>>>>> pos = 1;
>>>>>> }
>>>>>>
>>>>>> public void stopRecording() {
>>>>>> recording = false;
>>>>>> pos = 0;
>>>>>> events.clear();
>>>>>> }
>>>>>>
>>>>>> public void rewind() {
>>>>>> pos = 0;
>>>>>> currentEvent = events.size() > 0 ? (Event) events.get(pos) : null;
>>>>>> }
>>>>>>
>>>>>> public int next() throws IOException, XmlPullParserException {
>>>>>> if(pos >= 0 && pos < events.size() - 1) {
>>>>>> ++pos;
>>>>>> currentEvent = (Event) events.get(pos);
>>>>>> return currentEvent.type;
>>>>>> } else {
>>>>>> currentEvent = null;
>>>>>> int type = super.next();
>>>>>> if(recording) {
>>>>>> events.add(new Event());
>>>>>> ++pos;
>>>>>> }
>>>>>> return type;
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> // now every event state access method in XmlPullParser must follow
>>>>>>this pattern (get* ...)
>>>>>>
>>>>>> public int getEventType() throws XmlPullParserException {
>>>>>> return currentEvent != null ? currentEvent.type :
>>>>>>super.getEventType();
>>>>>> }
>>>>>>
>>>>>> public String getNamespace() {
>>>>>> return currentEvent != null ? currentEvent.namespace :
>>>>>>super.getNamespace();
>>>>>> }
>>>>>>
>>>>>> public String getName() {
>>>>>> return currentEvent != null ? currentEvent.name : super.getName();
>>>>>> }
>>>>>>
>>>>>> public String getText() {
>>>>>> return currentEvent != null ? currentEvent.content :
>>>>>>super.getText();
>>>>>> }
>>>>>>
>>>>>> //more work goes here ...
>>>>>>
>>>>>> private class Event {
>>>>>> int type;
>>>>>> String namespace;
>>>>>> String name;
>>>>>> String content;
>>>>>> List attributes = new ArrayList();
>>>>>>
>>>>>> Event() throws IOException, XmlPullParserException {
>>>>>> type = getEventType();
>>>>>> if(type == XmlPullParser.START_TAG || type ==
>>>>>>XmlPullParser.END_TAG) {
>>>>>> namespace = getNamespace();
>>>>>> name = getName();
>>>>>> // TODO: get lsit of all attributes ...
>>>>>> } else if(type == XmlPullParser.TEXT) {
>>>>>> content = getText();
>>>>>> } else if(type == XmlPullParser.START_DOCUMENT || type ==
>>>>>>XmlPullParser.END_DOCUMENT) {
>>>>>> } else {
>>>>>> throw new XmlPullParserException(
>>>>>> "unsupported event type
>>>>>>"+XmlPullParser.TYPES[type]+getPositionDescription());
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>}
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>Yahoo! Groups Links
>>>>>
>>>>>To visit your group on the web, go to:
>>>>>http://groups.yahoo.com/group/xmlpull-user/
>>>>>
>>>>>To unsubscribe from this group, send an email to:
>>>>>[email protected]
>>>>>
>>>>>Your use of Yahoo! Groups is subject to:
>>>>>http://docs.yahoo.com/info/terms/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>Yahoo! Groups Links
>>>
>>>To visit your group on the web, go to:
>>>http://groups.yahoo.com/group/xmlpull-user/
>>>
>>>To unsubscribe from this group, send an email to:
>>>[email protected]
>>>
>>>Your use of Yahoo! Groups is subject to:
>>>http://docs.yahoo.com/info/terms/
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
>
>Yahoo! Groups Links
>
>To visit your group on the web, go to:
> http://groups.yahoo.com/group/xmlpull-user/
>
>To unsubscribe from this group, send an email to:
> [email protected]
>
>Your use of Yahoo! Groups is subject to:
> http://docs.yahoo.com/info/terms/
>
>
>
>
--
The best way to predict the future is to invent it - Alan Kay
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/2U_rlB/TM
---------------------------------------------------------------------~->
Yahoo! Groups Links
To visit your group on the web, go to:
http://groups.yahoo.com/group/xmlpull-dev/
To unsubscribe from this group, send an email to:
[email protected]
Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/