Re: Re: XML outline view

Hi Jesse,

when I was designing XML navigation, I was thinking about the view (XML  
outline) you mentioned here as well. But because of time constraints  
and for already mentioned reasons (web.xml, ejb-jar.xml), I decided to  
use only structural view for all types of xml files for this release.

I disagree that we should get rid of the structural view, IMO we should  
provide both views: for known (XML outline) and unknown (XML raw)  
files. In addition to that I agree that XML outline view should be the  
default one.

About improving the navigation:
I was thinking about same improvements as Tim: show tags that are worth  
showing, simplify the tree.

-J.

On 10.8.2005, at 13:43, Marek Fukala wrote:

> Jesse, I really do share your and Tim's ideas about the navigator  
> outline view. I consider this approach as very usefull though I have  
> some doubts about the mechanism which decides what parts of the  
> documents are important, and so worth of displaying in the outline  
> view, ant what aren't. Doing such thing for know filetypes can be  
> fairly easy (a one (human) can decide what is important or not), but I  
> am not sure whether the generic 'autodecide' algorithm will be  
> effective and will do what users expects.
>
> I call the navigator implemented by me as XML RAW view, which is IMO  
> self-explanatory :-). I agree with you, that there is not much overlap  
> between the outline and raw views. Each has its advantages as well as  
> its disadvantages. Which one is more suitable depends on the  
> particullar usecase. My personal opinion is that the outline view can  
> be used for known XML-based mimetypes by default if there isn't any  
> other "high level" view (as web.xml or ejb-jar.xml UI editor mentioned  
> in section #2 of the UI spec). For the unknown XML types I would  
> rather use the RAW view.
>
> One more thing related to the xml RAW view I want to mention is that  
> its implementation is based on a new API from xml/tageditorsupport  
> module. The API is divided to an API allowing access to the document  
> elements and an SPI which provides a document model (a tree of  
> elements) over a text document. There is an implementation of the  
> model provider for the XMLs which is used not only for the XML RAW  
> navigator, but also for XML folding and some other modules. Since the  
> model is update incrementally there is no need to reparse the entire  
> document after each change (which is the bad think what the SAX based  
> approcach must do AFAIK). I can imagine a very simple implementation  
> of the outline view built upon the document model API. For more  
> details see "[Issue 61412]  Tag Based Editor Support API review".
>
> -marek
>
> Jesse Glick wrote:
>
>> I'd like to start a discussion on how XML navigation should look -  
>> i.e. what appears in the Navigator window when you have an XML file  
>> open in the editor (or otherwise selected). Right now, nothing  
>> appears, but:
>>
>>
>> 1. There seems to be an XML navigation feature planned for inclusion  
>> in 4.2:
>>
>> http://ui.netbeans.org/docs/ui/code_navigator/inspector_docs.html
>>
>> I haven't seen it running yet, but apparently it is being worked on  
>> already.
>>
>>
>> 2. Before I knew about that, I wrote a very simple XML navigation  
>> module. It's in contrib/xmlnavigation in netbeans.org CVS (should be  
>> on Dev AU very soon if anyone's interested). I will attach a typical  
>> screenshot. Still has many bugs etc. Based loosely on conversations  
>> I've had with Tim Boudreau about the old xml/tree-edit module.
>>
>>
>> I'm glad to see there is a plan to work on XML navigation and make it  
>> fast and supported, since what I wrote was just a proof of concept.  
>> Still, there is not much overlap between what contrib/xmlnavigation  
>> does and what the UI spec above proposes.
>>
>> The proposed UI spec in sections 1, 3, and 4 shows full XML structure  
>> in a tree, like the old xml/tree-edit module, which I guess could be  
>> useful in odd circumstances (like a huge OpenOffice document with no  
>> newlines) but doesn't seem very compelling. Why would you spend time  
>> browsing through a big complex tree, overloaded with detail, when you  
>> could much more quickly go through the XML text that provides the  
>> same information in a format everyone is familiar with and knows how  
>> to edit?
>>
>> contrib/xmlnavigation is more similar to section 2 of the proposed UI  
>> spec, except rather than being specialized for particular J2EE  
>> document types, it does a rough job of showing structure for a wide  
>> range of XML types. Most details of XML syntax and structure are  
>> ignored - only a handful of elements are selected for indexing. The  
>> basic idea is that many common XML document types are broken into  
>> logical sections that can be recognized by common patterns such as  
>> certain elements being used for headers, certain attributes like 'id'  
>> being used for names, etc. For example, an XHTML document like
>>
>> <?xml ...?>
>> <html xmlns="...">
>> <head>
>> <!-- stuff... -->
>> </head>
>> <body>
>> <h1>Title</h1>
>> <p>....content....</p>
>> <p>....</p>
>> <h2>Subtitle</h2>
>> <p>....</p>
>> <h1>Next Part</h1>
>> <!-- ..... -->
>> </body>
>> </html>
>>
>> is displayed as
>>
>> Title
>> Subtitle
>> Next Part
>>
>> Obviously it could be improved by indenting "Subtitle" under "Title"  
>> etc., but the point is to show the real minimum of detail that would  
>> be useful for actual navigation. If I am editing a big XHTML  
>> document, I really do not need to see <p> tags in a tree view! I just  
>> want to be able to jump to major sections of the document quickly and  
>> get back to the text editor: Ctrl-7, Down, Down, Enter. The navigator  
>> component shows an outline, just as for Java files it shows methods  
>> and fields, not every instance of the '+' operator.
>>
>> Same for XSD, XSLT, DocBook, etc. - if the bugs were worked out,  
>> xmlnavigation would show only the important header info: one index  
>> entry per XSD type definition, XSLT template definition, DocBook  
>> named section, etc. (*)
>>
>> Given the current UI spec, my advice would be to get rid of the XML  
>> structure view entirely, and focus on providing views more like those  
>> for web.xml and ejb-jar.xml, though preferring lists (possibly with  
>> minimal indentation) to trees in all cases, for fast navigation and  
>> compact display.
>>
>>
>> Tim Boudreau also wrote in response:
>>
>> <tim>
>>
>> What I want, really, for navigating XML, is something that will show  
>> me things that are likely to be important to me and let me navigate  
>> between them.  Most XML documents have a structure which contains  
>> elements that are what the user thinks of as the "contents" of the  
>> file, and miscellaneous sub-elements that are important to a machine  
>> reading the XML, but are noise to the person who wants to find  
>> something or edit something.
>>
>> A tool that does a best-guess at what those important elements are  
>> (or ideally, understands some types and actually knows) would be  
>> useful anytime I need to edit XML;  a tree showing all the gory  
>> details of an XML file is only useful when I'm interested in the  
>> *structure* of the XML document.  Usually I'm not interested at all  
>> in the structure, what I'm interested in is the content - and  
>> mentally the "content" is much less granular - a tool that shows me  
>> every <p/> tag will make me less, not more productive.  The computer  
>> is interested in the structure;  the user is interested in the  
>> content.  A tool that shows the content well will be valuable;  a  
>> tool that shows the thing that it's easy for the computer to show  
>> will not be.  The point of Navigator is *navigation* - making it  
>> easier to use the editor.  Structural views are secondary, navigation  
>> is primary.
>>
>> It would be fine if the XML navigator view has a structural view in  
>> the views combo, but it should probably not be the default, and I  
>> would prioritize it considerably below the kind of navigation Jesse  
>> is talking about - not even necessary for the first rev.
>>
>> As for algorithms to figure out what is likely to be "important",  
>> there are plenty of possibilities:
>>
>>  - For known types of documents, actually keep a list of tags that  
>> are worth showing - for example, HTML: h1/2/3, table, form, img;  for  
>> DocBook:  book, chapter, section (and others...)
>>
>>  - For unknown file types, you're going to have to read the whole  
>> thing anyway, so do something like
>>
>>     - Ignore any top level tags that only occur once and contain  
>> everything else in the document
>>
>>     - Find those tags that occur more than once, esp. if each one has  
>> a differentiating string attribute
>>
>> </tim>
>>
>> and with respect to parsing (contrib/xmlnavigation runs a SAX parser  
>> when an XML document is displayed, while the proposed new module is  
>> said to use the editor's syntax elements, which should be able to  
>> reparse incrementally and thus perhaps be faster) he wrote:
>>
>> <tim>
>>
>> The choice of SAX is a good one for two reasons:
>>
>>  - You want to be able to show file structure even when the document  
>> is invalid
>>
>>  - It has the potential to scale - you can stop once you have as many  
>> elements as you can display
>>
>> Now if you wanted to do something *really* useful for a malformed  
>> document, then, in the case of a malformed document
>>
>>  - Run the SAX parser up to the point the problem is found, then stop
>>
>>  - Run a SAX-like parser *backwards* from the bottom of the document  
>> (unless it is huge) and display elements up to the discontinuity, and  
>> a nice red "malformed element" item at the point where it's broken
>>
>> </tim>
>>
>> I will note that using SAX
>>
>> - You can show the outline for a well-formed but invalid file, *but*  
>> undeclared entity refs (like "&nbsp;") abort the parse *if* there is  
>> no DOCTYPE in the document at all (not sure why Xerces behaves this  
>> way but I can't seem to override it); this is annoying if you have a  
>> big DocBook document split into several files using entity includes.  
>> Yet another reason to stick to pure XML Infoset and avoid DTDs and  
>> DOCTYPE.
>>
>> - The current impl does not attempt to stop at the display limit of  
>> the navigation JList, but it probably could.
>>
>> - There is no library that I know of for running a SAXiform parser  
>> backwards as Tim suggests, so this could be tricky.
>>
>>
>> Anyway, just wanted to bring this up in case other people have ideas  
>> about it. Of course there is limited time left for the implementation  
>> in 4.2, so nothing too fancy is feasible, but it would be good to  
>> agree on goals.
>>
>> Cheers,
>> -J.
>>
>>
>> (*) Details for those who are interested:
>>
>> The current contrib/xmlnavigation impl looks for <h1> thru <h6> (for  
>> XHTML) as well as anything with a <title> beneath it (should handle  
>> DocBook etc.). Could be much more precise.
>>
>> It also checks for attributes 'name' and 'id' which are the most  
>> common ways in which this idiom is expressed. This has some bugs,  
>> e.g. XSLT
>>
>> <xsl:call-template name="something"><xsl:param .../><!--...--></>
>>
>> gets indexed (as 'something'), which is wrong since it is using  
>> rather than defining a template.
>>
>> Fortunately it is pretty easy to write unit tests which take various  
>> example documents (preferably real cases you found on the web or  
>> something) and assert that your expectations for a natural-looking  
>> index are matched exactly.
>>
>>
>> ---------------------------------------------------------------------- 
>> --
>>
>