Re: XML outline view

Marek Fukala wrote:
> Jesse, I really do share your and Tim's ideas about the navigator 
> outline view. I consider this approach as very usefull though I have 
> some doubts about the mechanism which decides what parts of the 
> documents are important, and so worth of displaying in the outline view, 
> ant what aren't. Doing such thing for know filetypes can be fairly easy 
> (a one (human) can decide what is important or not), but I am not sure 
> whether the generic 'autodecide' algorithm will be effective and will do 
> what users expects.

Start with what Jesse did - special case tags with attribute "id" and 
"name" and such.  Nine times out of ten, it will do the right thing; 
have some kind of fallback that prefers longer tag names, or tags that 
contain more elements, or something such.

> I call the navigator implemented by me as XML RAW view, which is IMO 
> self-explanatory :-). I agree with you, that there is not much overlap 
> between the outline and raw views. Each has its advantages as well as 
> its disadvantages. Which one is more suitable depends on the particullar 
> usecase. My personal opinion is that the outline view can be used for 
> known XML-based mimetypes by default if there isn't any other "high 
> level" view (as web.xml or ejb-jar.xml UI editor mentioned in section #2 
> of the UI spec). For the unknown XML types I would rather use the RAW view.

With a good algorithm for figuring out what is probably important, I 
think most of the time you'd find you didn't prefer the RAW view.  These 
kind of smarts are pretty doable - its not as scary as it sounds.

> One more thing related to the xml RAW view I want to mention is that its 
> implementation is based on a new API from xml/tageditorsupport module. 
> The API is divided to an API allowing access to the document elements 
> and an SPI which provides a document model (a tree of elements) over a 
> text document. There is an implementation of the model provider for the 
> XMLs which is used not only for the XML RAW navigator, but also for XML 
> folding and some other modules. Since the model is update incrementally 
> there is no need to reparse the entire document after each change (which 
> is the bad think what the SAX based approcach must do AFAIK).

With SAX, you only have to parse down to the last element you can 
display.  For large documents, that's usually going to be preferable 
(hmm, I'm liking the backwards SAX parser idea more and more).

> I can 
> imagine a very simple implementation of the outline view built upon the 
> document model API. For more details see "[Issue 61412]  Tag Based 
> Editor Support API review".

I should probably zip you a copy of the entire sources for the NetBeans 
book in DocBook XML.  When your new code will handle that without a 
hiccup, it's ready.  The old XML tree view took half an hour to parse it 
and update.

-Tim

> 
> -marek
> 
> Jesse Glick wrote:
> 
>> I'd like to start a discussion on how XML navigation should look - 
>> i.e. what appears in the Navigator window when you have an XML file 
>> open in the editor (or otherwise selected). Right now, nothing 
>> appears, but:
>>
>>
>> 1. There seems to be an XML navigation feature planned for inclusion 
>> in 4.2:
>>
>> http://ui.netbeans.org/docs/ui/code_navigator/inspector_docs.html
>>
>> I haven't seen it running yet, but apparently it is being worked on 
>> already.
>>
>>
>> 2. Before I knew about that, I wrote a very simple XML navigation 
>> module. It's in contrib/xmlnavigation in netbeans.org CVS (should be 
>> on Dev AU very soon if anyone's interested). I will attach a typical 
>> screenshot. Still has many bugs etc. Based loosely on conversations 
>> I've had with Tim Boudreau about the old xml/tree-edit module.
>>
>>
>> I'm glad to see there is a plan to work on XML navigation and make it 
>> fast and supported, since what I wrote was just a proof of concept. 
>> Still, there is not much overlap between what contrib/xmlnavigation 
>> does and what the UI spec above proposes.
>>
>> The proposed UI spec in sections 1, 3, and 4 shows full XML structure 
>> in a tree, like the old xml/tree-edit module, which I guess could be 
>> useful in odd circumstances (like a huge OpenOffice document with no 
>> newlines) but doesn't seem very compelling. Why would you spend time 
>> browsing through a big complex tree, overloaded with detail, when you 
>> could much more quickly go through the XML text that provides the same 
>> information in a format everyone is familiar with and knows how to edit?
>>
>> contrib/xmlnavigation is more similar to section 2 of the proposed UI 
>> spec, except rather than being specialized for particular J2EE 
>> document types, it does a rough job of showing structure for a wide 
>> range of XML types. Most details of XML syntax and structure are 
>> ignored - only a handful of elements are selected for indexing. The 
>> basic idea is that many common XML document types are broken into 
>> logical sections that can be recognized by common patterns such as 
>> certain elements being used for headers, certain attributes like 'id' 
>> being used for names, etc. For example, an XHTML document like
>>
>> <?xml ...?>
>> <html xmlns="...">
>> <head>
>> <!-- stuff... -->
>> </head>
>> <body>
>> <h1>Title</h1>
>> <p>....content....</p>
>> <p>....</p>
>> <h2>Subtitle</h2>
>> <p>....</p>
>> <h1>Next Part</h1>
>> <!-- ..... -->
>> </body>
>> </html>
>>
>> is displayed as
>>
>> Title
>> Subtitle
>> Next Part
>>
>> Obviously it could be improved by indenting "Subtitle" under "Title" 
>> etc., but the point is to show the real minimum of detail that would 
>> be useful for actual navigation. If I am editing a big XHTML document, 
>> I really do not need to see <p> tags in a tree view! I just want to be 
>> able to jump to major sections of the document quickly and get back to 
>> the text editor: Ctrl-7, Down, Down, Enter. The navigator component 
>> shows an outline, just as for Java files it shows methods and fields, 
>> not every instance of the '+' operator.
>>
>> Same for XSD, XSLT, DocBook, etc. - if the bugs were worked out, 
>> xmlnavigation would show only the important header info: one index 
>> entry per XSD type definition, XSLT template definition, DocBook named 
>> section, etc. (*)
>>
>> Given the current UI spec, my advice would be to get rid of the XML 
>> structure view entirely, and focus on providing views more like those 
>> for web.xml and ejb-jar.xml, though preferring lists (possibly with 
>> minimal indentation) to trees in all cases, for fast navigation and 
>> compact display.
>>
>>
>> Tim Boudreau also wrote in response:
>>
>> <tim>
>>
>> What I want, really, for navigating XML, is something that will show 
>> me things that are likely to be important to me and let me navigate 
>> between them.  Most XML documents have a structure which contains 
>> elements that are what the user thinks of as the "contents" of the 
>> file, and miscellaneous sub-elements that are important to a machine 
>> reading the XML, but are noise to the person who wants to find 
>> something or edit something.
>>
>> A tool that does a best-guess at what those important elements are (or 
>> ideally, understands some types and actually knows) would be useful 
>> anytime I need to edit XML;  a tree showing all the gory details of an 
>> XML file is only useful when I'm interested in the *structure* of the 
>> XML document.  Usually I'm not interested at all in the structure, 
>> what I'm interested in is the content - and mentally the "content" is 
>> much less granular - a tool that shows me every <p/> tag will make me 
>> less, not more productive.  The computer is interested in the 
>> structure;  the user is interested in the content.  A tool that shows 
>> the content well will be valuable;  a tool that shows the thing that 
>> it's easy for the computer to show will not be.  The point of 
>> Navigator is *navigation* - making it easier to use the editor.  
>> Structural views are secondary, navigation is primary.
>>
>> It would be fine if the XML navigator view has a structural view in 
>> the views combo, but it should probably not be the default, and I 
>> would prioritize it considerably below the kind of navigation Jesse is 
>> talking about - not even necessary for the first rev.
>>
>> As for algorithms to figure out what is likely to be "important", 
>> there are plenty of possibilities:
>>
>>  - For known types of documents, actually keep a list of tags that are 
>> worth showing - for example, HTML: h1/2/3, table, form, img;  for 
>> DocBook:  book, chapter, section (and others...)
>>
>>  - For unknown file types, you're going to have to read the whole 
>> thing anyway, so do something like
>>
>>     - Ignore any top level tags that only occur once and contain 
>> everything else in the document
>>
>>     - Find those tags that occur more than once, esp. if each one has 
>> a differentiating string attribute
>>
>> </tim>
>>
>> and with respect to parsing (contrib/xmlnavigation runs a SAX parser 
>> when an XML document is displayed, while the proposed new module is 
>> said to use the editor's syntax elements, which should be able to 
>> reparse incrementally and thus perhaps be faster) he wrote:
>>
>> <tim>
>>
>> The choice of SAX is a good one for two reasons:
>>
>>  - You want to be able to show file structure even when the document 
>> is invalid
>>
>>  - It has the potential to scale - you can stop once you have as many 
>> elements as you can display
>>
>> Now if you wanted to do something *really* useful for a malformed 
>> document, then, in the case of a malformed document
>>
>>  - Run the SAX parser up to the point the problem is found, then stop
>>
>>  - Run a SAX-like parser *backwards* from the bottom of the document 
>> (unless it is huge) and display elements up to the discontinuity, and 
>> a nice red "malformed element" item at the point where it's broken
>>
>> </tim>
>>
>> I will note that using SAX
>>
>> - You can show the outline for a well-formed but invalid file, *but* 
>> undeclared entity refs (like "&nbsp;") abort the parse *if* there is 
>> no DOCTYPE in the document at all (not sure why Xerces behaves this 
>> way but I can't seem to override it); this is annoying if you have a 
>> big DocBook document split into several files using entity includes. 
>> Yet another reason to stick to pure XML Infoset and avoid DTDs and 
>> DOCTYPE.
>>
>> - The current impl does not attempt to stop at the display limit of 
>> the navigation JList, but it probably could.
>>
>> - There is no library that I know of for running a SAXiform parser 
>> backwards as Tim suggests, so this could be tricky.
>>
>>
>> Anyway, just wanted to bring this up in case other people have ideas 
>> about it. Of course there is limited time left for the implementation 
>> in 4.2, so nothing too fancy is feasible, but it would be good to 
>> agree on goals.
>>
>> Cheers,
>> -J.
>>
>>
>> (*) Details for those who are interested:
>>
>> The current contrib/xmlnavigation impl looks for <h1> thru <h6> (for 
>> XHTML) as well as anything with a <title> beneath it (should handle 
>> DocBook etc.). Could be much more precise.
>>
>> It also checks for attributes 'name' and 'id' which are the most 
>> common ways in which this idiom is expressed. This has some bugs, e.g. 
>> XSLT
>>
>> <xsl:call-template name="something"><xsl:param .../><!--...--></>
>>
>> gets indexed (as 'something'), which is wrong since it is using rather 
>> than defining a template.
>>
>> Fortunately it is pretty easy to write unit tests which take various 
>> example documents (preferably real cases you found on the web or 
>> something) and assert that your expectations for a natural-looking 
>> index are matched exactly.
>>
>>
>> ------------------------------------------------------------------------
>>
> 
>