Re: Bugs introduced that affected jde-import-all (insert semantic tag)

Hi Jeff,

Semantic-fetch-tags is an interface to get you the list of tags in the 
current buffer as quickly as possible.  That's why the buffer has a 
parser state, such as 'up to date' or 'needs reparse'.  It checks to see 
what the current state is and does the fastest parse trick possible and 
then returns the cache.  If the buffer is up to date, it returns the 
cache.  If it needs reparse, it runs the incremental reparser.  If the 
buffer is in a bad state, it returns the cache, and marks the buffer 
that it needs a full reparse sometime in the near future.

The only proper way to get a tag into the buffer cache is to have the 
text in the buffer, and have the buffer get reparsed.  It is possible to 
splice your own tags into the cache, but that has the possibility of 
corrupting the cache.

In Semantic parlance, the "cache" is the list of all tags in a buffer. 
A "table" is a tag list associate with a file/buffer.  A "database", is 
a set of tables, usually associated with a directory, or a specialized 
tag source, like GNU Global, or a Jar file.

If I understand your question correctly, you want to scan through the 
tags, find a good spot to put a import statement, then add an import.

Finding the location, if the file is organized in a classic way, would 
be as simple as:

(goto-char (semantic-tag-end (last (semantic-find-tags-by-class 'include))))

doing the insert, you probably then want to do an:
(end-of-line)
(insert "\n")
(insert "... whatever")

Once that is done, a 'semantic-fetch-tags' will update the buffer cache. 
  If you need to actually know if the refresh was successful, then you 
probably want to use semantic-refresh-tags-safe, which provides a status.

For SRecode, I have indeed contemplated the idea of inserting a tag, and 
splicing the known text back into the tag list as a speedy way of 
modifying the buffer, but I found that the parsing speed is just fine, 
and the 'inhibit-change-hooks' trick I posted about earlier keeps things 
running quickly.

Does that help?
Eric

Jeff Peck wrote:
> Eric, thanks for the info.
>    And, yes, the question is how to make the tag discoverable in the 
> buffer tag list.
> So later, if/when we want to add more "import" statements, we can find 
> the last of the 'include-tag tags,
> and insert into the buffer after that point.
>  
> The docs on semantic--buffer-cache eventually say:
> If you need a tag list, use `semantic-fetch-tags'.  If you need the
> cached values for some reason, chances are you can, add a hook to
> `semantic-after-toplevel-cache-change-hook'.
> Two observations/questions:
> A) yes, we "need" a tag list, and calling semantic-fetch-tags does seem 
> to be beneficial,
>      but i was expecting that semantic-fetch-tags would "fetch" /from/ 
> the elisp/buffer datastructures
>     (which i called a "database" in previous email)
>     Does it, in fact, fetch from the buffer text and put things into the 
> buffer tags list?
> B) semantic-fetch-tags does not seem to actually use or care about any 
> output from (semantic-tag-new-include...)
>    So I still wonder: if we insert text "import foo.bar;" and we want 
> that text to be marked/recognized
>    with a "tag", what is the proper code sequence?
>    Should we specifically use (semantic-tag-new-include), or just call 
> (semantic-fetch-tags)(semantic-parse-changes)
>  
> ps,
>   I did not follow any of your discussion about jar files, etc.
>   This is just an attempt to get the elisp buffer to do the right thing,
>   no acutal Java exectution or compiled code or jars are in the question.
>   Just trying to get correct elisp buffer data, so the existing jdee 
> does more or less the right thing.
>  
>  
>  
> ----- Original Message -----
> From: "Eric M. Ludlam" <[email protected] 
> <mailto:[email protected]>>
> To: "Jeff Peck" <[email protected] <mailto:[email protected]>>
> Cc: "JDEE Development" <[email protected] 
> <mailto:[email protected]>>
> Sent: Saturday, January 09, 2010 2:05 PM
> Subject: Re: [jdee-devel] Bugs introduced that affected jde-import-all 
> (insert semantic tag)
> 
>  > Hi
>  >
>  > Jeff Peck wrote:
>  >> Eric, this is encouraging.
>  >> Without going all the way to srecode templates *today*,
>  >> Can you say what is the appropriate incantation to insert a tag into 
> the
>  >> 'database'?
>  >> I can do: (semantic-tag-new-include new-import nil) to create the tag,
>  >> but how to i make that tag visible to the rest of semantic?
>  >> [so it will be found by: (semantic-brute-find-tag-by-class 'include
>  >> (semantic-fetch-tags)) ]
>  >
>  > When a buffer is parsed with the java grammar it has to fabricate the
>  > tag data structures.  For example, in wisent-java-tags.wy you will see:
>  >
>  > import_declaration
>  >   : IMPORT qualified_name SEMICOLON
>  >     (INCLUDE-TAG $2 nil)
>  >   | IMPORT qualified_name DOT MULT SEMICOLON
>  >     (INCLUDE-TAG (concat $2 $3 $4) nil)
>  >   ;
>  >
>  > The INCLUDE-TAG is a wisent macro that expands to
>  > (semantic-tag-new-include ...)
>  >
>  > Thus, if you use the code as you quoted above, it just creates a data
>  > structure, but doesn't insert it into a buffer, a data structure, or
>  > anything else.
>  >
>  > To make it discoverable, you need to explain why it should be
>  > discoverable.  Ie, should it be a part of a buffer tag list?  In that
>  > case, you would just modify semantic--buffer-cache, though I recommend
>  > against that.
>  >
>  > If you have some constant list of tags that should always be referenced,
>  > such as a library or something derived from a .jar file, then you would
>  > make a new kind of Semantic database object, much like that found in
>  > semanticdb.el, which will be accessed by search routines for your files.
>  >  In CEDET/CVS, there is a prototype for accessing jar files that Joakim
>  > wrote.
>  >
>  >>> srecode specifically knows how to insert tags, so you could do this:
>  >>>
>  >>> (srecode-insert-tag (semantic-tag-new-include "myInclude" nil))
>  >> I'm using cedet-1.0pre6 and do not find a "srecode-insert-tag"
>  >
>  > Sorry, it is srecode-semantic-insert-tag.
>  >
>  >
>  >>> ... I have a rough template already that would probably need a real
>  >>> java programmer to improve it.
>  >>
>  >> If you have something, send it along!
>  >
>  > It is easy to try.  If you have CEDET/CVS, enable
>  > global-srecode-minor-mode.  Then try this:
>  >
>  > Visit /tmp/new.java
>  >
>  > C-c / /  ;;  srecode-insert
>  >
>  > The default it offers should be file:empty.  Press RET
>  >
>  > You now have some misc java code inserted.
>  >
>  > Go where you want to add an import statement.  Type:
>  >
>  > C-c / /  ;; srecode insert
>  >
>  > It should offer  declaration:
>  >
>  > Add "import", and it should insert an import statement, or bits of one
>  > anyway.  Things change a bit here depending on which version of the
>  > template file you have and if you have field-editing turned on or not.
>  >
>  >> Also, if you have any theory why doing a (semantic-parse-changes) works
>  >> when evaluated as a separate command, but not as around/after advice,
>  >> that might also be helpful information.
>  >
>  > The incremental parser will short-circuit and do nothing whenever the
>  > quotes ", or parens {, (, [, etc don't match up.
>  >
>  > You can set semantic-edits-verbose-flag to non-nil to get additional
>  > messages about when the partial reparse needs to bail.
>  >
>  >> CEDET is really great stuff, but I haven't found the magic decoder ring
>  >> to explain the design/threory behind it; and there is so much there, 
> with
>  >> many levels of tools and abstraction that it is difficult to 'discover'
>  >> a way in.
>  >
>  > Please ask questions if you have something specific you need to
>  > understand.  I like answer specific questions, as often that helps me
>  > update update the doc.  Otherwise the doc is a good place to start.
>  >
>  > As for how buffers are updated, there are several states a buffer can be
>  > in.   If you use semantic-load-enable-semantic-debugging-helpers, it
>  > will turn on several things I've mentioned so far.  One new one is
>  > 'show-parser-state-mode'.  This will show a small character combo in the
>  > mode line on the left.  Check the help for the meaning.  You will see
>  > that there are several states:
>  >
>  >  `-'  ->  The cache is up to date.
>  >  `!'  ->  The cache requires a full update.
>  >  `~'  ->  The cache needs to be incrementally parsed.
>  >  `%'  ->  The cache is not currently parseable.
>  >  `@' <mailto:`@'>  ->  Auto-parse in progress (not set here.)
>  >
>  > Thus, if you are in a debugger stepping through code, you can now see
>  > what the current buffer parse state is, which could also help explain
>  > what you see.
>  >
>  > A typical buffer life cycle goes like this:
>  >
>  > file loaded - requires full update
>  > idle timer goes off - cache up to date
>  > user edits - cache needs incremental parse
>  > idle timer goes off - not currenty parseable (user added on ( for 
> example)
>  > user edits - no change in state
>  > idle timer goes off - cache is up to date (parens now line up)
>  >
>  > Every once in a while, user edits might force a full reparse, but that
>  > is (hopefully) rare.  Usually happens when you wrap or unwrap large
>  > parts of a buffer in new { }.
>  >
>  > The incremental parser is a bit finicky in what it can do.  The chars
>  > all need to line up, otherwise it leaves the buffer un-reparsed.
>  >
>  > Enjoy
>  > Eric

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev