Re: Iterative visiting of all source files

No worries!

So I'm using GNU Global, and that's definitely a big help. Good to know I'm on the right track there.

The reality of this project is that symbols are spread all over the place. I often end up in a situation where I need something that's crosslinked from somewhere else. There are also a lot of auto-generated files and C++ tends to give you a lot of rope to hang yourself with. As a result, I can't really guarantee that everything I need is somewhere tidy. :/

I've noticed that things are a lot better with 24.4 and a 64-bit build. I haven't run into the OOM or weird hanging problems since I switched.

It's good to know I'm basically on the right path. I can tweak things from here as I go.

Thanks!

-----Original Message-----
From: Eric Ludlam [mailto:[email protected]] 
Sent: November-24-14 10:43 AM
To: Jan Goh; [email protected]
Subject: Re: [cedet-semantic] Iterative visiting of all source files

Hi Jan,

Sorry for the delay in replying.  It's been a busy month.  :)

On 10/30/2014 02:09 PM, Jan Goh wrote:
> Hi there,
>
> I mentioned briefly in another response that I have to kind of work 
> around the common semantic paradigm a bit  to get things to work. I 
> just want to check if this is a good way to do things or if there's a better way.
>
> Keeping in mind that I have zero control of the project layout, my 
> source files aren't arranged the way that we'd normally see in an OSS 
> project. Files are arranged by subsystem, with headers and source 
> files intermingled in their own directories. (EDE doesn't really seem 
> to like that, from what I've seen, so I wasn't able to use that. I'm 
> happy to be wrong on this, though.)

There are many kinds of EDE projects, and some care about layout, and 
some don't.   For those that don't, such as any of the generic projects, 
or a cpp-root project, the limitation is that there is only 1 include path.  It would be nice to be more precise, but this has generally been good enough since header names usually don't overlap.  A side effect is that when you customize the include path, it may be very long.

If you use semantic to jump around between tags, I recommend using GNU Global for the "full project parse", and use the semantic setup for it. 
   Semantic will use Global for what it is good at, and then when it falls short, parse your files for the real details.  That will keep all your database tables restricted just to the files you actually use, and keep things running smoothly.

> Anyway, assuming that EDE doesn't meet my needs, I've written this 
> little snippet (with some help from the Internet). I do a save of the 
> semanticdb every time I finish with a directory so that if my 
> processing is interrupted partway through that I still managed to save 
> out some partial data. I've noticed that when I get back to my machine 
> after

Your function seems like a fine way to force lots of files to be parsed. 
  I am curious why you need to parse all the files.

If your project is particularly large, the emacs Process can become very big and in that case, the garbage collector can be a real hassle.  Since most of what you need for basic programming is in the headers, semantic will parse those headers automatically as needed.

> running an update over the whole source tree that sometimes buffers 
> have been opened with the files that were visited, which confuses me-I 
> was under the impression that this code wouldn't do that. I've also 
> had

Check your *Messages* buffer, when an error is throw during parsing or some other aspect of finding a database table, it may leave a buffer open.

I've also fixed a "buffer was left open" problems over the past year.  I think it was related to semantic-symref.  If that sounds similar, perhaps you just need an update.

> emacs lock up part way through because it exceeds some imaginary 
> memory limit (Windows claims that emacs is using less than 800MB of 
> memory-out of the 32GB total; this is also confusing). Shouldn't this 
> be fairly low-impact memory wise, or should I be doing something to 
> clean up after myself after the files are visited?

Emacs/Semantic keeps all DB tables in memory after you load them.  I keep thinking I need to write something that flushes a database if you 
haven't used it in a while, but haven't gotten around to it.   Memory 
needed for parsing is indeed flushed after use.

Eric

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk