RE: [NeoStats-Devel] [Commits] r2699 - in trunk: include src

> 
> GET_CUR_MODULE() should never be empty since the zero level 
> is the core
> psuedo module. The codepath does not need run level for core 
> level events
> but I suppose it is possible. As mentioned in another email, the event
> system now includes a tremendous amount of debug information 
> in order to try
> to track that issue so hopefully that will show up the problem. 
> 
> The DBA system also has additional debug information so if 
> this issue still
> occurs, I should be able to fix it from an appropriate debug.log file.
> 

Coming soon if it still persists.


> > Now, regarding that "path" I was talking about. 
> > GDBM is currently consuming around 1.8Mb of memory for each 
> > open Database.
> > When you have a lot of modules opened up, that memory adds up 
> > (6Mb roughly if we say Set + modexclude + one table for each 
> > module) and go even higher with more tables. I believe its 
> > relating to the cache settings of GDBM which is fine (because 
> > the memory useage only jumps when we open a database and 
> > drops again when we close it. Found that out after hours of 
> > stepping through code with valgrind and gdb).
> > 
> > But I was thinking to trim that memory useage down, can we 
> > close some of the databases (say the Set/modexclude Database 
> > after loading or changing a
> > value?) or Potentially put in a timer that will close 
> > "unused" databases after a period of time. 
> 
> We can potentially keep most of the databases closed, most of 
> the time.
> Currently my DBA layer assumes that a database is open for certain
> operations and errors if not but the general read is used to open them
> rather than an explicit open. There is no reason I cannot 
> always open on
> demand but there would obviously be some performance loss in 
> doing so if the
> database was likely to be closed. 
> 
> Since the majority of database access is config, for our 
> current systems, an
> open read close at boot would suffice for most operations. Changing of
> settings would then also likely need an open write close 
> sequence which is
> where we would suffer performance wise for config info. Other 
> than first run
> installations, this is I guess unlikely to be a big issue since config
> settings are unlikely to change that often. 
> 

Agreed, I don't see a huge problem with doing it for config info. At the
very least, I'm sure you could write it so its open, read all SET config
vars, then close, and on a /msg <bot> set item value can open/close straight
away?


> A similar approach could be assumed for data lists such as 
> excludes and
> access lists.
> 
> Data is generally adhoc access so might need to be more 
> module specific E.g.
> StatServ would need to open writeblock close or open 
> writeblock close within
> it's own code. A timer might work, but the chances of us 
> timing a close at
> the right time from the core are likely to be slim since whatever time
> period we choose might be just before it gets reopened anyway 
> giving us
> worse performance. StatServ is the most database intensive 
> module so if we
> did not cache reads, but just read the whole database, we 
> could manage with
> a block open/read/close at boot then just block 
> open/write/close at runtime.
> Currently IIRC StatServ reads from the databases during runtime when
> channels are created.
> 

Agreed here that there might be issues. I'd be inclined if we go with a
Timer to include a large timeout (30 mins? 45 mins?) if a database isn't
used in that period of time, I think we can suffer the hit of opening it
again when needed.... Or:

How about a optional DBAOpen Call, with a flag to say keep it open or not.
So, If a module writter needs a database that needs to stay open for all
time, they make the effort to call DBAOpen before a DBAFetch, and if they
don't need it to be open they continue to use the DBA interface as it is
not, with just a call to DBAFetch that will automatically open the DB if not
opened, set the timer, and close it after XX>30 minutes

(DBAFetch/DBAOpen are the assumed names, sorry late and I'm not looking at
the code right now)



> Mark.
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: devel-unsubscribe-wool9L35kifE9wlyV4mCnKxOck334EZe@public.gmane.org
> For additional commands, e-mail: [email protected]
> 
>