Re: Persistent outrigger doesn't shrink logstore
Asaf Lahav <[email protected]> Tue, 25 Nov 2008 11:08:19 +0200
| Newsgroups | gmane.comp.java.sun.jini |
|---|---|
| Message-ID | <[email protected]> |
Hi John,
Thank you for helping.
Here are the configuration files:
startAll.config:
---------------------------------------------------------------
import com.sun.jini.start.NonActivatableServiceDescriptor;
import com.sun.jini.start.ServiceDescriptor;
import com.sun.jini.config.ConfigUtil;
com.sun.jini.start {
private static policy = "C:\\Program
Files\\jini2_1\\installverify\\support\\jsk-all.policy";
private static host = ConfigUtil.getHostName();
private static port = "8081";
private static jskdl = " http://" + host + ":" + port + "/jsk-dl.jar";
serviceDescriptors = new ServiceDescriptor[]{
new NonActivatableServiceDescriptor(
"",
policy,
"C:\\Program Files\\jini2_1\\lib\\classserver.jar",
"com.sun.jini.tool.ClassServer",
new String[]{"-port", port, "-dir",
"C:\\Program Files\\jini2_1\\lib-dl",
"-verbose"}),
new NonActivatableServiceDescriptor(
"http://" + host + ":" + port + "/browser-dl.jar" + jskdl,
policy,
"C:\\Program Files\\jini2_1\\lib\\browser.jar",
"com.sun.jini.example.browser.Browser",
new String[] { "C:\\Program
Files\\jini2_1\\installverify\\support\\browser.config" }),
new NonActivatableServiceDescriptor(
"http://" + host + ":" + port + "/reggie-dl.jar" + jskdl,
policy,
"C:\\Program Files\\jini2_1\\lib\\reggie.jar",
"com.sun.jini.reggie.TransientRegistrarImpl",
new String[] { "C:\\Program
Files\\jini2_1\\installverify\\support\\reggie.config" }),
new NonActivatableServiceDescriptor(
"http://" + host + ":" + port + "/fiddler-dl.jar" + jskdl,
policy,
"C:\\Program Files\\jini2_1\\lib\\fiddler.jar",
"com.sun.jini.fiddler.TransientFiddlerImpl",
new String[] { "C:\\Program
Files\\jini2_1\\installverify\\support\\fiddler.config" }),
new NonActivatableServiceDescriptor(
"http://" + host + ":" + port + "/mahalo-dl.jar" + jskdl,
policy,
"C:\\Program Files\\jini2_1\\lib\\mahalo.jar",
"com.sun.jini.mahalo.TransientMahaloImpl",
new String[] { "C:\\Program
Files\\jini2_1\\installverify\\support\\mahalo.config" }),
new NonActivatableServiceDescriptor(
"http://" + host + ":" + port + "/mercury-dl.jar" + jskdl,
policy,
"C:\\Program Files\\jini2_1\\lib\\mercury.jar",
"com.sun.jini.mercury.TransientMercuryImpl",
new String[] { "C:\\Program
Files\\jini2_1\\installverify\\support\\mercury.config" }),
new NonActivatableServiceDescriptor(
"http://" + host + ":" + port + "/norm-dl.jar" + jskdl,
policy,
"C:\\Program Files\\jini2_1\\lib\\norm.jar",
"com.sun.jini.norm.TransientNormServerImpl",
new String[] { "C:\\Program
Files\\jini2_1\\installverify\\support\\norm.config" }),
new NonActivatableServiceDescriptor(
"http://" + host + ":" + port + "/outrigger-dl.jar" + jskdl,
policy,
"C:\\Program Files\\jini2_1\\lib\\outrigger.jar",
"com.sun.jini.outrigger.PersistentOutriggerImpl",
new String[] { "C:\\Program
Files\\jini2_1\\installverify\\support\\outrigger.config" })
};
}
Outrigger.config:
--------------------------------------------------------
com.sun.jini.outrigger {
initialLookupGroups = new String[] { System.getProperty("user.name") +
"InstallVerifyGroup" };
persistenceDirectory = "C:/Program
Files/jini2_1/installverify/persistenceDir";
store = new com.sun.jini.outrigger.snaplogstore.LogStore(this);
}
-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Tuesday, November 25, 2008 2:47 AM
To: Asaf Lahav
Cc: [email protected]
Subject: Re: Persistent outrigger doesn't shrink logstore
Asaf Lahav wrote:
> Hi John,
>
> If "store reduction" is performed every 1000 actions, then 5000 actions
were
> supposed to trigger the reduction process.
It wasn't clear from your messages if the logs were growing or the
object database, 5000 should be enough to see the logs GCed, but it
might not be quite enough to see the object database be GCed.
[as an aside, it turns out you aren't using logstore, but snaplogstore,
which doesn't use PSEPro but just std. Java object serialization - so
the object DB GC evey 5 logs isn't an issue....]
> Another problem is that the process holding the java space is constantly
> accumulating more and more RAM.
>
> The version I'm using is jini2.1...
> Here is the files list where the persistent space logs the
entries/actions:
>
> Volume in drive C has no label.
> Volume Serial Number is 28C5-1A79
>
> Directory of C:\Program Files\jini2_1\installverify\persistenceDir
>
> 11/23/2008 10:40 AM <DIR> .
> 11/23/2008 10:40 AM <DIR> ..
> 11/23/2008 10:00 AM 25,006,180 LogStore.10
> 11/23/2008 10:04 AM 24,938,420 LogStore.11
> 11/23/2008 10:08 AM 25,057,032 LogStore.12
> 11/23/2008 10:12 AM 24,799,524 LogStore.13
> 11/23/2008 10:16 AM 24,542,008 LogStore.14
> 11/23/2008 10:20 AM 25,108,532 LogStore.15
> 11/23/2008 10:24 AM 24,851,184 LogStore.16
> 11/23/2008 10:28 AM 24,747,860 LogStore.17
> 11/23/2008 10:32 AM 25,161,200 LogStore.18
> 11/23/2008 10:36 AM 24,799,520 LogStore.19
> 11/23/2008 09:28 AM 25,045,356 LogStore.2
> 11/23/2008 10:39 AM 14,640,429 LogStore.20
> 11/23/2008 09:32 AM 25,165,292 LogStore.3
> 11/23/2008 09:36 AM 24,700,372 LogStore.4
> 11/23/2008 09:40 AM 24,700,280 LogStore.5
> 11/23/2008 09:44 AM 25,371,304 LogStore.6
> 11/23/2008 09:48 AM 24,592,520 LogStore.7
> 11/23/2008 09:52 AM 25,152,096 LogStore.8
> 11/23/2008 09:56 AM 24,609,056 LogStore.9
> 11/23/2008 10:40 AM 0 output.txt
> 11/23/2008 10:36 AM 362,413 Snapshot.175
> 21 File(s) 463,350,578 bytes
> 2 Dir(s) 1,089,073,152 bytes free
>
Looking at the number of log files, I would have guessed that the log
processing thread died, is deadlocked, or fallen behind, but that
doesn't seem consistent with the Snapshot file having an update time
later than the update time of the last log file (suppose that could
happen if the processing thread was really bogged down, but I would
expect the log process thread to catch up and that doesn't seem to be
happening in your case). I am also confused by the store being on the
175th snapshot - I don't have the source handy, but that seems
inconsistent with the number of logs.
I assume there no messages in stdout or stderr?
I think the next step would be to turn on logging, be a while, so I will
need to check the code tomorrow what the right logger would be.
> Are there any definition I can perform in order to control the
reduction/GC
> process?
There are a few configuration options that control the snaplogstore.
They are listed in the top level java doc for the com.sun.jini.outrigger
package - but my guess is none of them are relevant here. Can you post
your complete Outrigger config file?
> Any help on this would be very appreciated.
>
> Thanks,
> Asaf
>
>
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Sent: Thursday, November 20, 2008 4:01 PM
> To: Asaf Lahav
> Cc: [email protected]
> Subject: Re: Persistent outrigger doesn't shrink logstore
>
> Asaf Lahav wrote:
>> The logStore files are constantly growing in size.
>>
> There are a bunch of different files logstore uses, which one are you
> seeing a problem with? An "ls -l" would be helpful.
>
>> I would expect the logStore to reduce the size of the repository as
> entries
>> are being taken.
>> On a test I did I write about 5000 entries and then took all the existing
>> 5000 entries.
>> Yet, two hours later, the logStore files are still a few megabytes in
> size.
>> On another test I did, I defined 2 minutes in lease time for 5000 entries
>> and after 3 minutes tried to take those entries. Non were returned (as
>> expected) yet, the logStore files were still as big as if the entries are
>> still stored in there.
>>
>> I would expect the logStore files to be reduced as entries either expire
> or
>> taken.
>> Currently it seems as if the logStore files constantly increase in size
> and
>> may consume all the disk space if given enough time.
>>
> The "store reduction" process isn't continuous with logStore (or
> snapLogStore) it happen in chunks. By default (in v2.0.2 - BTW, what
> version are you using?) individual log files grow until they contain
> 1000 operations (takes, writes, lease expirations, or event
> registrations - don't remember if event deliveries get logged). Once a
> log is full the operations are replayed into an object db (PSEPro), and
> once the update to the db is committed the log file is removed. We
> periodically need to GC the object DB, too, that we do every 5 logs
> (again by default). It isn't clear to me which files you are seeing a
> problem with. If you have 10 log files, that is a problem, if you only
> have one then the log file part of the process is working. If files
> associated with the object database appear to be growing without bound,
> it may just be that you haven't done enough operations to trigger the
> necessary GC.
>
> --------------------------------------------------------------------------
> Getting Started: http://www.jini.org/wiki/Category:Getting_Started
> Community Web Site: http://jini.org
> jini-users Archive: http://archives.java.sun.com/archives/jini-users.html
> Unsubscribing: email "signoff JINI-USERS" to [email protected]
--------------------------------------------------------------------------
Getting Started: http://www.jini.org/wiki/Category:Getting_Started
Community Web Site: http://jini.org
jini-users Archive: http://archives.java.sun.com/archives/jini-users.html
Unsubscribing: email "signoff JINI-USERS" to [email protected]