Re: AW: how can I prevent "too many open files"?

Eickvonder Bjoern wrote:
> I got a fixed number of 16 collections including the root collection plus 2 collections per "test that a user publishes". In my current state I have 29 of these tests in my system (which should only be a medium amount). Now Xindice does somehow have two file handles for each idx, tbl and tbl from the meta directory. This results in a total number of 12 handles per collection, i.e. (16+29*2)*12 = 888 file handles held open. If now adding all the jars, cached jars, log-files and so on that are held open by tomcat for each running webapplication...
> The problem is that I could increase the size by changing the parameters within the kernel and then recompile it, but how do I tell customers, that install the application on their on servers, that they first have to recompile their kernel?

JFYI, Xindice can use up to *16* file descriptors open for each paged file 
(configurable). Collection filers, indexes, system and meta collections all use 
paged file. Reason for several descriptors is really obvious: you want several 
clients to be able access the database at the same time, don't you? Hence, you 
have to have several descriptors to be used in parallel threads.

See Paged.java for details.

Vadim


> Bjoern 
> 
> -----Ursprüngliche Nachricht-----
> Von: Todd Byrne [mailto:[email protected]] 
> Gesendet: Dienstag, 7. Juni 2005 18:43
> An: [email protected]
> Betreff: Re: how can I prevent "too many open files"?
> 
> Xindice and all XML Databases are really designed to have a few fixed number of collections open and lots of individual xml files. I would recommend finding another way to use the collections.
> 
> If you set on having lots of collections which no matter what probably won't scale try using the Embeded database where you could get better access to the in memory collections.
> 
> Quick question how many collections are you opening?
> 
> Todd
> 
> Eickvonder Bjoern wrote:
> 
>>Hello,
>>
>>I'm using Xindice 1.1b4 (but tried it with current head too) embedded 
>>within a webapplication using tomcat 4.1.27 / jdk 1.4.2 on a suse 
>>linux system. I've configured 4 indexes for each collection. My 
>>problem is now for each collection the java process opens the tbl-file 
>>and the four idx-files when I register my database and only closes 
>>them if I shutdown my database. The problem is now that I don't have a 
>>fixed number of collections, but they can increase depending on what 
>>the user is doing in the application. The result is that if the number 
>>gets too large i get a "too many open files"-exception.
>>I know that I can increase the maximum number of open files per 
>>process within linux but as the number of collections is not fixed 
>>this is not a real solution.
>>Is there any possibility to tell Xindice to close a single collection 
>>in such a way that the idx- and tbl-files are no longer held open and 
>>are only reopened if I access them? Note that I already close my 
>>collections after each operations by
>>
>>Collection col = org.xmldb.api.DatabaseManager.getCollection(m_dbURL + 
>>path); ... some operations on col ...
>>col.close();
>>
>>but this does not have the desired effect.
>>
>>Thank You
>>
>>Bjoern Eickvonder