Re: pagesize/pagecount change in beta4

Hello

Thanks for the reply

I changed the pagesize/pagecount in the code (if everything turns well 
I'll make them configurable) and I am now testing what are the 
implications of having 20-60k collections (memory/descriptors/disk space 
usage). Up to now it didn't turned out too well (I'll write a summary 
once I finish and reach some conclusions). I might need to play with the 
number of descriptors / collection as well.

Vadim Gritsenko wrote:
> Catalin Braica wrote:
>> I have a database that uses a lot of collections (several thousand) 
>> and wanted to move from xindice 1.1 beta2 to 1.1 beta4, and found the 
>> following problem: 
>> http://issues.apache.org/bugzilla/show_bug.cgi?id=31159
>>
>> Looking at the code (Paged.java) I see that in both beta2 and beta4 
>> the pagesize was 4096, and pagecount was 1024, but looking at the 
>> .tbl files on the disk (just a 'blank' database with system 
>> collection) shows great difference in size between beta2 (12Kb per 
>> .tbl) and beta4 (6Mb per .tbl). So what really changed ?
>
> beta2 did not allocate file properly, and beta4 also has an error: it 
> should be 4Mb, not 6Mb. Current SVN has the fix.
>
>> Having a database with many collections upgrading to beta4 means a 
>> lot of space. So what can I do ?
>
> I'll be adding configurability for pagecount parameter (IIUC pagesize 
> is already configurable) sometime soon. You can try doing it yourself 
> if you need it sooner.
>
>
>> Looking at the code, I can edit Paged.java and changed the pagesize 
>> and pagecount to match my needs. Does anyone have any suggestion on 
>> finding the right ones (I assume changing these can impact 
>> performance) ? Have anyone played with these ?
>
> Reduce pagecount to whatever you feel comfortable with. Do not reduce 
> pagesize too much.
>
>
>> And moreover, is there a way so I can change through the 
>> configuration file the pagesize for all the collections created (some 
>> of their names are IDs computed at runtime)?
>
> This feature is planned.
>
>
>> I did not succeeded  in setting pagesize in the configuration file 
>> not even per collection name, for a single collection.
>> Also since collections in my case are created through XML RPC, can I 
>> supply a pagesize option when creating a collection ? (as an 
>> alternate method, relying on the client instead of server).
>
> Yes. You can pass collection config to the CreateCollection method. 
> See CreateCollection.java.
>
> If your client is in java see 
> org.apache.xindice.integration.client.XmlDbClient for sample code.
>
> Vadim
>