Re: getFolderContents: serious performance problem

On Oct 24, 2006, at 5:39 PM, Benoit Delbosc wrote:

> Hi
>
> Yves Bastide a écrit :
>> Hi,
>> CPSDefault.utils.getFolderContents causes a big performance  
>> problem on a site
>> we installed. A subsection containing about 4000 documents takes  
>> some 40
>> secondes to display for a 'SectionReader' user, versus 7 seconds  
>> for the Manager.
>> The profiler points out one main culprit: getRolesInContext (and also
>> DateTime's _parse_args). (*)
>> My first try was to give the View permission to the Authenticated  
>> role in the
>> subsection instead of using a SectionReader local role, but there  
>> wasn't much
>> improvement. After reading CPSDefault.utils.filterContents, the  
>> reason is obvious:
>> {{{
>>         # filter non effective or expired published content
>>         if not _checkPermission(ModifyPortalContent, item):
>>             review_state = wtool.getInfoFor(item, 'review_state',  
>> 'nostate')
>>             if review_state == 'published':
>>                 doc = item.getContent()
>>                 if now < doc.effective() or now > doc.expires():
>>                     continue
>> }}}
>> (_checkPermission could be pushed down in the tests; but I'm not  
>> sure whether
>> that would be much more efficient. The dates tests may also be  
>> costly.)
>> After my colleague remarked that searches were much quicker (~ 5  
>> seconds, and
>> with a more complex display), we tried to force using
>> getCatalogFolderContents, and lo! folder_view became 8 times faster.
>> So I'd like to know
>> * How to systematically use getCatalogFolderContents instead of
>> getFolderContents (in a clean way :) ) -- I'm not yet comfortable  
>> with CPS
>> views' many levels of inclusion
> Using the ZMI add a boolean property to the portal named  
> 'use_catalog_for_folder_contents' with a True value.
>
>> * Why it's not the case already (I've had incomplete indexes, but  
>> assume it's
>> my fault)
> It is a bit slower if you have only few items in a folder.
>
>> * Bonus question: can one sort on several indexes -- i.e. some of  
>> my documents
>> have a specific field which should be used as a default sort key;  
>> how to get
>> also documents of other types?
> I think you need to replace the default sort on  
> position_in_container with 2 sort keys: your new index first then  
> position_in_container.

If you want to go further, check also CPSDashboards, a component  
that's basically meant for this (at the expense of more  
configuration, it's not a drop-in replacement).

It'd be applicable if
	- you want to control according to portal_type of folder (typically  
you'd have BTree folders to hold lots of docs, and you'd need a new  
portal_type anyway for that)
	- you don't need the "move up" etc buttons (didn't make them yet,  
but you can specify an action category to get buttons)
	- you want to fully customize the columns, etc.

CPSCourrier's Mailbox document type can be used as a fully working  
example/starting point.

It's also worth noting that for really big sites (> 2.10^5  
documents), switching to CPSLuceneCatalog is just a matter of  
changing a widget's meta_type within CPSDashboards (basically because  
it was designed with Lucene in sight ).

---------
Georges Racinet,   Nuxeo SAS
Open Source Enterprise Content Management (ECM)
Web: http://www.nuxeo.com/ and http://www.nuxeo.org/ - Tel: +33 1 40  
33 79 87



_______________________________________________
cps-devel mailing list
http://lists.nuxeo.com/mailman/listinfo/cps-devel