Dexterity performance issues.

Hi all,
  During trying to optimise reindexObjectSecurity in Plone, we managed to discover that Dexterity is significantly slowed than Archetypes. This may be already known by some, but seemed to go very much against what I’d generally been led to believe by various people ;)

In short, if I use plone.api.content.create to create a dummy tree of folders in Plone with each folder containing 7 child folders to a level of 5 deep (ie. 19,600 objects in total) it takes:

ATFolder: 19 minutes
p.a.ct Folder: 37 minutes

That is with the catalog disabled during the loop adding, and a batch reindex at the end:
https://github.com/ploneintranet/experimental.securityindexing/blob/master/src/experimental/securityindexing/benchmarks.py#L63

To run reindexObjectSecurity over that tree (ie. calling allowedRolesAndUsers on each object and indexing result) takes:

ATFolder: 32 seconds
p.a.ct Folder: 136 seconds

(stock Plone 4.3 with p.a.contentypes, no local roles optimisations)

After digging around for most of a day, it seems like the main culprit is the __get__ call in p.dexterity.content.FTIAwareSpecification:
https://github.com/plone/plone.dexterity/blob/master/plone/dexterity/content.py#L69

In short, this is a method that on AT types is in C, and in Dexterity is in Python (I think). The DX call also has a mutex around a module level schema cache which may not be helping things (although in this case there is only one thread). In this *specific* case, setting a class-level attribute of __ac_local_roles_block__ = None does help as is means that accessing that attribute on a DX object stops there and don’t trigger then custom __getattr__ code. But it is a wider problem. Anytime you call e.g. IFoo.providedBy(my_dx_object) then it takes a long time… and we do that *a lot*.

Anyone looked into this?

I love Dexterity, and it gives a much better development experience than AT… but I’m worried that this may start to come as a shock to people with large sites if they upgrade to plone.app.contenttypes and then find certain operations taking over 4 times longer to complete.

-Matt

-- 
Matt Hamilton, Technical Director
Netsight Internet Solutions Limited
http://www.netsight.co.uk/matth
Tel: 0117 90 90 90 1 Ext. 13

Registered in England No. 3892180
Registered office: 40 Berkeley Square, Clifton, Bristol, BS8 1HU


------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs