Re: Schema provider/CA lookup status

On Thu, 2006-07-27 at 06:02 -0700, Martin Aspeli wrote:
> 
> Martin Aspeli wrote:
> > 
> > I tried to experiment with this in 'company' last week, but
> > unfortunately, I didn't get very far.
> > 
> 
> Now I got a bit further :)
> 
> The current state of affairs can be seen here:
> http://svn.plone.org/svn/collective/company/trunk/content/department.py
> 
> Basically, the problem is that if the schema changes, you need to re-run
> ClassGen to get accessors and mutators. Various bits of AT depend on those
> being place. ClassGen works on the class object itself, and assumes a
> .schema class attribute exists (it uses klass.schema.fields() to find out
> which accessors to generate).
> 
> My first thought was to make .schema a dynamic property. Unfortunately, this
> may not be possible, because there's no class-property equivalent to the
> instance property() generator in Python. A meta-class may be an option, but
> even then there may not be enough context to do an adapter lookup.
> 
> It turns out, though, that this has been thought about before.
> VariableSchemaSupport, which is what I'm using at the moment, calls a hook
> getSchema(). It then hashes the schema and caches it in a module-level
> variable. The next time, it'll look it up in the cache if it can. If the
> schema has changed, however, it calls ClassGen (or rather, it's own version
> of ClassGen that doesn't depend on klass.schema) to generate the accessors
> and mutators.
> 
> ATSchemaEditorNG has a similar mix-in at
> http://dev.plone.org/collective/browser/ATSchemaEditorNG/trunk/ParentManagedSchema.py.
> It's rather complex, though, and I can see a few problems with it (bare
> excepts, use of hasattr). It makes use of _v_ attributes for some of the
> schema fields, presumably as a cache again.
> 
> This brings me to the performance question. Rob suggested
> VariableSchemaSupport is very slow. Given that it hashes and caches the
> schema (once per Zope re-start I guess, since it's a module-level variable)
> I can't see how it can be *that* bad (unless calculating the hash is very
> expensive). Rob also said CMFMember experienced better speed when switching
> to ATSchemaEditorNG, but that seems to cache things in _v_ variables which
> are likely to be re-calculated more often. Perhaps the difference is that
> VSS uses ClassGen whereas ATSENG uses its own class attribute generation.

We've debugged and profiled this extensively and the hashing is the
culprit.
http://sourceforge.net/mailarchive/forum.php?thread_id=12989125&forum_id=32048

If you fix the hashing the performance equals that of schemas defined at
module level. If you want to verify this, just change the hashing line
to "hash = id(s)" (This is obviously not a proposed fix)

> So, is VSS a viable option? Or should I try to repllicate what ATSENG is
> doing?

Let's just fix the hashing. Maybe we can just invalidate the cache when
schema's are edited or hash the schema with it's modification date.

-- 
Roché Compaan
Upfront Systems                 http://www.upfrontsystems.co.za


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV