Re: Schema-by-interface options

On Thu, 21 Sep 2006 05:55:31 -0700, Martin Aspeli <[email protected]> wrote:


<reorder>
>
> So - this stuff is in production and keeping you happy? Good :)

well, its not in production yet, it is being heavily utilized for a system  
to be in production by plone conference. i just wanted to introduce  
contentflavors, as it spoke to the conversation here on thread, and is of  
general utility. it handles all the use cases presented in this thread,  
the discussion to me seems to have shifted to implementation details.  
after thinking about the global scenarios, i'm not really interested in  
making contentflavors deal with that scenario via class monkey patching,  
i've outlined an alternative global implementation using contentflavors  
and some brainstorming notes on global scenarios below.

if your going to put schema delegation directly in AT, please do make that  
delegation, 100% policy free and make adapters implement policy.

rest of response inline.

cheers,

kapil

> Hi Kapil,
>
>> okay let's say i have an article content type, i have several different
>> flavors so to speak of article, feature, book review, movie review,
>> biography, etc. ie. a common base type w/ different applicable schemas  
>> and
>> views based on usage.
>>
>> another example... i have a places content type, which has movie  
>> theaters,
>> restaurants, museums, parks, bars, etc (ad nauseum). this is a better
>> example of the notion of wanting multiple flavors on a content type,  
>> ie. a
>> place is both a bar and a restaurant, both flavors contribute additional
>> specific schema.
>>
>> last example, plone.rules ;-) firing to add a press release flavor to a
>> news items put in press folder.
>
> Cool. :)
>
> Could you elaborate a bit more on how the "global" flavour thing may
> look? For example, imagine a scenario where you have a "global"
> 'tagging' flavour that adds a 'tag' field to any object marked with
> ITaggable.
>
> Let's say that you then *also* want to add a specific 'article'
> flavour or whatever.
>
> How would this look?

you would setup a local event adapter within a site folder for content  
created, and apply flavors to the content on whatever basis ( in this case  
implementing ITaggable).

i've been doing some more thinking about the global scenario, with global  
implementations. part of the difficulty of using a global implemetation  
are the opposing use cases, ie. applying global behavior local to a site.

global behavior implementation based on class manipulation is a radically  
different notion and is inherently global in nature. additionally  
archetypes content have way too many responsibilities as regards schema,  
which makes such monkey patching both fragile and complex to support class  
weaving imho.

the sort of implicit global monkeying of arbitrary classes seems much more  
error prone and which in imho is the opposite of what makes a good  
framework.

also, notions of expected ordering application of schemas are also harder  
on a purely global basis, as they require awareness of configuration space  
ordering.

as before, i think the easiest ability to support global AT behavior with  
current local flavors implementation is via an event listener.

both local flavors and application of global behavior are much simpler  
when you assume z3 schema weaving, formlib even makes it fun ;-), and you  
basically get the behavior both explicitly ( via formlib view class form  
fields definition ) with delegation to adapters for storage.

ie. in the case of z3 schemas/behaviors, you could already do this without  
any additional infrastructure, using plain old adapters, of which flavors  
are really just sugar for working with AT and some additional behavior.

class IPloneTagMarker( Interface ): pass

class MyTagger( object ):
     def __init__(self, context):
         self.context = context

     def tag(self, tags):
         # or any other alternative implementation
         self.context.Schema()['mutator'].getMutator( self.context )( tags )

     def getTags(self):
         return self.context.Schema()['tags'].getAccessor( self.context )()

<adapter
   class="ITaggable"
   for="IPloneTagMarker"
   handler=".MyTagger"
   />

<five:implements=".myContent.IPloneTagMarker"/>


to support the global case w/ a global implementation  cleanly (without  
side effects) you have to delegate to some local notion but, making  
behavior local while not affecting side effect global changes is hard, you  
get to use a local adapter registry sure.. but you can't avoid the side  
effects of global modification when doing class modification. to get  
around global modifications you'd need, a new primitive is needed to tie  
local configuration to global.

one solution, you could have a class provides which queried for a provider  
utility, though it also creates dependencies on traversal for site  
discovery. though testing frameworks that utilize would be straightforward  
via implementations of the applied interfaces, the runtime integration  
does bring up the importance of functional testing.

building on that primitive a marker utility for classes to support the  
runtime persistent global case, ie. enabled via ui, in which case your  
primary concern is persistability of declarations in a utility  
implementation.

i took a look at it, its seems possible, though it might not be, there a  
couple of invariants that need to be maintianed by interface  
specifications and its fragile to constructs like classOnlyImplements  
which replace class implements specifications. here's a possibly broken  
prototype

 from zope.declarations import Implements

class LocalClassImplements( Implements ):
     """ we don't setup subscription of local bases
         any setBases api should be called without local context.
     """

     def setBases(self, bases):
         # Register ourselves as a dependent of our old bases
         for b in self.getBases()
             b.unsubscribe(self)

         # Register ourselves as a dependent of our bases
         self.__dict__['__local_bases__'] = bases
         for b in bases:
             b.subscribe(self)

         self.changed()

     def getBases( self ):
         b = self.__dict__['__local_bases__']
         marker = component.queryUtility( IClassMarkerUtility )
         if marker is None:
             return b
         return

     __bases__ = property( getBases, setBases )

<snip>

> Look at the example immediately above the header "Registering portlet
> managers" and the use of it immediately below that header.
>
> Also take a look at GSLocalAddons in the collective.
>
>> aye, variableschemasupport had some borked and expensive hash method  
>> which
>> always failed afaicr, volatile caches are compariatively fast and well
>> known.
>
> True, but it also depends on the lifetime of that cache. The hash was
> expensive (and did the opposite of what it was trying to do), but
> running ClassGen when it's not necessary is expensive too.
>

i added an additional cache layer, such that, class gen semantics (not  
implementation) are never run when its not nescessary. the implementation  
has some docs and details (trunk/schema.py)


>> if the object is in memory its _v_schema is only going to be
>> composed once till the object is either ghosted or mutated,
>
> Yeah - which may be reasonably often still, especially if it's  
> per-instance.
>

the new cache scheme uses a two layer cache, global by class, composition  
combination and instance, it saves memory and never needs extraneous  
composition calls.

> Also, I assume you generate methods on the instance and not on the  
> __class__?

of course, mucking with a global like class multiply at runtime is way too  
error prone and a horrible framework semantic.

>
>> incidentally,
>> contentflavors already does instance cache invalidation via event,  
>> though
>> moving to the cache keys i proposed previously, you never need to
>> invalidate a composed schema, just signal accessor/mutator gen on  
>> instance.
>
> Why is it that you'd never need to invalidate?

i treat schemas as immutable post startup. as you mentioned before,  
persistently storing schemas leads to all kinds of issues, once you get  
away from that notion, you can compose schemas and save composition,  
because for its definition combination its always valid. you can utilize  
have mutable schemas if mutation triggers events that can be utilized for  
cache invalidation.

>
>> > That's true. But if AT says, "I support the application of schemas by  
>> an
>> > adapter lookup", then that *does not work* without calling a
>> > ClassGen'ish thing at least once (after the lookup of the schema has
>> > been made the first time). This is because AT's basic ClassGen works  
>> at
>> > class loading time, and cannot do the adapter lookup (no context), and
>> > it relies on the .schema class-variable, which cannot be made dynamic
>> > like an instance-variable could wither @property. So, I think it'd  
>> dumb
>> > to support some way of delegating schemas to an adapter if you know  
>> that
>> > anyone who actually uses that functionality *must* do a dance with
>> > generating methods, opening that up to re-invention and bugs and
>> > possibly conflicts.
>>
>>
>> but those schema adapters are all going to have different semantics
>> potentially different semantics about lifecycle (ie. the always on case,
>> really only needs once in a process, instance mutation on mutation )..
>> which is exactly why people have reimplemented classgen semantics that
>> work for their use cases, coming up with a base set of common primitives
>> for doing the drudgery is a step forward, but the application is always
>> going to be adapter driven. you can separate out to two stage adapters
>> which is what contentflavors does (although it takes a step further in
>> that you don't need an adapter just point a zcml flavor to a schema),  
>> such
>> that folks just wanting to use the framework to add schema don't have to
>> deal with class gen semantics, but the top level integration adapter
>> ( schemasetprovider for lack of a better term) has to know when it needs
>> to apply accessor/mutator generation (or defer via event to an adapter  
>> who
>> does). ie when you talk about changing things at runtime which need
>> initialization, you have to know there was a change, and knowing that
>> depends on what your doing.
>
> I think you're right - at least insofar as we're talking about
> per-instance versions. I think that for the per-class variations,
> having a generalised re-class-gen is acceptable, since the semantics
> there really are the same (it must happen *once*, on the first load of
> the Schema of that content type, and probably not again unless it's
> explicitly invalidated).
>
> Maybe we should consider having an adapter in AT for this stuff, with
> a default implementation that does what b-org does (generate for the
> __class__), and let things that need per-instance semantics override
> this? It just feels a bit wrong to me to not make it explicit that
> this stuff *is* needed. And further, if at some point in the future we
> manage to refactor AT so that it's no longer needed (the main reason
> it's needed now, I think, is that the security checks on fields rely
> on the method and its security assertions, but there may be others) we
> would be better insulated.
>
> An event that's fired on-schema-lookup may be another option .

our implementation viewpoints are very different, i don't want to modify  
global variables like classes at all, it breaks any local semantic be that  
instance or site.


>
> My three main concerns then are:
>
>  - How well does it handle the global per-class (possibly "global"
> only to a Plone site, via a local adapter) use case (i.e. the ITagger
> scenario above).

it does so by utilizing events, the global usage doesn't need to entail  
global implementation, esp. since you've stated as a use case local  
customization / installation to a plone site.

>
>  - Are we in the clear about performance of composing schemas and
> possibly re-generating methods. Again, an explicitly invalidated cache
> of (class, flavours-tuple) may make sense.

that cache works nicely w/ content flavors, and doing an implementation of  
regenerating methods on the class is broken if you want local behavior or  
customization without side affects.

>  - What is the best way of doing the delegation in AT to avoid the
> mixin, but still keep ContentFlavours as an optional dependency for
> third party products? In particular, if ContentFlavours ends up being
> dependent on an overrides.zcml to override the Schem adapter on
> IBaseObject, then it will be incompatible with any other generalised
> solution that does the same. Not necessarily a huge problem, but it
> would be nice to avoid it.

don't depend on additional policy beyond delegation would be best. the  
incompatiblity is always an issue with multiple framework implementations,  
esp. when their of the mythical variety.



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV