Re: Schema-by-interface options

On Sat, 23 Sep 2006 05:54:39 -0700, Martin Aspeli <[email protected]> wrote:

> Hi Kapil,

<snip>

>
>> if your going to put schema delegation directly in AT, please do make  
>> that delegation, 100% policy free and make adapters implement policy.
>
> +1
>

cool.

>>> Could you elaborate a bit more on how the "global" flavour thing may
>>> look? For example, imagine a scenario where you have a "global"
>>> 'tagging' flavour that adds a 'tag' field to any object marked with
>>> ITaggable.
>>>
>>> Let's say that you then *also* want to add a specific 'article'
>>> flavour or whatever.
>>>
>>> How would this look?
>>  you would setup a local event adapter within a site folder for content  
>> created, and apply flavors to the content on whatever basis ( in this  
>> case implementing ITaggable).
>
> That feels a bit strange, but I suppose it makes sense. The flavour's  
> schema is still on-disk, just keyed by name (right?) so it's not like  
> you're stuffing the schema of each instance in an event. I think I like  
> it. :)

yes, flavors are on disk keyed by name.  created events would be  
subscribed by a handler adapting the content to flavorprovider, and  
setting applicable site wide flavors, although its a bit of science  
fiction till it gets ported to zope 2.10 and has access to the new  
persistent registry stuff.


>
>> i've been doing some more thinking about the global scenario, with  
>> global implementations. part of the difficulty of using a global  
>> implemetation are the opposing use cases, ie. applying global behavior  
>> local to a site.
>>  global behavior implementation based on class manipulation is a  
>> radically different notion and is inherently global in nature.  
>> additionally archetypes content have way too many responsibilities as  
>> regards schema, which makes such monkey patching both fragile and  
>> complex to support class weaving imho.
>
> I'm not sure it has to be done with monkey-patching as such. My solution  
> basically looked like this:
>
>   BaseObject's Schema() does: schema = ISchemaComposer(self)()
>
>   the SchemaComposer's __call__ method would look up adapters to  
> ISchemaProvider - which could come from local or global site managers
>
>   each ISchemaProvider would provide a part of a Schema object (still  
> not attached to a class or instance)
>
>   the final schema would be cached (keyed on __class__ in the current  
> implementation, though I realise that may be too restrictive) and then  
> returned
>
>   subsequent lookups would use the cache until invalidated via an event
>
>   you need to run class-gen to build out methods (on the class, in my  
> case) for AT to work - the schema composer took care of this.
>
> I wouldn't call this monkey patching. All it does is defer the decision  
> of what Schema() should return until run-time (i.e. the first time it's  
> called/after the cache is invalidated).
>
> However, I'm agreeing with you that the subscriber-based version will  
> work with the kind of decoupling/indirection that ContentFlavors favours  
> (sorry), and that the above solution makes per-instance much more  
> difficult unnecessarily.
>

the above solution is broken for the use cases you've presented, is what  
i've been try relate. you want to use possibly local settings to modify  
globals. ie. as soon as you have site specific adapters in place, let's  
say two different schemas in place on sites with overlapping fields, they  
both dump possibly incompatibly on the class, and you have broken  
behavior. classgen on the class IS monkey patching a global variable. the  
only reason it works in at, is because its by convention done only once,  
but its not reliable when you have different schemas trying to do it at  
runtime on the same class.


>> the sort of implicit global monkeying of arbitrary classes seems much  
>> more error prone and which in imho is the opposite of what makes a good  
>> framework.
>>  also, notions of expected ordering application of schemas are also  
>> harder on a purely global basis, as they require awareness of  
>> configuration space ordering.
>
> Yes. My solution had an 'order' attribute that specifies an int, and  
> it's sorted by this. klass.schema was 0, and other could use numbers as  
> they wished. But I never liked this. Note that this is how menus work in  
> Zope 3...

i think its just as much of an issue in zope3 as much as it is here,  
needing awareness of configuration space ordering, esp. when ordering has  
semantic importance to the runtime is really not fun. maybe its okay for  
z3, where its meant to be an app server not a product, but requiring plone  
end users/admins to deal with it is just a recipe for frustration.

>
>> as before, i think the easiest ability to support global AT behavior  
>> with current local flavors implementation is via an event listener.
>
> The only negative I can think of here, really, is that if you uninstall  
> the thing that provided the "global" (site-local) schema, you need to  
> find all the content objects that have it attached. You can build some  
> safety into that, of course - if the flavour can't be found, don't bomb.  
> But it still seems a bit more tricky.

this safety net, is already done in content flavors during runtime  
composition, with logging, and the content cleaned up when flavors are  
edited.


<snip bad z3ish pseduocode>

>  >
>> to support the global case w/ a global implementation  cleanly (without  
>> side effects) you have to delegate to some local notion but, making  
>> behavior local while not affecting side effect global changes is hard,  
>> you get to use a local adapter registry sure.. but you can't avoid the  
>> side effects of global modification when doing class modification. to  
>> get around global modifications you'd need, a new primitive is needed  
>> to tie local configuration to global.
>
> Yes. In my solution above, that'd mean that the schema cache would have  
> to be in a local utility. I think it'd work otherwise, though - since  
> when the cache was invalidated, all the adapters it would find would be  
> local ones (and possibly some global ones).
>
>> one solution, you could have a class provides which queried for a  
>> provider utility, though it also creates dependencies on traversal for  
>> site discovery. though testing frameworks that utilize would be  
>> straightforward via implementations of the applied interfaces, the  
>> runtime integration does bring up the importance of functional testing.
>>  building on that primitive a marker utility for classes to support the  
>> runtime persistent global case, ie. enabled via ui, in which case your  
>> primary concern is persistability of declarations in a utility  
>> implementation.

<snip code>

> Now you've lost me - not sure what you're trying to achieve/demonstrate  
> here...

i was trying to demonstrate a new primitive to utilize z3  
interfaces/schema specifications that delegated to local components for a  
z3 solution that allowed for global application of local behavior without  
per instance modifications and without stomping on a global variable.  
still not sure if its feasible, but perhaps worth exploring.

<snip>

>>  the new cache scheme uses a two layer cache, global by class,  
>> composition combination and instance, it saves memory and never needs  
>> extraneous composition calls.
>
> I trust you to get things like caches right. :)
>

i added some caches stats in debug mode into the user interface for  
verification ;-), the global cache only gets one miss per ordered flavor  
set schema, the local cache with weakrefs, approaches 98% cache hit  
success.

>>> Also, I assume you generate methods on the instance and not on the  
>>> __class__?
>>  of course, mucking with a global like class multiply at runtime is way  
>> too error prone and a horrible framework semantic.
>
> Yes. As I said, the per-instance use case was never part of my original  
> thinking. Agree with you that it's undesirable. I'd rather not have to  
> generate methods at all, of course, but then we'd have to fix all sorts  
> of things in AT (I think). I think it's an anti-pattern for things that  
> use Flavors to rely on getMyField in any case. They should use  
> Schema().getField('myField'). And of course, that should all be hidden  
> behind adapters.
>

most of AT uses the field.getAccessor( instance ) semantic, which is fine,  
but yes its anti-pattern for composition and mutable schemas to spell  
accessor names directly on instances, since their all volatile (_v_  
prefixed), but code doing that would be broken anyways if a flavor was  
removed. experience with ATSENG has shown its not an issue in practice.  
ie. context.Schema['rabbit_soup']

>>>> incidentally,
>>>> contentflavors already does instance cache invalidation via event,  
>>>> though
>>>> moving to the cache keys i proposed previously, you never need to
>>>> invalidate a composed schema, just signal accessor/mutator gen on  
>>>> instance.
>>>
>>> Why is it that you'd never need to invalidate?
>>  i treat schemas as immutable post startup. as you mentioned before,  
>> persistently storing schemas leads to all kinds of issues, once you get  
>> away from that notion, you can compose schemas and save composition,  
>> because for its definition combination its always valid. you can  
>> utilize have mutable schemas if mutation triggers events that can be  
>> utilized for cache invalidation.
>
> Restarts are fine, I think. I've never been a fan of TTW schema editing.  
> :)
>
>>> An event that's fired on-schema-lookup may be another option .
>>  our implementation viewpoints are very different, i don't want to  
>> modify global variables like classes at all, it breaks any local  
>> semantic be that instance or site.
>
> They're converging - I think you're absolutely right.
>

cool.

<snip>
>
> So, specifically, this is what I'd like to do:
>
>   1. Merge current trunk into 1.5 whitmo-schema-by-iface so that it's  
> 1.5 compatible. This contains more fun with interfaces and so on that  
> helps all of this.
>
>   2. This uses schema = ISchema(self). The adapter factory here is a  
> class that does things that scares me - like 'self =  
> ImplicitAquisitonWrapper(schema)' in __init__(). Yipes. :) We can do it  
> better (imho) by using a function as a custom adapter factory - this is  
> trivial.

could you give a link?

>
>   3. I also think the aq-wrapping in Schema() should stay in the  
> Schema() method, so that it does:
>
>    def Schema(self):
>        schema = ISchema(self)
>        return ImplicitAcquisitonWrapper(schema)
>
>
> That way, at least we're not requiring all ISchema adapters to remember  
> to aq-wrap (which may even lead to some funny semantics if the ISchema  
> adapter is used elsewhere).
>

sounds good.

>   4. We slap a big, scary note on the ISchema interface that people need  
> to worry about generating methods, and caches.

and rabbits with big pointy teeth ;-)

>
>   5. I update VariableSchemaSupport to get the cleaner VarClassGen  
> that's on my branch atm. I don't propose we really use this, but in the  
> case where people *do* want their own ISchema adapter for a specific  
> class, they will be able to use it. Lots of caveats need to go on it, of  
> course, but since the code is already there I see no problem putting it  
> in for those who do want it.

sure.

>
>   6. We let ContentFlavors mark BaseObject with IFlavorAware or  
> something like that, and register our own ISchema adapter that has  
> ContentFlavors semantics. That way, if someone wants to play with the  
> marker interfaces to get different semantics again, they can, but by  
> default, ContentFlavors is applicable to everything that wants it.
>

sounds good.

> Just to be sure:
>
>   - Do you see any problems (performance or otherwise) with having a  
> ContentFlavors version of Schema() (via the ISchema adapter) in the  
> cases where it's not actually used?

no issues, its fine.

>
> Oh - and can we eventually move this to AT svn?

yes.

cheers,

-kapil

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV