Re: RFC: Known working sets

On 4 Sep 2007, at 01:21 , Tres Seaver wrote:
> Philipp von Weitershausen wrote:
>> Wichert Akkerman wrote:
>>>> The only problem is that distributing grok-0.11.cfg is a bit  
>>>> tedious.
>>>> How about if buildout could get it from the web?
>>> How about making it available from an egg, through a hook in egg- 
>>> info
>>> perhaps?
>>
>> This is a very good point. So good in fact that I thought of it  
>> myself
>> :) I was already writing the email when your message came in.
>>
>>
>> Martijn and I discussed the known working set problem over IRC this
>> afternoon. He brought up a few good points which suggest that the
>> version data could be associated with the egg:
>>
>> The approach that I described in my original posting (and which  
>> actually
>> works *today*, as I found out!) has some disadvantages. For instance,
>> the discoverability and release mechanism of the known working set
>> configuration file differs a lot from our normal packages.  
>> Releasing a
>> package is a well-known routine by now. How and where would we  
>> release
>> the working set configuration file? SVN?
>
> Why not just release a meta-egg with hard-wired dependencies, plus the
> scripts required to set up and run the application?  If the other
> components are as relaxed as possible about their dependencies,  
> then it
> shouldn't matter that the meta-egg nails them down:  it isn't going to
> be possilbe to re-use the meta-egg anyway (and I for one wouldn't want
> to -- I'd be creating a new environment to run it in).

I think the use case of being able to override a particular version  
of a package is valid. Let's say I'm on Zope 3.5.0 (meta egg or  
whatever) and I'd like to get the newest ZODB because it has a really  
cool feature. Sure, the warranty is void now, but I'd still like to  
be able to do it *without* having to reproduce all the version  
dependencies that the meta egg specifies.

>> Another problem are dependencies and how we'd like to depend on other
>> working sets. Let's say we made a 'Zope' working set that  
>> contained the
>> stable versions of the zope.* packages. It would make sense for  
>> grok to
>> depend on this information. And packages using grok should depend on
>> that. It'll be complicated to maintain such a chain in separate text
>> files, especially across version updates. It only seems natural to  
>> use
>> the egg dependency mechanism for this.
>
> Depending on another WS would basically Just Work (TM) if we used egg
> dependencies.

It would, but setuptools doesn't allow us to specify "overrides" in  
case of version conflicts. Perhaps zc.buildout could be taught how.  
Then I suppose I could settle for using '==' in a meta egg's setup.py.

>> When EGG-INFO is written, a custom writer will then take this
>> information and generate 'known_working_versions.txt' or whatever in
>> EGG-INFO. The only problem is that this custom writer needs to be
>> installed when setup.py is called to create egg, in other words to
>> generate the right kind of eggs you'd need to have something  
>> installed
>> first. Not sure if this is better than maintaining .egg-info  
>> directories
>> in SVN...
>
> I guess I don't get the usecase for maintaining "known good"
> dependencies outside of setup.py (for the "meta" egg).

I don't mind maintaining them in setup.py, but the "install_requires"  
mechanism is insufficient. Sooner or later you're going to end up  
with a version conflict.