Re: Advice wanted on dependency building...

On Wed, May 22, 2013 at 11:53 PM, Ronald Oussoren
<[email protected]> wrote:
>> I'm using the
>> system zlib -- is that a bad idea? Should I build it too, to make sure
>> it matches the rest of it?
>>
>> (I do want the binaries to run anywhere the binary Python I'm using runs)
>
> It depends on the library.

OK -- it would be nice to have rule to follow, but I guess that
decision should be made for each particular lib.

> I agree w.r.t. homebrew and macports, but it would be nice if 'pip install' would work with your system with minimal changes to the pip configuration (e.g. "just add ... to your piprc and then 'pip install foo' will install a binary from the repo instead of building the binaries itself").

yes -- it sure would -- though this is enough to do without trying to
hack on pip as well....

>> Yeah, and he never gave anyone else permission to push to it...
>
> I wouldn't have done that either until the someone else has a proven trackrecord (both in providing usable binaries and in being known in the community).

well, when I say anyone else, I mean anyone else -- I, and a few
others regularly contributed, but it involved sending it to him, and
hoping he'd find the time to put it up...So if I do this, I'd like to
have at least the option of a handful of folks contributing directly.

>> But if we put the shared libs in amore central location, then all your
>> virtual-ens could use the same ones, yes?
>
> Yes. It would make it harder to switch library versions, but not by much.

hmm -- this is getting tricky for me to wrap my head around -- could
we make it so a pip install inside a virtual env would install a
dependency in the main python? Or would you have people install the
libs outside of the virtualenv first, if they didn't want multiple
copies ?

> Uninstall can be a problem with that, you'd have to refcount installed files to ensure that libraries are only removed when the last user is uninstalled. I don't know if the installation format used by pip supports having two packages that install the same file.

and I don't want to write that code anyway ;-)

> This can be worked around with fake PyPI packages that only install the shared libraries and have the real packages depend on that (that is a "macbins-libpng" package with libpng.dylib and have the Imaging package depend on that).

I like that idea -- and It looks like that's how Anaconda deals with it.

> /Library can be used, we'd just have to pick a name that Apple is unlikely to use.

I think it should go in /Library/Frameworks/Python somewhere, helps
the uninstall issue -- if folks clear that out, they won't have
anything left behind.

> I'm probably atypical, but my main account doesn't have admin privileges. It would suck if I'd have to use sudo to install.

How do you install the python from Python.org? I"m just thinking we
should match that...

> The @loader_path option you mentioned in a followup e-mail could help there. That way the shared libraries can be installed in a fixed location relative to sys.prefix, while still supporting virtualenvs. You wouldn't be able to share shared libraries between python versions or virtualenvs, but that's not really a problem (disk space is cheap).

That may be the way to go.


>> Any idea what the time scale is on this?
>
> Before Python 3.4 is out, which means sometime this summer.

OK -- I figure I"ll wait until it's there, and then try it out.


>> Have the pip folks made any commitment at all to supporting binary
>> installs? That's a big missing feature.
>
> Yes, through wheels. The development branch in pips repo (<https://github.com/pypa/pip/tree/develop/pip>) contains support for wheels (both creating and installing), although AFAIK installation of pips requires a command-line argument at the moment because wheel support is experimental at this point.

fair enough -- have you looked into the universal binary issue at all?
easy_install was always getting confused by universal binaries...

> I'll provide mental support at the least, and hope to do more than that but don't know if I can do that time-wise.

You've provided an enormous amount already!

> If wx is hard to package it would be a good stress test of the tools, even if you'd end up not distributing the binaries :-)

yup -- but I'm still not sure if I want to deal with it! -- we'll see.
Maybe Robin would be interested in supporting this shared lib system
if we do get to that.

I'm thinking of setting up a gitHub project for this..I'll let you all
know if/when I do.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[email protected]
_______________________________________________
Pythonmac-SIG maillist  -  [email protected]
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG