Re: Pypi cdn for hosted packages

You me talk - free things are afootsie 


On Thursday, February 28, 2013 at 10:13 AM, Noah Kantrowitz wrote:

> Reponding from my phone quickly before this gets any further, will write more later. Plan is to have pypi move package download links to a new hostname (probably pypi-download.python.org (http://pypi-download.python.org)) and then throw that behind fastly. This sidesteps 100% of issues with dynamic pages, etc. Simple index with be handled secondarily.
> 
> Jim Fulton <[email protected] (mailto:[email protected])> wrote:
> > On Thu, Feb 28, 2013 at 5:39 AM, Jesse Noller <[email protected] (mailto:[email protected])> wrote:
> > > Thread fork.
> > > 
> > > Anyway. I know we have at least 1 major rep of a cloud provider on the list, and I have at least one off in my pocket.
> > > 
> > > I'd like to start discussing (completely ignoring past efforts and discussion which got bogged down) how we can start distributing the package data we host via CDN rather than the mirroring system.
> > Woohoo!!!
> > 
> > One issue is how to keep the relatively dynamic stuff up to date.
> > One option is to bypass the CDN for dynamic content, but a better
> > approach might be to use CDN-provided invalidation APIs.
> > 
> > I'm mainly thinking of the simple index pages, which are relatively
> > static, but, wh en they change, you want to get the change quickly.
> > Ideally, when there's an update to package, PyPI would
> > invalidate the index page. Everything should have a really long
> > cache interval.
> > 
> > BTW, we use CloudFront fairly extensively for media. We get a lot
> > more origin requests than we expected, given the cache intervals
> > we'd set. (It was still a big win for us.)
> > 
> > Jim