Re: jenkins and git repository monitoring

On 2/3/12 2:07 PM, William Deegan wrote:
> All,
> On Nov 20, 2011, at 7:59 AM, Patrick Gerken wrote:
>
>> On Sun, Nov 20, 2011 at 15:51, Timo Stollenwerk<[email protected]>  wrote:
>>> Hi Patrick,
>>>
>>> I'm not sure if polling github is really what we want for CI (the
>>> ScriptTrigger plugin does poll after all if I understand it correctly).
>>> Polling works fine for smaller projects, but I don't think it does
>>> really scale for a project like Plone. With a large number of jobs
>>> polling this is a waste of network resources and we don't really get
>>> atomic commits (which is not really CI then).
>> Yes, it's really slow, right now it takes 20 minutes. We are polling
>> github and svn right now, and the git plugin is too stupid to work
>> with multiple checkouts reliably, and believes every time something
>> has changed.
>>
>> My idea would be a small improvement of the current situation, while
>> yours below certainly is better, but it needs to be done.
>> I'd love to join the efforts, but right now I can't even keep up with
>> mail traffic :-(
> Was wondering what's the current status on this.
> Apparently github has a API which would allow us to walk all the plone repos and set the hook programmatically
> http://developer.github.com/v3/repos/hooks/
>
> Would that resolve some of these issues?
> Likewise would a commit hook on the plone SVN repo be possible?
> So no more polling, just responding to posts?
>
>
The plone repositories currently have 2 hooks set up:
1. CIA (for the IRC bot)
2. A custom one that pings a Pyramid webservice I've set up on my 
personal server. This webservice creates a diff for each commit pushed 
and emails it to the plone-cvs mailing list so that we can have full 
diffs. Code at https://github.com/davisagli/collective.githubmailer

The githubmailer thing could probably be modified to also ping jenkins.

I have a script which uses the API you mentioned to make sure all the 
repositories have these hooks set up. See attachment.

cheers,
David


----------		
David Glick
 Web Developer
 [email protected]
 206.286.1235x32

Engagement technology for social and environmental change.

http://www.groundwire.org

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

_______________________________________________
Plone-developers mailing list
Plone-developers-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/plone-developers

import sys
import json
import requests

requests.settings.verbose = sys.stderr

GH_URL = 'https://api.github.com'

# FILL THESE IN:
username = ''
password = ''

with requests.session(auth=(username, password)) as s:

    repos = list([r['name'] for r in json.loads(s.get(GH_URL+'/orgs/plone/repos').content)])
    for r in repos:

        hooks = json.loads(s.get(GH_URL+'/repos/plone/%s/hooks' % r).content)

        if not any(h['name'] == 'web' and h['config']['url'] == 'http://wglick.org/githubevent/' for h in hooks):
            req = {
                'name': 'web',
                'active': True,
                'config': {
                    'url': 'http://wglick.org/githubevent/',
                }
            }
            s.post(GH_URL+'/repos/plone/%s/hooks' % r, data=json.dumps(req))
        
        if not any(h['name'] == 'cia' for h in hooks):
            req = {
                'name': 'cia',
                'active': True,
                'config': {
                    'project': 'plone',
                    'branch': '%s',
                }
            }
            s.post(GH_URL+'/repos/plone/%s/hooks' % r, data=json.dumps(req))

        for h in hooks:
            if h['name'] == 'email' and h['config']['address'] == '[email protected]':
                s.delete(GH_URL+'/repos/plone/%s/hooks/%s' % (r, h['id']))