Notes for today's CI meeting

Hi all,

I think I will make it to the meeting, but unfortunately at the same time
of the meeting I'm putting my kid to the bed, so in case I miss it.

My ideas/suggestion to improve the current situation:
1. integrate more closely jenkins and github
2. make our setup as easy to reproduce locally as possible
3. keep our PR as clean as possible
4. use Travis/other free CI to run code analysis and tests for the single
package
5. have a tool (huboard.com/waffle.io or the like) to coordinate which PR
are still pending to be merged and such
6. do not allow anyone (but eric?) to push directly to master and instead
have that handled by jenkins or a review tool (gerrit or such)
7. our target should be to replace eric (no offense) by Jenkins
8. use the Plone workflow also on your companies

More in detail:

1 Jenkins <-> Github

If we are able to do a Travis-like answer to pull requests so that we know
if a pull request is going to break, that will help the one that send the
pull request to evaluate what went wrong on jenkins ... and jump to 2. point

(and yes, dependencies on PR and such can be dealt with some consistent
guidelines and if we stick to some rules)


2. easy setup

If everyone can configure their servers/laptops as close as possible to how
jenkins runs tests, you don't even need a Travis-like functionality, you
can run that on your own!

(of course, test do take a lot of time, so help on that would be greatly
appreciated)


3. clean PRs

if our PR are just what the subject says and not a combination of that and
cleanups and whatnot it would be more and more easier to review and to spot
where the problems come, just for the sake of an example (sorry mauro if
you are reading this):
https://github.com/plone/plone.app.stagingbehavior/pull/10

a PR to remove grok from p.a.stagingbehavior which actually also:
- updates bootstrap.py
- adds a buildout test cfg
- renames files from txt to rst
- updates gitignore
- bumps setup.py version
- cleans unused imports
- merges the branch into itself(*)

So out of 7 commits just 1 (actually 3) are about removing grok.

Another example (sorry zopyx here), compare these two pull requests:

https://github.com/plone/plone.api/pull/179
https://github.com/plone/plone.api/pull/205

Actually its exactly the same code, one within 15 commits, the other within
3, which makes things more self contained and straight to the point.

* I've seen that pattern quite a few times here at Plone, please don't do
so, rebasing a pull request (IMHO what should be always done before
merging) is way way better than keep merging the PR into master, of course
keeping the branch to the point and not doing superfluous stuff (see above)
makes that waaaay more easier.


4. Use Travis for code analysis and run package tests

For the lazy ones not running tests on the code you will push as a PR, this
can help a lot to already spot errors or code conventions being broken.
Still that's not a 100% guarantee that if Travis says OK jenkins.plone.org
will say the same (refer to 2. for that)


5. overview tool to coordinate

Having an overview on what's pending to be merged and such can be extremely
helpful for the ones that do care the most about CI. This way, and before
we can automate that, we could create jenkins jobs only meant to try pull
requests and have a clear overview of how much work (CI related is needed),
that would bring not only lots of possibilities to the testing team,  but
also to the FWT, release team, etc etc


6. a review tool to orchestrate jenkins

Openstack or LibreOffice (just to name two examples) do not allow anyone to
merge directly, you send a pull request to gerrit (their review tool of
choice) and if tests passes and someone approves it and such (you can
define your own workflows) code gets merged, so in really rare cases master
is broken.


7. replace eric!

Because at the end what we all care about is that plone gets better, nice,
more functional, easy to use, etc etc, so we want our code to be integrated
with as less hassle as possible and play nice with others.

So if we are able to never break master (see 6.) and we improve enough our
CI setup, we could get to a position that every single commit (see 3.)
could end up being the next 5.0 release without any more effort than
pushing a commit with a well-known keyword on it...


8. use it on your own companies

Or "eat your own dog food", i.e. if all of us use the same approach (to a
bigger or smaller extend) in our own companies, we will not only understand
better how things work and why the decisions are like that, but at the same
time we all will contribute to the tooling and the chain that makes all of
this possible.

The closer we work as upstream works the easier will be to contribute,
realize things, and at the same time, fixes on your own tooling will also
help on fixing upstream's tooling.



And as a closing:
The bottom line here is that we all want to contribute all our brilliant
ideas, but if doing so prevents others from doing the same (i.e. breaking
the build and then not caring to fix it, not knowing how to proceed) makes
contributors frustrated and keeps them away/shy/scared of the process we
have in place.

The more emphasis we put on not only sticking to the rules, but also on
making our tooling (python, buildout, p.a.testing, git, jenkins...) better
and more fit for our purpose the more we will be able to keep contributing
without being worried about making changes that break things without
knowing it and/or not having the knowledge to fix the situation and go back
to where we were.

I have plenty more ideas on how to improve things, I hope to have
highlighted the most important ones.


Have a nice and productive meeting (hopefully will be there!)
Gil

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/

_______________________________________________
Plone-developers mailing list
Plone-developers-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/plone-developers