RE: The Harvesting process and the BFAV

Hi Daniel,

> Ideas to solve this or alternative explanations would be welcome.

Let me throw in a few thoughts on this issue. Speaking only for myself I
find the following issues interesting/relevant in this context:

* External reviews

A key problem with external reviews is that you have to know the area
involved very well to judge the wider implications of changes. It is not
surprising that typically a few people review certain areas since that's
their primary area of expertise. One of the key problems in the current
process is that these people need reviews themselves - even though they =
are
experts in the area and an external review is likely to just say "sure =
it
works - what did you expect"? Or even worse, there simply isn't anybody =
who
feels confident to review that code (who exactly even _could_ review =
Ned's
and my joint changes for adding/removing morphs?)

Here's an interesting observation about myself: I tend to _not_ look at =
code
from people I consider "experts" unless I am suspecting something
non-obvious or outright questionable. In all other cases my basic
inclination is "go ahead - if there's a problem we can fix that too" =
rather
than think ahead and try to foresee what might be affected (which =
doesn't
work anyway).

Personally, there are various people which I would immediately endorse =
to
submit fixes and enhancements in various areas. In other words, I trust
those people enough to be willing to live with their changes without =
having
to explicitly review them. And yes, things will break but they'd break =
with
or without a review by someone who doesn't really understand it. If so,
what's the point?

* External testing

If you look at the traffic on Squeak-dev, I think you'll find many more
responses that just say "thanks! that did it" than "[er, et]" or =
whatever
tags. The problem is that feedback from people using a fix is mostly
informal (if at all present). In a perfect world, I would want something
that I can flag as a fix for some problem. When some person installs the =
fix
the person should (later) get asked "did this solve your problem?" with =
a
choice of "don't know yet", "yes", "no", "yes but ..." (as in: it fixes =
the
problem but shows another one), and "no but ..." (as in: it doesn't but =
I
have package Foo and Bar loaded which might interfere).

One way to solicit this feedback would be to install in particular =
[FIX]es
in a way that the system _knows_ this is a fix and can ask the user =
about it
after that fix has been in use for a while. Again, here's an example:
Someone posts a fix for a problem. I load it and keep going. Some time =
later
(perhaps when I am about to save/quit the image) the system reminds me =
that
I have a bunch of fixes loaded and whether I'd want to comment on them. =
If I
choose to, I get the above mentioned options and this feedback is then
automatically sent to the list.

The key point about [et] tags for fixes is that people are individually
bitten by the bugs and that loosing feedback from a single person means
loosing _all_ of the feedback here. You don't run a [fix] right away =
unless
you're bitten by it.

Cheers,
  - Andreas

> -----Original Message-----
> From: [email protected]=20
> [mailto:[email protected]]=20
> On Behalf Of Daniel Vainsencher
> Sent: Thursday, October 16, 2003 5:57 PM
> To: [email protected]
> Cc: [email protected]; Brent Vukmer; Cees de Groot; Bert Freudenberg
> Subject: [Squeakfoundation]The Harvesting process and the BFAV
>=20
>=20
> Hi guys.
>=20
> The BFAV has become the technical basis for harvesting=20
> progress. In this
> it has improved the rate of harvesting by removing much of the dirty
> work that was involved in the old processes. It has improved the
> efficiency of the approvals many fold.
>=20
> It has also allowed us to elaborate the process by adding the quality
> tags, and for the first time having a more or less closed list of
> patches to be dealt with.
>=20
> The purpose of the QA tags was to distribute the work of harvesting
> among more people, by allowing casual participants to help by=20
> reviewing
> or testing code without accepting the responsibility of the=20
> formal role
> of harvester.=20
>=20
> As the efficiency of approvals went up, and we have an=20
> in-squeak list of
> posts and their full status, it has become much more visible that our
> bottleneck is not approvals. It is very clearly in the=20
> reviewing/testing
> stage. The reasons are very clear - rather few people help with the
> reviews/testing, and that quite sporadically.=20
>=20
> This bottleneck is a result of our current structure, and=20
> won't go away
> by itself. It causes a situation where most patches meant for=20
> the squeak
> image are ignored. I think this is not likely to last - either we
> balance this by removing the bottleneck, or contributors of=20
> patches will
> balance it by posting less, an outcome I find regrettable.
>=20
> In fact, I think it likely that people are already deciding=20
> not to post
> fixes they have available because they're likely to be ignored.
> Therefore, I think that if we can remove the bottleneck, and reach a
> state where patches submitted are all processed in reasonable time, we
> will get far more patches, and be able to make much better=20
> progress than
> we are now.=20
>=20
> I think the following steps are in order -
> 1. We want to lower the barriers to casual participation. So,=20
> the BFAV,
> as a prerequisite for participating, should never be absent=20
> for anyone.
> It should be included in the base image.
> 2. We should improve and simplify the tools to help=20
> reviewers/testers do
> their job.=20
> - ClassBoxes, when they're ready, could become a big aid to=20
> testing, by
> allowing patches to be installed temporarily and then completely
> removed.
> - The BFAV should be improved, especially for casual users. Brent, I
> recommend to open its development so more people can help=20
> with this. The
> Swiki page is already pretty good, just point to it from the=20
> help in the
> BFAV. Also it might be good that its future development be carried on
> using Monticello, to make it easier to create and integrate
> improvements.=20
> 3. The current flood of BFAV traffic makes people automatically ignore
> the harvesting process. Therefore it MUST be selectively=20
> moved away from
> squeak-dev. Cees, or whoever has administrative rights on=20
> lists.sqf.org,
> can you please create a new mailing list called "squeak-harvest"? This
> fits Berts existing filters. Having this list would allow some
> selectivity without losing information.
> 4. When all is said and done, many people currently won't=20
> participate no
> matter how low the barriers are made, because of different priorities.
> Some of this is obviously reasonable, people need to pay the rent and
> play with whats fun for them, of course. But some of this is IMO not
> reasonable, and comes from a misperception. I think the NewLook code
> showcases this very clearly. The voices for its inclusion are=20
> very loud,
> but nobody bothers to review it, therefore, it isn't included. So I
> guess that the people doing the requesting somehow fail to=20
> perceive that
> the condition for inclusion is within their grasp. Of course, I could
> just be wrong about this. Ideas to solve this or alternative
> explanations would be welcome.
>=20
> Daniel
> _______________________________________________
> Squeakfoundation mailing list
> [email protected]
> http://lists.squeakfoundation.org/listinfo/squeakfoundation
>=20