Re: Re: FYI: Lessons from Kai Wu at ArsDigita

On Thu, 13 Feb 2003, Paul Everitt wrote:

> Note: the point here is risk management.  This transition is exciting,=20
> but it's also dangerous.  Thus, we must be brutally honest, even=20
> paranoid, about learning from the examples in the industry.  Simply=20
> finding a few differences related to our situation, and then throwing=20
> out the similarities -- that's not risk management, it's wishful=20
> thinking.

I by no mean mean to dismiss the prospect of risk in the transition -
but i _do_ think, from reading that account, that many of the risks
peculiar to ArsDigita's experience with ACS transitions are different
than the risk that we are facing with Zope 2/Zope 3.  Further, i think
we're facing some risks that are different than the common ones.  I'll
say more about this all.

Incidentally, in the course of answering you and in seeing your
responses i'm getting a clearer picture of what you're (ultimately,
"we're") trying to do - to eke out and discuss the risks we're facing,
by example and contrast - and agree that's a good idea.  However, i
see many of the details particular to ArsDigita's situation not
applying to us - which can obscure the lessons we learn from them, and
from the general considerations for any community facing the prospect
of such a transition.

> On mardi, f=E9v 11, 2003, at 10:20 Europe/Paris, Paul Everitt wrote:
>=20
> > From Kai Wu
> >
> > 	Ah, well (grimace) there wasn't a clear plan for transitions -
> > we built ACS 4 Tcl anew, while ACS 5 Java, of course, shared
> > little besides the data model with ACS 4 Tcl.  If you started on
>=20
> Us too.  ACS 5 had a new data model, Zope 3 has a new API and new=20
> component model.

I see their transition as being much more drastic, for two reasons:

 - They were changing their underlying implementation language, which
   is a monumental kind of change, and presumably reduces not just the
   amount of code but also interferes with the strategies and
   techniques that transfer.  We are not changing the underlying
   implementation language, and i believe that a lot of the
   accumulated wisdom does transfer pretty closely.  Zope 2 may have
   different code than Zope 3 for an event service and event
   listeners, or full text indexing, but they'll be doing the same
   thing in both versions - and may even entail the same code, in some
   part.

 - They were compounding a recent big change (from ACS 3 to ACS 4)
   before the dust had even settled.  This prospect would terrify me,
   because the strategies and techniques for ACS 4 that might transfer
   could not have had time to shake-out and settle - the developers
   could not have had time to see the consequences of their decisions
   for the ACS 4 changes play out.

In general, i think Zope 2 is sufficiently mature, and our perspective
on it sufficiently substantial, that we *do* have a good basis for
gauging a refactoring - that we are in a good position to formulate
and gauge the refactoring for Zope 3.  I think that's extremely
different from/extremely less disorienting than a marketing-based
decision to go with Java.

Maybe that's an exaggeration of the ArsDigita situation, but according
to the report i don't think it's completely off base.  It doesn't
dismiss the risk in making our transition, but it suggests that we
will not hit some of the significant pitfalls that they did.

> > a given system (ACS 3, ACS 4, or ACS 5 Java) you usually stayed
> > on it - translating software plumbing is a nauseating task for
> > smart programmers, nor is it an appealing expense for clients.
>=20
> Based on interactions in the last few weeks, I also worry about the=20
> "nauseating task" part of ensuring a smooth transition.  But presumably=
=20
> this falls under the official promise, meaning it will get official=20
> manpower to fulfill the promise.

This is the kind of thing that any company - particularly a small
company - must deal with when progressing through major software
revisions.  As i indicated in my last message, i happen to think we
have some particular relief in the things that transfer, due to Zope
3's changes originating in the CMF and the services it offers.

In the larger picture, my experience as a Zope employee and user has
often been one of ongoing change.  I don't think Zope 3 is going to be
particularly out of place, in that respect - and am hopeful that it
will be to a more regular framework, and hence a move to where further
changes are less disruptive as the flow continues.

> > 	Zip back to Summer 2000; ArsDigita had a handful of our sharpest
> > developers coding ACS 4 Tcl (note: we did not have a QA team, or
> > dedicated doc people). We pre-sold new customers on ACS 4 Tcl for
> > a while.  We told old customers (on ACS 3 Tcl) we'd take care of
> > 'em - meaning, a developer or two on our end was essentially
>=20
> I presume this will apply to any existing Zope company.  While this is=20
> feasible, supporting two very different platforms has a cost.  Do you=20
> train some of your guys in supporting Zope 2 and some on Zope 3?  Do=20
> you train all of them on both?

Again, i see this as a general issue for this process.  It's a
resource balancing thing, and i see our particular needs as a little
different than the typical ones, including those of ArsDigita.  More
about that below.

> > stuck in support mode, tied to a legacy client.  At that point we
> > had a separate engineering team forging ahead with ACS 4 Tcl
> > pretty much freed of any obligation to do backwards compatibility
>=20
> That's the story for us too.  Elaborating the migration issues happens=20
> some point in the future.
>
> > with ACS 3; the changes were quite radical.  (We actually tried
>=20
> Changes are quite radical for us.

... but, at least, without pre-selling on the new platform,
accellerated and multiple transitions, and the drastic churn of an
implementation language change.  Phew!-)

> [... i'm cutting out some of the general issues which any transiting co=
mpany
> must face ...]

> > were pretty much abandoning it within a few months.  Clients just
> > started on ACS 4 Tcl were confused and worried.  We had at end of
>=20
> We're likely to have some of this as well.  Some people will accept a=20
> vendor statement that says everything will be ok.  Others will remember=
=20
> how vendors have said that in the past, and things didn't work out well.

Yikes.  Are you saying that we're being sudden in the change and
cryptic (at best) in our intentions, so that we're confusing the
clients?  If that's the way it appears from the outside, than maybe we
do need a serious sanity check!  I just find it hard to believe that
it's very comparable, what with the length of the process and the
transparency entailed in the community involvment, plus the zope.com
roadmap statement and the like.

Seriously, considering the proportion of Zope 3 work that is occurring
in the community - more than is happening inside Zope Corp! - and the
fact that the rest is happening in public view - the situation seems
quite different to me than vendor statements about their commitments
as they work furiously behind closed doors.

> > 2000 almost nothing working in Java, just orders to get it done
> > ASAP.  As it turned out, 5-6 months later in summer 2001, the
> > first client projects were tentatively deployed on ACS 5 Java.
>=20
> We won't have too much of this, as long as the message is sent: don't=20
> use Zope 3 in production until (some time in 2004).
>=20
> I worry because I hear some of the core developers saying they want to=20
> start using it first half of the year.  If that happens, we'll have a=20
> different ballgame (IMO).  We'll start to lose flexibility.

I think it's, again, quite different if it's the developers making
that choice, with good information about the options, and not the
vendor making it for them, and in a rush.  !

> > 	Back to the past: Of course, our open-source external community
> > (see openacs.org) was up in arms about all these changes; first
> > they were asked to adopt the ACS 3 Tcl orphan.  Then the ACS 4
> > Tcl newborn came along.  It took them a long time to translate
> > and begin improving ACS 4 Tcl (they use Postgres, we used
> > Oracle) - in some ways, that learning curve sucked up such
> > critical amounts of time and energy that their whole community
> > suffered; growth in ACS adopters slowed greatly.  I don't have
> > numbers, but anyone who visited the openacs.org site during 2001
> > would, with a little investigation, find lots of FUD around
> > ArsDigita, the ACS, and the community - enough to say, "Screw it,
> > I'll go with PHP" (small budget) or "Screw it, I'll go with J2EE"
>=20
> Us too.  We need to understand and appreciate that others, out in the=20
> non-Zope universe, are NOT going to have such a sublime assessment of=20
> our transition.  For business people, the specter of Zope 3 will=20
> probably be as much a hindrance as a help in competing for contracts in=
=20
> 2003.

Even here, i note that the community is directly involved in the
_frontier_ development, not trying to catch up to a mad rush by the
corporate team.  I think that community involvement, and the
transparency necessary to conduct it, should help diseminate
understanding about what's really going on, and participating in
forums (like this!) to field concerns and address them, rather than
having them fester.

> > 	Always have a few good technical scouts providing
> > well-considered feedback as to long-term product and marketplace
>=20
> I don't really think we're doing too much of this.  Then again, that's=20
> probably out of scope.

It's funny - i think this effort is specifically a response to very
clear signals, both internal and from the community, that a good
refactoring is warranted and due.  I think the principle is that,
being open source and a small company, we need to cater to the
developers, make their jobs easier in order to enable us all to help
Zope grow - be used by more developers to do more things.  If that
can, in fact, be considered our marketplace, than i think we're
listening, and responding well.

> > 	And always, always have something to show people (users,
> > developers, business people, the press) that works today, is
> > polished (good installation, ample docs, high quality, and
> > completed functionality with slick UI), and is clearly being
> > supported (lists, IRC, forums, newsletters with buzz, and regular
> > module/Product releases) - that gets you the gold of credibility.
> > Perhaps today's pace is just a bit slower than the boom time, but
> > I think even a period of 6 months where you're promising The Next
> > Great Thing while reluctantly showing your old software with
>=20
> Let's face it, we're stuck pretty hard in this one.  We talk up Zope 3=20
> as being worthy of the energy of a total rewrite.  You only rewrite=20
> when the old thing was unfixable, right?  "Shuffling feet" is fairly=20
> accurate.

I thought our credibility is sustained with continuing improvement and
_use_ of a good product, Zope 2.

I see the current situation being one where we're balancing resource
between continuing it (Zope 2) as more than just a viable product -
and as the basis of our business - while also investing the resources
to have the next version free of the fundamental limitations of the
current one.

We obviously can't do as much for Zope 2 or Zope 3 as we could if we
dove into just one or the other.  However, i think we would be making
a big mistake if we neglected either for the other!  Even further, i
happen to think we're walking the line pretty well - because we have
to, we have to keep our business viable while we proceed.

> > 	Finally, if your product is any good, you will always have your
> > smartest and fastest developers clamoring to push ahead - let
> > them, but don't let their buzz stir FUD with the much bigger
>=20
> This is the point that we're trying to address on this list: external=20
> communications.  The "much bigger group" cares less, IMO, about new=20
> component architectures.  If you look at a standard evaluation matrix=20
> for application servers, the kinds of things the bigger group cares=20
> about -- is much of that getting attention?

Mostly in Zope 2, i think - hopefully in ways that will transfer more
or less directly to Zope 3, as it catches up to Zope 2 in
capabilities.

And make no mistake - not every effort goes into results that will
satisfy a check-off list - if i understand the principle correctly,
developing a framework that will ease the development and reuse of
those kinds of features is as important, or more, than the end
features, at least in this stage of development.  Zope 2 has been
proof that we can provide those features in an extensible way, Zope 3
(as i think you know) is about refining the framework to make it
easier for us (in the larger sense - Zope developers) to provide
those features.  We still need to satisfy our direct customers in
customer gigs, which i think helps keep our efforts on the
underpinning calibrated...

> Talking about risks is never fun nor popular.  It always comes across=20
> as fighting the bandwagon, heresy, being overly-pessimistic, etc.
>=20
> I was hoping that we could move past the analysis of risks pretty=20
> quickly and talk more about the steps to minimize them.  However,=20
> consensus has proven elusive on the risk assessment part.

I'm sorry if i'm obscuring the picture.  I still feel that the
pitfalls i saw trap ArsDigital in the account are different than the
ones we're facing.  I even think we're trying to do something
different than the basic crossing-the-chasm perspective - that is,
we're trying to maintain our current business, and improve it, while
substantiating the groundwork/framework that frees us up from some of
the limitations in the current system.  I can see how some of the
typical risks in balancing the r&d versus sustaining existing
production, plus managing the transition to the new version, applies.

I actually think the necessary transparency of our open-source/
community-involved development process makes this a bit of a different
beast, and that continuing to address the issues that come up in the
transparency mitigates some of the common "appearance" problems that
more conventional efforts entail.

I even think that these differences specifically require a somewhat
different assessment of the risks.

It's like the situation we're seeing in a customer project - one i'm
on.  We're collaborating directly with the customers developers to
develop the system.  On the one hand, we're spending more time up
front mentoring and dealing with churn as they travel along with us as
apprentices, increasingly as full partners.  On the other hand, i see
radically different, and better, prospects for the success of the
system through transition, because they will already be acquainted
with it, know how to use and change it.  Even mroe, acceptance is much
less of a question, because they are seeing the system moment to
moment as it develops, and will not be surprised by features different
than they expect because, in many cases they're involved in
implementing them, or at least dealing with the changes as it affects
the part of the system they're implementing.

I think you can see how this is not altogether different than Zope 3
development.  It's for these reasons that i keep mentioning the
transparency we must maintain for the process, and that i feel we can
afford to take the time that is needed to do it right (but not get
complacent and take any more time than that).  We have an advantage
over someone who is doing it faster by doing it in with smaller visiblity
- we have better assurance that our customers, the developers, will
accept it, because they're intimately involved in building it.

> Even if acknowledged the risks, and then decide not to do anything=20
> about them, *at least* that would be a risk assessment.  We'd know more=
=20
> about the task ahead.  Still, I think many of these risks can be=20
> addressed with some good thinking, IMO.

I'm don't mean to dismiss risks.  I think some of them - not all - are
different than the ones conveyed in the ArsDigita story, that's all.

In fact, i do feel i understand through this exchange more about what
you're trying to achieve - the clarification of and focus on what
exactly the risks are.  I even got to articulate some of the ones that
i think are particular to our situation, different than those for
ArsDigita, and think i see better the ones you feel are crucial.  I
guess we disagree about some of those.  All in all, i'm glad we're
doing this - sorry about the length of this (and i don't expect to be
able to regularly sustain this level of involvement - do i hear a
collective sigh of relief?-) - but will definitely try to keep a hand
in.

--=20
Ken
[email protected]