Re: FYI: Lessons from Kai Wu at ArsDigita

I'll try to point out the places where I think Kai's email provides=20
similarities.

Note: the point here is risk management.  This transition is exciting,=20=

but it's also dangerous.  Thus, we must be brutally honest, even=20
paranoid, about learning from the examples in the industry.  Simply=20
finding a few differences related to our situation, and then throwing=20
out the similarities -- that's not risk management, it's wishful=20
thinking.

On mardi, f=E9v 11, 2003, at 10:20 Europe/Paris, Paul Everitt wrote:

> =46rom Kai Wu
>
> 	Ah, well (grimace) there wasn't a clear plan for transitions -
> we built ACS 4 Tcl anew, while ACS 5 Java, of course, shared
> little besides the data model with ACS 4 Tcl.  If you started on

Us too.  ACS 5 had a new data model, Zope 3 has a new API and new=20
component model.

> a given system (ACS 3, ACS 4, or ACS 5 Java) you usually stayed
> on it - translating software plumbing is a nauseating task for
> smart programmers, nor is it an appealing expense for clients.

Based on interactions in the last few weeks, I also worry about the=20
"nauseating task" part of ensuring a smooth transition.  But presumably=20=

this falls under the official promise, meaning it will get official=20
manpower to fulfill the promise.

> 	Zip back to Summer 2000; ArsDigita had a handful of our sharpest
> developers coding ACS 4 Tcl (note: we did not have a QA team, or
> dedicated doc people). We pre-sold new customers on ACS 4 Tcl for
> a while.  We told old customers (on ACS 3 Tcl) we'd take care of
> 'em - meaning, a developer or two on our end was essentially

I presume this will apply to any existing Zope company.  While this is=20=

feasible, supporting two very different platforms has a cost.  Do you=20
train some of your guys in supporting Zope 2 and some on Zope 3?  Do=20
you train all of them on both?

> stuck in support mode, tied to a legacy client.  At that point we
> had a separate engineering team forging ahead with ACS 4 Tcl
> pretty much freed of any obligation to do backwards compatibility

That's the story for us too.  Elaborating the migration issues happens=20=

some point in the future.

> with ACS 3; the changes were quite radical.  (We actually tried

Changes are quite radical for us.

> to get close to Zope's object model - no easy task using an RDBMS
> and a procedural language.)  Of course, the old customers who
> managed to survive (the 'net bubble was just starting to deflate
> at that point, in mid 2000) were rather curious as to what ACS 4
> had.  Some were willing to pay for developer time to do custom
> migrations, most not - new features, and not new plumbing, are

This is true for us as well.  It will take quite a while for the=20
features of Zope 3 to catch up with the features of Zope 2.  Also, it=20
will take a while for the maturity and bug-proofedness to catch up.

> almost always brighter lights for aggressive dot.coms to be drawn
> to.
> 	By Fall 2000, with our sales and marketing guys telling us for
> the past year that Tcl sucked in the marketplace and that all big
> clients were asking for Java, the first ACS 4 release was done.
> Almost immediately, we re-focused on ACS 5 Java, while expanding
> our engineering team dramatically.  Well this ticked off a lot of
> people, internally and externally.  Our professional services
> people, having just ramped up on ACS 4 Tcl, were ticked that we

Zope is just starting to get big in the marketplace.  New companies=20
will want to enter, and they too will be faced with this issue: do I=20
train my 15 people on Zope 2 or Zope 3?  (Although the "few months" for=20=

us will be "few quarters".)

> were pretty much abandoning it within a few months.  Clients just
> started on ACS 4 Tcl were confused and worried.  We had at end of

We're likely to have some of this as well.  Some people will accept a=20
vendor statement that says everything will be ok.  Others will remember=20=

how vendors have said that in the past, and things didn't work out well.

> 2000 almost nothing working in Java, just orders to get it done
> ASAP.  As it turned out, 5-6 months later in summer 2001, the
> first client projects were tentatively deployed on ACS 5 Java.

We won't have too much of this, as long as the message is sent: don't=20
use Zope 3 in production until (some time in 2004).

I worry because I hear some of the core developers saying they want to=20=

start using it first half of the year.  If that happens, we'll have a=20
different ballgame (IMO).  We'll start to lose flexibility.

> Upon reflection I get a little freaked just writing this; in
> retrospect it was an insane schedule, driven by no small amounts
> of fear and desperation, and as one of the lead managers I walked
> a constant tight-rope with client teams who did their best
> delivering systems on alpha software.

This risk is definitely under control.

> 	Even by the time I left (got axed, urk!) aD in December 2001,
> client teams were still ramping up on ACS 5 Java, and product
> development was still intense.  And if you visit the Red Hat site
> (they bought us out in a rather, uhm, closed-door deal) and look
> for CCM, you'll see it continues; I'm quite curious to see it
> eventually since I like my former colleagues and liked our
> product idea, but I've decided to go with Zope for several of my
> own projects.
> 	Back to the past: Of course, our open-source external community
> (see openacs.org) was up in arms about all these changes; first
> they were asked to adopt the ACS 3 Tcl orphan.  Then the ACS 4
> Tcl newborn came along.  It took them a long time to translate
> and begin improving ACS 4 Tcl (they use Postgres, we used
> Oracle) - in some ways, that learning curve sucked up such
> critical amounts of time and energy that their whole community
> suffered; growth in ACS adopters slowed greatly.  I don't have
> numbers, but anyone who visited the openacs.org site during 2001
> would, with a little investigation, find lots of FUD around
> ArsDigita, the ACS, and the community - enough to say, "Screw it,
> I'll go with PHP" (small budget) or "Screw it, I'll go with J2EE"

Us too.  We need to understand and appreciate that others, out in the=20
non-Zope universe, are NOT going to have such a sublime assessment of=20
our transition.  For business people, the specter of Zope 3 will=20
probably be as much a hindrance as a help in competing for contracts in=20=

2003.

> (big budget).  Suffice to say, by the time we said we'll be going
> to Java, it was essentially a divorce; a process that is costly
> and damaging to both parties, no matter how much either one
> denies it.
> 	Lessons?
> 	Always have a few good technical scouts providing
> well-considered feedback as to long-term product and marketplace

I don't really think we're doing too much of this.  Then again, that's=20=

probably out of scope.

> trends so your development efforts have a high ROI and stay
> relevant, and so that your product plans are proactive and can be
> communicated consistently (perhaps aD started out on the wrong
> technical base; perhaps, if we had gone from ACS 3 Tcl to an ACS
> Java, and told everyone so for a while, we'd be OK).
> 	If you have the luxury of time and a slow transition, use it.

Certainly we're doing this.

> 	And always, always have something to show people (users,
> developers, business people, the press) that works today, is
> polished (good installation, ample docs, high quality, and
> completed functionality with slick UI), and is clearly being
> supported (lists, IRC, forums, newsletters with buzz, and regular
> module/Product releases) - that gets you the gold of credibility.
> Perhaps today's pace is just a bit slower than the boom time, but
> I think even a period of 6 months where you're promising The Next
> Great Thing while reluctantly showing your old software with

Let's face it, we're stuck pretty hard in this one.  We talk up Zope 3=20=

as being worthy of the energy of a total rewrite.  You only rewrite=20
when the old thing was unfixable, right?  "Shuffling feet" is fairly=20
accurate.

> downcast eyes and shuffling feet will do you serious harm.  And
> we all know 6 months is not enough time for anything as big and
> cool as Zope 3 to mature.
> 	Finally, if your product is any good, you will always have your
> smartest and fastest developers clamoring to push ahead - let
> them, but don't let their buzz stir FUD with the much bigger

This is the point that we're trying to address on this list: external=20
communications.  The "much bigger group" cares less, IMO, about new=20
component architectures.  If you look at a standard evaluation matrix=20
for application servers, the kinds of things the bigger group cares=20
about -- is much of that getting attention?

> group of regular coders, users, clients, etc.  We know those
> uber-developers are clever and eager to prove it with some great
> software; sometimes too eager.  Disdain for continuity in the

Increasing the amount of continuity is what some of us are advocating. =20=

With Jim's recent note, the path from Zope 2 to Zope 3 will be smoothed=20=

by backporting important parts of Zope 3.  That's good news.  Previous=20=

statements indicated some "translation" would be employed instead.

> product and disdain for its community can kill or maim the whole
> product; a real pity for open-source products, where many people
> give very generously of their time and energy.
> 	There may be more pithy lessons, but that's what I can think of
> for now.  Hope this helps,
>
> Kai

Talking about risks is never fun nor popular.  It always comes across=20
as fighting the bandwagon, heresy, being overly-pessimistic, etc.

I was hoping that we could move past the analysis of risks pretty=20
quickly and talk more about the steps to minimize them.  However,=20
consensus has proven elusive on the risk assessment part.

Even if acknowledged the risks, and then decide not to do anything=20
about them, *at least* that would be a risk assessment.  We'd know more=20=

about the task ahead.  Still, I think many of these risks can be=20
addressed with some good thinking, IMO.

--Paul