Re: Need some bug/version info

Hello David,

I don't know if you know me - I am Vin Shelton, the Release Engineer
for the stable branch of XEmacs.  Therefore, the 21.4.16 and 21.4.17
releases are my responsibility.

You raise a number of points below.  I agree with the gist of your
argument: XEmacs 21.4.16 should be withdrawn and 21.4.17 should be
released as soon as possible.  XEmacs 21.4.17 will be released on
February 6.

David Kastrup <[email protected]> writes:

> Ralf Angeli <[email protected]> writes:
>
>> * Stephen J. Turnbull (2005-02-02) writes:
>>
>>>>>>>> "Ralf" == Ralf Angeli <[email protected]> writes:
>>>
>>>     Ralf> As we are about to release AUCTeX 11.55, we are in need of a
>>>     Ralf> workaround for 21.4.16.
>>>
>>> (when (and (featurep 'xemacs)
>>>             (emacs-version>= 21 4 16)
>>>             (not (emacs-version>= 21 4 17)))
>>>   (error 'invalid-state "XEmacs 21.4.16 (and only that release) has a
>>> fatal bug in the regexp code that AUCTeX triggers in several places.
>>> Upgrade to 21.4.17."))
>>>
>>> in some appropriate place in the auctex load sequence is what I
>>> would suggest; even if AUCTeX works around it, other applications
>>> could trigger it.
>>
>> That's pretty consequent.  Besides people building current stable
>> versions of XEmacs directly from xemacs.org, it would currently lock
>> out everybody facilitating a distribution with a frequently updated
>> package/port system, like Debian/sid, Fedora Core Devel, Mandrake
>> Cooker or the BSDs.  And then we can only hope that none of the bigger
>> distributions will ship XEmacs 21.4.16 with their next stable
>> releases.
>
> Actually, shy regexps are by now used rather widely in Lisp packages
> (for example, preview-latex).  XEmacs' version of regexp-opt _also_
> produces shy regexps, and it is used all across the XEmacs code base
> and external packages, even those that don't explicitly use shy
> regexps themselves.
>
> As far as I can see, the main symptom would be random core dumps.  I
> don't think that people will assign the blame to AUCTeX for core
> dumps.

Most of the reports have actually been of infinite loops rather than
core dumps.

>
> So basically this is major XEmacs problem affecting its operation all
> across the board, and it does not make sense for AUCTeX to deal in an
> isolated manner with it.  This is a matter of XEmacs release and
> quality management.  The release should be pulled and people
> prominently given notice.  If package authors in general are asked to
> include code like the above, then we will probably follow suit.
>
> But you would be doing a disservice to your users by pretending this
> matter is even close to being dealt with responsibly if AUCTeX refuses
> to load.

I agree with your assessment of the situation.  XEmacs 21.4.17 will be
released on February 6.

> People who are using 21.4.16 will be hosed also without using AUCTeX.
> Shy regexps are an integral part of XEmacs via regexp-opt.

Thanks for the data of which packages use regexp-opt in a separate
post.  Although the number of packages may seem large, my empirical
evidence is it's extremely easy to work for days without stumbling
across any problems.  I'm sure that some of the packages are more
likely to tickle this problem than others.

> So my take on this matter is that we will _not_ change our code in the
> above way _unless_ there is an _official_ call of the XEmacs
> development team to _all_ package developers to do this.  Such a call
> would probably only make sense in connection with a _general_ warning
> about that XEmacs release (which is supposed to be in the stable
> series).
>
> Unless the XEmacs developers declare this step necessary for _all_
> package developers in order to control the spread of this
> fundamentally buggy release, I don't think we should muddy our own
> code base, but instead just mention this problem in the release notes,
> and maybe as package conflicts.

That would be one approach, but I think that forcing (or encouraging)
all the package maintainers to do that is a loser.  (Our
decentralization/parallelization is both a weakness and a strength.)
Having the XEmacs maintenance team do it ourselves is also probably a
loser, because getting the changes accepted back upstream can be
somewhat challenging.

> It is also a possibility that our installation procedure check for
> working shy regexps (after all, this is not the first time they have
> created havoc within XEmacs, as can be witnessed from the
> preview-latex change log entries from Nix).
>
> But the proper place for such checks would really be in regression
> tests for XEmacs, not in the installation procedures of AUCTeX.

The regression tests for XEmacs have been extended to verify that the
fix is in place.

How bullet-proof you want to make the installation procedures for
AUCTeX is a decision for the AUCTeX team to decide.  Personally, I
would think it would be worthwhile to your users that AUCTeX verify
that the enviroment is suitable for safely running AUCTeX.  How far
you want to take that verification is a matter for the project leaders
to decide.

>> Unless somebody has a better idea I will insert a code stanza
>> similar to the one proposed by Stephen into tex-site.el and add a
>> prominent note to the release notes that people who are using XEmacs
>> 21.4.16 should stay away from AUCTeX 11.55 or upgrade to 21.4.17 as
>> soon as it is available.
>
> It does not make sense to make AUCTeX the poster child for telling
> this to people.  My opinion is to mention this prominently in the
> release notes as long as it is to be feared that this XEmacs version
> is around.  And how long this is to be feared very much depends on how
> the XEmacs developers will choose to deal with the problem.  If it is
> their opinion that an occasional core dump in normal operation is no
> reason to warn people off a release in a prominent manner, then it is
> _us_ that is going to get blamed when some stock XEmacs/AUCTeX
> combination gets made available and refuses to work.

This is a defensible position, but I don't think I agree with it.  As
I implied above, different packages can be affected quite differently
by the bug.  It's possible that some packages may need to defend
themselves against broken enviroments more carefully than other
packages need to.  I don't have enough data to tell for sure, but it
sounds like AUCTeX is one package that might want to defend itself
against this particular breakage in XEmacs.

> This XEmacs version must not get put on _any_ software distribution
> intended to be stable since XEmacs' operation as a whole is affected,
> and it is beyond the power of the AUCTeX developers to tell
> distributors "don't use 21.4.16", except possibly by package
> conflicts.  And if a distributor finds AUCTeX conflicting with XEmacs,
> he will in case of doubt throw out AUCTeX, obviously.  We can't
> reasonably substitute for XEmacs quality management in this instance.

I will make a post to the xemacs-announce mailing list and will
withdraw the 21.4.16 tarballs.  Do you have any suggestions for how
else to communicate the bad news about XEmacs 21.4.16 and the good
news about XEmacs 21.4.17?

> I consider this of high relevance to the XEmacs community, and
> I would appreciate it if the auc-tex list were kept informed in case
> the XEmacs developers decide to take any measures for deprecating
> 21.4.16 outside of this particular thread.

I agree that this matter is of high importance, and appreciate your
including xemacs-beta on this thread.

I apologize for the quality of the 21.4.16 release.  The fault is
entirely mine.

Sincerely,
  Vin Shelton