Re: Unittests!

On Monday 08 November 2010, Inge Wallin wrote:
> On Sunday, November 07, 2010 13:04:42 Boudewijn Rempt wrote:
> > For Krita, I've the habit of sending out a weekly mail with all tests that
> > fail. Here's the same for all of KOffice:
> > 
> > 
> > The following tests FAILED:
> >          10 - kword-part-TestRdf (Failed)
> >          13 - kformula-LoadAndSave (Failed)
> >          19 - KPlato-EstimateTester (Failed)
> >          30 - kspread-Formula (Failed)
> >          50 - kspread-Sheet (Failed)
> >          80 - krita-image-KisConvolutionPainterTest (Failed)
> >         170 - krita-plugin-format-jpeg_test (Failed)
> >         213 - kotext-odf-TestLoading (Failed) (this one fails because i
> > haven't qtscript installed. I guess it would fail even if I had). 272 -
> > kchart-TestCellRegion (Failed)
> >         278 - shapes-text-TestDocumentLayout (Failed)
> >         279 - shapes-text-TestChangeListCommand (Failed)
> >         284 - shapes-text-TestTextTool (Failed)
> >         285 - shapes-text-TestChangeTrackedDelete (Failed)
> >         286 - kspread-xlsx-FormulaParser (Failed)
> > 
> > Note that failing unittests are release blockers. That is, if this list
> > isn't empty, we cannot release 2.3.
> 
> Actually, the page about release blockers say that unittests *in the 
> libraries* are release blockers [1].
> 
> >          13 - kformula-LoadAndSave (Failed)
> 
> This one will not get fixed before the release, afaict. The failings is 
> because entities, i.e. things like &LargerThan; in MathML, aren't supported. 
> That's a regression from 1.6, but was never working in any 2.x version. It is 
> indeed a big deficiency, but is it enough to break the feature freeze?
> 
> I guess I can disable these parts of the tests if it's important, but that 
> seems like a very cosmetic change without much substance.
> 

I think the canonical solution is to set those tests to EXPECT_FAIL and document that (and not merge that commit to essen, of course.)

-- 
Boudewijn Rempt | http://www.valdyas.org
Ceterum censeo lapsum particulorum probae delendum esse