Test suite changes month-by-month this year

I mentioned I was going to check test results weekly.  There were no
commits this past week, so instead I did a look back at past test
performance, about once a month since the 1.12.16 release:

                     1.12    11/26   12/16   1/29    2/27    3/29
Tests run:           24005   24838   24838   24838   24838   24838
Passed:              15959   16167   16164   16166   15883   16725
Failed:              4554    5103    5101    5099    5385    5529
Expected Failed:     336     330     330     330     330     332
Error:               2       0       1       1       1       1
Crashed:             982     1015    1019    1019    1016    28
Untested:            2172    2223    2223    2223    2223    2223
Total:               24005   24838   24838   24838   24838   24838

Listings of what test results changed between each snapshot:

 http://www.bryceharrington.org/Cairo/changes-1.12-to-20131126.txt
 http://www.bryceharrington.org/Cairo/changes-20131126-to-20131216.txt
 http://www.bryceharrington.org/Cairo/changes-20131216-to-20140129.txt
 http://www.bryceharrington.org/Cairo/changes-20140129-to-20140227.txt
 http://www.bryceharrington.org/Cairo/changes-20140227-to-20140329.txt

Between 1.12.16 and 11/26, a number of surface and fill tests started
failing.  I could probably narrow this down further, as there were a
number of significant changes, such as to rounding behavior and
downscaling.  But here's the patches that directly changed surface:

$ git log --since "2013-08-21" --until "2013-11-26" --date=short --pretty=format:"%h %cd - %an : %s" | grep ' surface:'
935c64a 2013-09-05 - Alexander Larsson : surface: Inherit device scale in cairo_surface_create_similar()
bc792a5 2013-09-05 - Alexander Larsson : surface: Merge scratch construction into _cairo_surface_create_scratch
bdccf4f 2013-09-05 - Alexander Larsson : surface: Opencode create_similar
7ab34f3 2013-09-05 - Alexander Larsson : surface: expose the device scale


The result changes between 11/26 and 12/16 look to be just test cases
that randomly crash.  I believe these are all fixed up now.

Ditto for the 12/16-1/29 results.

1/29-2/27 just are from the mesh regression I introduced in 44a09f462c
and that was fixed in bb174036.

The 2/27-3/29 results are Uli's fix to the mesh bug, and fixes to a
bunch of crashes.

Bryce
-- 
cairo mailing list
[email protected]
http://lists.cairographics.org/mailman/listinfo/cairo