Re: stylo reftest-stylo jobs are green

\o/ \o/ \o/

On Wed, Feb 1, 2017 at 3:13 AM, Cameron McCormack <[email protected]> wrote:

> Thanks to Shing's manifest updating script which he ran recently, and
> some work to manually disable tests and update failure/assertion
> annotations, and modulo some current leaks, the reftest-stylo jobs are
> now green.  As we are doing with the crashtest job, let's try to keep
> them green, by including reftest-stylo in the try pushes we do before
> landing Stylo-affecting patches.
>
> Remember that the reftest-stylo job is the one that uses the
> reftest-stylo.list manifest files, and which compares a reftest test
> file with Stylo enabled and disabled.  So if you need to update the
> manifests as part of the patches you land, it's the reftest-stylo.list
> ones you need to change.
>
> Because the reftest-stylo.list manifests were originally generated by
> Shing's scripts, which disabled crashing/intermittent tests and (I
> think?) added assertion annotations automatically, we mostly don't have
> comments in the manifests pointing to bugs for the underlying reason the
> test is failing.  If, when you're updating annotations as part of your
> changes, you notice a new crash or assertion failure pattern, please
> file a bug blocking bug 1324620.  For now, I don't think we need to
> bother filing bugs for reftest image comparison failures.
> _______________________________________________
> dev-tech-layout mailing list
> [email protected]
> https://lists.mozilla.org/listinfo/dev-tech-layout
>