Re: Fwd: PROPOSED RESOLUTION: merge csswg-test into web-platform-tests

On 27/09/16 19:20, L. David Baron wrote:

> I'd like to see us run the reftests with our existing reftest
> harness, which is optimized to do image comparison in compiled code,
> and which also has a bunch of useful failure-annotation features [1]
> and features that test for other sorts of failures such as
> NS_ASSERTIONs.

I think there are a number of ways that this is non-trivial; for example 
I'm not aware that we have the infrastructure needed to generate reftest 
manifests from the source tree, I don't think the existing reftest 
harness can handle the possibility of a test causing a crash cleanly, I 
don't think we have the ability to autogenerate expectation data to 
account for the fact that many tests will fail, and so on. I would 
rather start with wptrunner, which already solves these problems, and 
improve the harness with better performance and error reporting since 
these can be added incrementally whereas the other things are 
prerequisites for any frequently updated testsuite from upstream.