Re: RFC: More accurate color conversion

Søren Sandmann <[email protected]> writes:
> If we add the additional assumption that the intervals between
> corresponding to successive integers must be a constant length, then it
> follows from f(0x0000) = 0.0 and f(0xffff) = 1.0 that f(i) = i /
> 65535.0. The easiest way to see that is to consider a conversion to
> 2-bit integers instead:
>
>      0.0                                             1.0
>       |-----------------------------------------------|
>      0b00            0b01            0b10            0b11

The picture I drew on my whiteboard yesterday was similarly for a 2-bit
integer, but I drew bars for the ranges of floating-point numbers that
would map to each integer. Like so:

        +-------+---------------+---------------+-------+
       0b00     |      0b01     |      0b10     |      0b11
        +-------+---------------+---------------+-------+

Then, my drawing for cairo's current algorithm was:

        +-----------+-----------+-----------+-----------+
       0b00         |  0b01     |      0b10 |          0b11
        +-----------+-----------+-----------+-----------+

Note that positions for each integer are the same with both approaches.

Then, yes, what you are saying is true: If we wanted to reduce cairo's
error for the roundtrip conversion starting with a floating-point value,
and we wanted to do this by only changing the integer-to-floating-point
mapping, then we would end up with (as you drew):

>      0.0                                             1.0
>       |-----------------------------------------------|
>            0b00        0b01        0b10        0b11

and with my bars:

        +-----------+-----------+-----------+-----------+
        |    0b00   |    0b01   |    0b10   |    0b11   |
        +-----------+-----------+-----------+-----------+

And yes, that would be undesirable in that we would lose access to the
floating-point extremes when converting from integers.

Color me convinced. Your change minimizes the error without introducing
unwanted deviations in the floating-point values obtained from integers.

> The main motivation is that I want to extend pixman's test suite with
> more tolerance based tests, where a reference pixel is computed in
> floating point and then compared to pixman's output. It's highly
> desirable here that the _reference_ computation doesn't introduce its
> own errors.

Thanks for explaining. I'm in favor of the change, (and particularly
since the change will be backed by additional testing).

-Carl

-- 
[email protected]

-- 
cairo mailing list
[email protected]
http://lists.cairographics.org/mailman/listinfo/cairo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)

iQIcBAEBCAAGBQJSVFA9AAoJEGACM7qeVNxhFI4P/3RyjmLDeAZK8NXzzGpK28TR
lehZhJOeQ3GO3CQ1TqKYzQHNxRtG6woXMVf/1zqziWkF8ihHIjUciIucJpoA2Wh5
1CwBNni/OrmCKhHvTa7KD1TCdIvr6mK8e0M+OUiGklbzJC5h3ZE2p7qkMI7VSOgR
xXdmsuglKYGw8gaZOOckhGrmDRiBB2+oE8VpLsgFg7hgEFjIO3akAPhwQez7WgY3
3RaMM2Tj4b6HNld1EOHNPGVzd+ncv3mj32LthSqyjupZtWwPhd6eqgkbzIPdJvvn
485lqXi6T5l3Ne2FWwu7EgiPGd195v/ZqECfPfLuwwXr+6ABDPyP4zus57BWM+Ug
0uopT2niMLAFxXEmpbRn60283lBJ5bUSAQDXEuOkhLTWtoZrPqiWEGuhoWeTMYPt
uZoEBZaZlmTvozo1LHdsIulc9tIu3aD3mk1CoZOk8IrwL8iOO+Fe6ZmBz3Q+3Nzw
GBUhzHk7UNov2Y/SEk0qSdfuoKULzyjLSkQ6s42hAsF2KzMoxVPWqZ6o3mAxjvKu
JScMvxrgSsdb31IEQzeGMKLBj5/xgSUrS6Td9NC3SiJH+hOHD9OD/YaxgM+H2Og7
IN/l3GIDR6Ik3emKjNrTmHfyN210BJMU+EikN/Yr5SC0axoYJIOxDA4xqtuL+Xgh
kk0jWthejYYlXZbRo1bq
=26aq
-----END PGP SIGNATURE-----