Re: Problem with deep data

This is a multi-part message in MIME format.
--===============0944451683523867232==
Content-Type: multipart/alternative;
 boundary="------------43809436224E8CA1C23E8803"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------43809436224E8CA1C23E8803
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit


Duplicating the problem will be hard without seeing your code. If the 
IlmImfTest deep tests are passing, but your code is failing, that would 
make me suspect there's either a bug in your own code, or that you are 
using the API in an unusual way - different to how the tests use it - 
that's triggering a bug we've not seen before. The IlmImfTest suite does 
write images similar to the ones you've been testing with.

A couple of perhaps more obvious things to double-check: make sure that 
you don't have a mysterious 8 bit variable there that's wrapping round 
in a weird way, or some calculation that's casting to a 'char' instead 
of 32 or 64 bit value. That could fit the your symptoms, particularly if 
it's in the part of the code where you set up the array of pointers to 
float arrays to store each pixel. Also, a common slip-up with OpenEXR is 
forgetting that the data and display windows are /inclusive/ - a 256 
pixel wide image has displayWindow.max.x set to 255.

On linux, you might try running your code through valgrind to see if it 
identifies any issues with accessing uninitialised or out-of-bound memory.

In your descriptions you don't mention how many samples per pixel are 
being written. Perhaps try writing 257 pixel wide scanline with one data 
sample in each pixel, then an image where the first or the last pixel 
has many samples and all the rest have 0 samples. This might shed light 
on whether the odd behaviour you are seeing is dependent on the total 
number of samples written, or the total number of pixels. You can also 
try writing with an offset dataWindow (e.g. a 256 pixel wide dataWindow 
with dataWindow.min.x = 100 and dataWindow.max.x=355) to see whether the 
256 pixel problem is relative to the dataWindow or the displayWindow.



On 21/04/18 07:06, Richard Hadsell wrote:
>
> Here is another clue that might help someone find the bug:
>
> The shift in data is independent of the number of channels in the 
> deep-data part.  When I tested with 6 channels (named A, B, G, R, X, 
> and Y), the channels are written and read in that (alphabetic) order.  
> Looking at the bad data read for pix[0] I discovered that pix[0].A was 
> garbage, but pix[0].B was pix[256].A.  Each channel of pix[0] was the 
> previous channel's value for the last sample.  It looks like the data 
> were in the buffer, but the start pointer was off by one float.
>
> On 04/20/2018 02:50 PM, Richard Hadsell wrote:
>> Your last suggestion was most helpful.  I had already examined the 
>> pointers and strides, set the part to NO_COMPRESSION, and generated 
>> only 1 sample per pixel, but I had not tried it with tiny images.
>>
>> I found that an image with a single scanline and up to 256 pixels was 
>> okay.  However, 257 pixels resulted in the first pixel (pix[0]) 
>> having junk and the other pixels being shifted, so that pix[1-256] 
>> had the values that should have been in pix[0-255].  This was the 
>> result from reading the file.
>>
>> Using 'od' to look at the file showed that the last float in the file 
>> was, indeed, the sample for pix[256].  Maybe the write was okay, but 
>> the read was not.
>>
>> (Testing with 258 pixels in the scanline resulted in bad data for 
>> pix[0-1], and values in pix[2-257] were the samples that should have 
>> been in pix[0-255].)
>>
>> For the test with 257 pixels, I looked at the data in TotalView and 
>> saw that readPtr, as calculated in line 673 of 
>> ImfDeepScanLineInputFile.cpp, points to floats that correspond to the 
>> samples that are bad.  The first float is junk, and the next 256 
>> samples are those that should have been in pix[0-255].  The float 
>> after that was 0, not the value that should have been in pix[256], so 
>> I conclude that the data in the buffer were not read correctly from 
>> the file.
>>
>> Of course, I don't know enough about the layout of data in the file 
>> to know whether the problem is in reading or writing.  I can see that 
>> the samples are there in the file, but maybe they are in the wrong place.
>>
>> I hope this is enough information for someone to duplicate the 
>> problem.  I don't think I can take it any farther myself.
>>
>
> -- 
> Dick Hadsell			203-992-6320  Fax: 203-992-6001
> Reply-to:			[email protected]
> Blue Sky Studioshttp://www.blueskystudios.com
> 1 American Lane, Greenwich, CT 06831-2560


--------------43809436224E8CA1C23E8803
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
      charset=windows-1252">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    Duplicating the problem will be hard without seeing your code. If
    the IlmImfTest deep tests are passing, but your code is failing,
    that would make me suspect there's either a bug in your own code, or
    that you are using the API in an unusual way - different to how the
    tests use it - that's triggering a bug we've not seen before. The
    IlmImfTest suite does write images similar to the ones you've been
    testing with.<br>
    <br>
    A couple of perhaps more obvious things to double-check: make sure
    that you don't have a mysterious 8 bit variable there that's
    wrapping round in a weird way, or some calculation that's casting to
    a 'char' instead of 32 or 64 bit value. That could fit the your
    symptoms, particularly if it's in the part of the code where you set
    up the array of pointers to float arrays to store each pixel. Also,
    a common slip-up with OpenEXR is forgetting that the data and
    display windows are <i>inclusive</i> - a 256 pixel wide image has
    displayWindow.max.x set to 255.<br>
    <br>
    On linux, you might try running your code through valgrind to see if
    it identifies any issues with accessing uninitialised or
    out-of-bound memory. <br>
    <br>
    In your descriptions you don't mention how many samples per pixel
    are being written. Perhaps try writing 257 pixel wide scanline with
    one data sample in each pixel, then an image where the first or the
    last pixel has many samples and all the rest have 0 samples. This
    might shed light on whether the odd behaviour you are seeing is
    dependent on the total number of samples written, or the total
    number of pixels. You can also try writing with an offset dataWindow
    (e.g. a 256 pixel wide dataWindow with dataWindow.min.x = 100 and
    dataWindow.max.x=355) to see whether the 256 pixel problem is
    relative to the dataWindow or the displayWindow.<br>
    <br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 21/04/18 07:06, Richard Hadsell
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:[email protected]">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      <br>
      <div>Here is another clue that might help someone find the bug:<br>
        <br>
        The shift in data is independent of the number of channels in
        the deep-data part.  When I tested with 6 channels (named A, B,
        G, R, X, and Y), the channels are written and read in that
        (alphabetic) order.  Looking at the bad data read for pix[0] I
        discovered that pix[0].A was garbage, but pix[0].B was
        pix[256].A.  Each channel of pix[0] was the previous channel's
        value for the last sample.  It looks like the data were in the
        buffer, but the start pointer was off by one float.<br>
        <br>
        <div class="moz-cite-prefix">On 04/20/2018 02:50 PM, Richard
          Hadsell wrote:<br>
        </div>
        <blockquote cite="mid:[email protected]"
          type="cite">Your last suggestion was most helpful.  I had
          already examined the pointers and strides, set the part to
          NO_COMPRESSION, and generated only 1 sample per pixel, but I
          had not tried it with tiny images.<br>
          <br>
          I found that an image with a single scanline and up to 256
          pixels was okay.  However, 257 pixels resulted in the first
          pixel (pix[0]) having junk and the other pixels being shifted,
          so that pix[1-256] had the values that should have been in
          pix[0-255].  This was the result from reading the file.<br>
          <br>
          Using 'od' to look at the file showed that the last float in
          the file was, indeed, the sample for pix[256].  Maybe the
          write was okay, but the read was not.<br>
          <br>
          (Testing with 258 pixels in the scanline resulted in bad data
          for pix[0-1], and values in pix[2-257] were the samples that
          should have been in pix[0-255].)<br>
          <br>
          For the test with 257 pixels, I looked at the data in
          TotalView and saw that readPtr, as calculated in line 673 of
          ImfDeepScanLineInputFile.cpp, points to floats that correspond
          to the samples that are bad.  The first float is junk, and the
          next 256 samples are those that should have been in
          pix[0-255].  The float after that was 0, not the value that
          should have been in pix[256], so I conclude that the data in
          the buffer were not read correctly from the file.<br>
          <br>
          Of course, I don't know enough about the layout of data in the
          file to know whether the problem is in reading or writing.  I
          can see that the samples are there in the file, but maybe they
          are in the wrong place.<br>
          <br>
          I hope this is enough information for someone to duplicate the
          problem.  I don't think I can take it any farther myself.<br>
          <br>
        </blockquote>
        <br>
        <pre class="moz-signature" cols="256">-- 
Dick Hadsell			203-992-6320  Fax: 203-992-6001
Reply-to:			<a class="moz-txt-link-abbreviated" href="mailto:[email protected]" moz-do-not-send="true">[email protected]</a>
Blue Sky Studios                <a class="moz-txt-link-freetext" href="http://www.blueskystudios.com" moz-do-not-send="true">http://www.blueskystudios.com</a>
1 American Lane, Greenwich, CT 06831-2560
</pre>
      </div>
    </blockquote>
    <br>
  </body>
</html>

--------------43809436224E8CA1C23E8803--


--===============0944451683523867232==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Openexr-devel mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/openexr-devel

--===============0944451683523867232==--