[rt.cpan.org #107663] Unpacking of byte array still buggy in 0.83_01

Fri Oct 09 10:29:18 2015: Request 107663 was acted upon.
Transaction: Ticket created by [email protected]
       Queue: Win32-API
     Subject: Unpacking of byte array still buggy in 0.83_01
   Broken in: 0.82, 0.83_01
    Severity: Normal
       Owner: Nobody
  Requestors: [email protected]
      Status: new
 Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=107663 >


Hi,

a few ago I stumbled over the error in Win32API::Struct 0.82 when unpacking a BYTE / UNSIGNED CHAR buffer.
After some debugging I found that the buffers are being packed using 'a'.$repeat but got unpacked using 'Z'.$repeat.

I'm using plain Strawberry Perl 32 and/or 64 bit on Windows 7 64 bit.
"This is perl 5, version 22, subversion 0 (v5.22.0) built for MSWin32-x86-multi-thread-64int"

I used Win32::API to access FTDI's FTCJTAG.DLL and imported the DLL functions using the corresponding .h file and Win32::API::more.
Great so far! Really easy to import a DLL using the .h file contents.

The buffers were declared :
Win32::API::Struct->typedef ('WriteDataByteBuffer', qw (
        BYTE data[64]));
Win32::API::Struct->typedef ('ReadDataByteBuffer', qw (
        BYTE data[64]));
Win32::API::Struct->typedef ('ReadCmdSequenceDataByteBuffer', qw (
        BYTE data[128]));

Win32::API::Type->typedef('PWriteDataByteBuffer', 'WriteDataByteBuffer*');
Win32::API::Type->typedef('PReadDataByteBuffer', 'ReadDataByteBuffer*');
Win32::API::Type->typedef('PReadCmdSequenceDataByteBuffer', 'ReadCmdSequenceDataByteBuffer*');

The JTAG_Read function in FTCJTAG.DLL in my case/for my device returns 5 bytes. (4 bytes data DWORD and 1 byte JTAG status bits) 

unpack ('Z64', $buffer->data) stops unpacking at the first 0 found -> WRONG

unpack ('a64', $buffer->data) unpacks all bytes even zeros and pads with 0.

I fixed it just by changing Z to a.

Then while finishing my tasks and documenting, I found the new version 0.83_01 that tries to fix something unpacking arrays.
But unpack ('Z'... for any byte sized array is still wrong, and when fixed by a 'a' then the Win32::API::_TruncateToWideNull($$itemvalueref) later on kills the buffer's values.

IMHO the Z is only correct if the buffer contains a Zero terminted string, if the buffer contains any binary data, 'a' is the only valid format specifier for the unpack, because 0s (zeros) are preserved.
Padding a zero-terminated string with 0s does not harm, whereas cutting binary data at the first zero is harmful.

I would suggest                 
$type = $type_size == 1 ? 
'Z'.$repeat #have pack truncate to NULL char
                        :'a'.($repeat*$type_size); #manually truncate to wide NULL char later

to be changed to 
$type = 'a'.($repeat*$type_size); 

Uuuh I can't find Win32::API::_TruncateToWideNull,
but commenting out that line (with type='a'....) solves my problem.

WideChars should at least be of original size 2...

Open for comments,
 Axel