Handle truncation in mpx_cleandir

Richard Copley <[email protected]> Thu, 12 Aug 2021 10:47:57 +0100
Newsgroups gmane.comp.tex.metapost
Message-ID <CAPM58ohUiMTrYcF777iOwTuOYrMrESTuqOb4_gjfeonwS2xRbA@mail.gmail.com>
On 64-bit Windows systems, when MetaPost calls mpx_cleandir to delete
temporary auxiliary files (e.g., from running TeX to format a label),
it segfaults in RtlEnterCriticalSection.

This affected, for example, the 64-bit binaries built by the MSYS2
project. (See <https://github.com/msys2/MINGW-packages/pull/9350>.)
It doesn't affect the 32-bit binaries shipped by the TexLive
project.

See mpx_cleandir in "mpxout.w". hFile is declared as long. The handle
returned by _findfirst is stored there and truncated. Then it is
sign-extended and passed to _findnext. There is a segfault when it is
first dereferenced, which happens in RtlEnterCriticalSection.

This can be fixed by changing "long" to "intptr_t" in mpx_cleandir.

An example MetaPost program that leads to the crash:

  beginfig(0); label(btex $x$ etex, (0,0)); endfig; end;

To reliably reproduce the crash, place the program in a file in an
otherwise empty directory and run mpost.exe in that directory.

Warm regards,
Richard.
--
http://tug.org/metapost/