Re: Mesa3D graphics library build failure when using Scons 3.0.3 and newer on Windows with Visual Studio 2017, Scons 3.0.1 is unaffected.

On Thu, Feb 28, 2019 at 5:59 AM Jose Fonseca <[email protected]> wrote:

> On 27/02/2019 21:47, Bill Deegan wrote:
>
> Is the windows thread/spawn bug fixed in Python now?
>
> No.  They explicitly stated they wouldn't fix, and applications should
> move away from os.* into subprocess.  See
> https://bugs.python.org/issue6476
>
>
> We have seen an infrequent issue with a race condition on windows as well.
> I'm fairly certain is due to Python dropping the GIL around file
> open/close..  something like env.Install(target,source), env.Program(...,
> target) so a linker or other program tries to use the file after it is
> copied and/or written by python Action before it's closed on the filesystem
> yielding file still open errors..
>
> Yes, I'm surprised how broken some of the Posix things are in msvcrt.
> It's safer to use Win32 directly whenever possible (as subprocess does.)
>
>
> Any builder with [] as the source is odd. Then adding the dependencies via
> Depend(). Generally you'd specify the script.  Most build's consider that
> to be sufficient.
>
> It's been a while since I last look into this, but I'll try to improve it.
>
>
> Your build goes a step further by examining all the python modules it
> loads. Very clever.  I think that logic could be moved to a python scanner
> and then you could get rid of most of the special logic there.
>
> You mean, add a Python scanner to SCons itself?
>
Yes.

> A difficulty with what we try to do is that the Python version used for
> SCons and for the scripts is not necessarily the same.  That probably
> explains why we ended up with a mixture of explicit dependencies and
> scanning.  To properly fix this we should write the Python dependency
> scanner as a standalone script, which would use the exact python version
>

The current solution does use the same python as SCons to find the
dependencies.
In what case would you use a different Python?

> When I get a chance I'll take a look at the rest.
>
> Thanks.
>
> Jose
>
>
> On Tue, Feb 26, 2019 at 11:28 PM Jose Fonseca <[email protected]> wrote:
>
>> On 26/02/2019 22:46, Bill Deegan wrote:
>> >
>> >
>> > On Tue, Feb 26, 2019 at 7:57 AM Jose Fonseca <[email protected]
>> > <mailto:[email protected]>> wrote:
>> >
>> >     Hi Bill,
>> >
>> >     If you spot anything on Mesa's SCons usage that looks like a bad
>> idea
>> >     then do let us know.
>> >
>> >
>> > Likely there's room for improvement.
>> > Let me finish vetting the patch to fix your current issue and then I'll
>> > see if I can find some time to do a review?
>> > In the meantime if you have questions please bring them here?
>>
>> I don't have any concrete questions -- except this matter, SCons has
>> been working fine.   My question is based upon your remarks that we're
>> doing odd stuff.  I'd just would like to better understand it, and
>> improve it, specially if that can avoid breakage in the future.
>>
>> >     One of the difficulties of Mesa is that there were always multiple
>> >     build
>> >     tools system, with the bulk of developers (those targeting Linux)
>> using
>> >     first autotools / now meson, and only a subset (those targeting
>> >     Windows)
>> >     using SCons.  So that's why I tried to make things somewhat
>> automatic.
>> >
>> >     Results have been mixed: when it works, it works well; and it fails,
>> >     nobody knows how to fix except me who most of this..
>> >
>> >
>> > Have you subscribed to the scons users mailing list before today?
>> > Honestly I wasn't aware Mesa was using SCons until Liviu brought it up.
>>
>> No, never subscribed.  We've been using it for 10+ years, and BTW, it
>> has been working very well for us (ie those who build Mesa for Windows,
>> VMware in particular.)  It's a pity SCons never gained traction with
>> Mesa Linux developers who stuck with the autotools until last year.  If
>> I had tried harder back then to get all Mesa tree fully ported to SCons
>> perhaps things would have been different, but one can only do so much,
>> and by then me and my team lost too many technical/political fights to
>> embark on yet another one.
>>
>> I can only recall two instances of SCons issues affecting us, this and
>> http://scons.tigris.org/issues/show_bug.cgi?id=2449
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fscons.tigris.org%2Fissues%2Fshow_bug.cgi%3Fid%3D2449&data=02%7C01%7Cjfonseca%40vmware.com%7C6129499ce9734e98a13108d69cfd35f1%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C636869008685509210&sdata=p8MDu4uju1rGouDGDi%2Fgmobuq4RgPkx0tBhnf37l%2Bcc%3D&reserved=0>
>> .  That was a pretty
>> nasty issue to debug from my side, as the race condition was pretty
>> rare.  In fact, I just realized we still have the workaround on
>> https://cgit.freedesktop.org/mesa/mesa/tree/scons/fixes.py
>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freedesktop.org%2Fmesa%2Fmesa%2Ftree%2Fscons%2Ffixes.py&data=02%7C01%7Cjfonseca%40vmware.com%7C6129499ce9734e98a13108d69cfd35f1%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C636869008685519219&sdata=pIZAutXygbKiYW0TKBD%2BlhdRuYCEiO3Q3KgiipEFATY%3D&reserved=0>
>> .  It's
>> probably safe to drop it now :)
>>
>> >
>> >
>> >       > Calling any builder with sources set to []...
>> >
>> >     What should we used instead?
>> >
>> >
>> > Most times the source for such a command is the script itself.
>> > Then you'd just add the python scanner and scons should (I think) do
>> > what you're manually doing via env.Depends()...
>> >
>> > -Bill
>> >
>> >
>> Jose
>>
>
>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users