Re: SCons bug when combining CacheDir and MSVC_BATCH

Hi Bill,

Disabling MSVC_BATCH for both Environments is successful:

$ scons --cache-debug=-
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Retrieved `build1\main.obj' from cache
CacheRetrieve(build1\main.obj):  retrieving from 271f0c07bd36befae43c546754892cb4
requests: 1, hits: 1, misses: 0, hit rate: 100.00%
CacheRetrieve(build1\foo.obj):  065661fe50a5dabcbe97c12c3851870c not in cache
requests: 2, hits: 1, misses: 1, hit rate: 50.00%
cl /Fobuild1\foo.obj /c src\foo.c /nologo
foo.c
CachePush(build1\foo.obj):  pushing to 065661fe50a5dabcbe97c12c3851870c
requests: 2, hits: 1, misses: 1, hit rate: 50.00%
CacheRetrieve(build1\a.exe):  53fb57b1111735e2542ad175c63f4401 not in cache
requests: 3, hits: 1, misses: 2, hit rate: 33.33%
link /nologo /OUT:build1\a.exe build1\main.obj build1\foo.obj
CachePush(build1\a.exe):  pushing to 53fb57b1111735e2542ad175c63f4401
requests: 3, hits: 1, misses: 2, hit rate: 33.33%
CacheRetrieve(build2\foo.obj):  a0dbce679b116a4af9f659f102aa6f37 not in cache
requests: 1, hits: 0, misses: 1, hit rate: 0.00%
cl /Fobuild2\foo.obj /c src\foo.c /nologo
foo.c
CachePush(build2\foo.obj):  pushing to a0dbce679b116a4af9f659f102aa6f37
requests: 1, hits: 0, misses: 1, hit rate: 0.00%
CacheRetrieve(build2\a.exe):  c12a2901cc6b9bdd3896e188e4e97fe5 not in cache
requests: 2, hits: 0, misses: 2, hit rate: 0.00%
link /nologo /OUT:build2\a.exe build2\main.obj build2\foo.obj
CachePush(build2\a.exe):  pushing to c12a2901cc6b9bdd3896e188e4e97fe5
requests: 2, hits: 0, misses: 2, hit rate: 0.00%
scons: done building targets.

Possibly more interesting is that I still get the link failure if env1 does *not* have MSVC_BATCH but env2 does. With the opposite, the build is successful.

Cheers,

S.

From: Bill Deegan
Sent: Wednesday, May 24, 2023 5:03 PM
To: SCons users mailing list <[email protected]>
Subject: Re: [Scons-users] SCons bug when combining CacheDir and MSVC_BATCH

Can you try without MSVC_BATCH enabled?
-Bill

On Wed, May 24, 2023 at 8:50 AM Steve Hill (Wireless) via Scons-users <[email protected]<mailto:[email protected]>> wrote:

Thanks for the reply, Mats.



1) Moving the setting of MSVC_BATCH to the constructor doesn't make any difference - I get the same link failure.

2) Adding debug, I get:



$ scons --cache-debug=-

scons: Reading SConscript files ...

scons: done reading SConscript files.

scons: Building targets ...

Retrieved `build1\main.obj' from cache

CacheRetrieve(build1\main.obj):  retrieving from 97427c14b8c8b53e9e7af4d7d8afd524

requests: 1, hits: 1, misses: 0, hit rate: 100.00%

CacheRetrieve(build1\foo.obj):  2c4b62611510a27efcd0dd7066b26b04 not in cache

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

cl /Fobuild1\\ /c src\main.c src\foo.c /nologo

main.c

foo.c

Generating Code...

CachePush(build1\main.obj):  97427c14b8c8b53e9e7af4d7d8afd524 already exists in cache

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

CachePush(build1\foo.obj):  pushing to 2c4b62611510a27efcd0dd7066b26b04

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

CacheRetrieve(build1\a.exe):  bcd159a2ee6e98e239022957d9dfe010 not in cache

requests: 3, hits: 1, misses: 2, hit rate: 33.33%

link /nologo /OUT:build1\a.exe build1\main.obj build1\foo.obj

CachePush(build1\a.exe):  pushing to bcd159a2ee6e98e239022957d9dfe010

requests: 3, hits: 1, misses: 2, hit rate: 33.33%

Retrieved `build2\main.obj' from cache

CacheRetrieve(build2\main.obj):  retrieving from 5619d81a934ee71939605662178f5d54

requests: 1, hits: 1, misses: 0, hit rate: 100.00%

CacheRetrieve(build2\foo.obj):  6d30ba71fee8872f9c2c8541225ead92 not in cache

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

cl /Fobuild2\\ /c src\foo.c /nologo

foo.c

CachePush(build2\foo.obj):  pushing to 6d30ba71fee8872f9c2c8541225ead92

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

CacheRetrieve(build2\a.exe):  e4533b0dc80dcb3d3b1cd2cdd081933c not in cache

requests: 3, hits: 1, misses: 2, hit rate: 33.33%

link /nologo /OUT:build2\a.exe build2\main.obj build2\foo.obj

LINK : fatal error LNK1181: cannot open input file 'build2\main.obj'

scons: *** [build2\a.exe] Error 1181

scons: building terminated because of errors.



The odd thing is that it is retrieving build2/main.obj from the cache when it is already up-to-date but that still doesn’t explain why it gets deleted…



S.



-----Original Message-----
From: Scons-users <[email protected]<mailto:[email protected]>> On Behalf Of Mats Wichmann
Sent: Wednesday, May 24, 2023 3:49 PM
To: [email protected]<mailto:[email protected]>
Subject: Re: [Scons-users] SCons bug when combining CacheDir and MSVC_BATCH



On 5/24/23 06:24, Steve Hill (Wireless) via Scons-users wrote:

> All,

>

> We have recently switched on CacheDir in our CI machine and, while

> this has largely been a positive experience, we have seen the odd link

> failure that we could not explain. I was eventually able to reproduce

> the issue and produce a minimal SConstruct that illustrates the problem.



Something's broken here.  I've tried to trace this a little bit and it looks like things can get removed after they're retrieved.  There are other oddities too, in your rebuild log:



> Retrieved `build1\main.obj' from cache cl /Fobuild1\\ /c src\main.c

> src\foo.c /nologo main.c foo.c Generating Code...



If there are two sources, but one object is up to date, SCons should not calculate a batched build is needed, in other words it should have emitted:



cl /Fobuild1\foo.o /c src\foo.c /nologo



> Would you like me to raise an issue on GitHub for this?

I'd say yes, but not being wise in the ways of MSVC_BATCH maybe there's something I'm missing.



Can you try a couple of tweaks to your reproducer (thanks for taking the time to come up with one!) while you're at things?



1) put the MSVC_BATCH initialization in the Environment call.  The msvc tool is notoriously sensitive to getting its initialization wrong if it doesn't have all its details in hand when it runs.  No idea if that moves the goalposts but worth trying.

2) turn on some debugging - add  --cache-debug=-   (the dash argument

sends the debug to the console)





MSVC_BATCH has several issues already in the tracker about interaction with things, didn't look to me like any of them were likely this issue, though.
_______________________________________________
Scons-users mailing list
[email protected]<mailto:[email protected]>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://urldefense.com/v3/__https:/pairlist4.pair.net/mailman/listinfo/scons-users__;!!Niha4SQ!4unNd_qrV1SXfyxNYK7N3FaGQ8MEV_5m5uGHpBCJ0aEkO7vz4eA36Eo4phi8NG0eH8Gxxy9xBwdTgKJ5jGShaoveWw4lww$>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users