RE: ?= $(shell ...) re-runs shell in sub-makes when $(origin VAR) is environment (regression in 4.4)

I have reverted to make 4.3 as my temporary workaround, just wanted to repo=
rt the issue

Once the new assignment operation ?:=3D is available I will update make to =
the latest and convert my makefiles.
will this change be available as an update in the ubuntu resolute repos ?


-----Original Message-----
From: Paul Smith <[email protected]>
Sent: Wednesday, May 20, 2026 8:54 AM
To: Gunter Woytowitz <[email protected]>; [email protected]
Subject: Re: ?=3D $(shell ...) re-runs shell in sub-makes when $(origin VAR=
) is environment (regression in 4.4)

On Mon, 2026-05-18 at 02:06 +0000, Gunter Woytowitz wrote:
> In GNU Make 4.4 (confirmed on 4.4.1), "export VAR ?=3D $(shell COMMAND)"
> re-evaluates $(shell) in every recursive sub-make, even though the
> variable was already evaluated and exported by the top make and the
> sub-make's $(origin VAR) correctly reports "environment".

You have definitely found a problem, but your diagnosis isn't accurate
(unfortunately: if it were the problem would be a simple bug, rather than a=
 deep issue that's hard to resolve).

I think you're getting confused by the facilities you are choosing in your =
makefile to show the problem.  If you choose a simpler reproducer you'll ge=
t a clearer view of the issue.

Consider this makefile:

  export FOO ?=3D $(shell echo foo)
  export BAR ?=3D $(shell echo bar)

  $(info origin FOO =3D $(origin FOO))
  $(info FOO =3D $(value FOO))

  $(info origin BAR =3D $(origin BAR))
  $(info BAR =3D $(value BAR))

  all: ; echo $$FOO $$BAR

Note this things: first, we use $(info ...) rather than echo to show inform=
ation directly.  Second, we use $(value ...) to show the actual value of th=
e variable, which is what make will evaluate when it expands the variable.

Finally, we don't bother with recursion which adds confusion to the output;=
 this is not needed since we can simply set the variables in the environmen=
t before invoking make.

So, no matter which version of make you use you get the expected
result:

  $ make
  origin FOO =3D file
  FOO =3D $(shell echo foo)
  origin BAR =3D file
  BAR =3D $(shell echo bar)
  echo $FOO $BAR
  foo bar

However, if you check how many instances of the shell are invoked in versio=
n 4.3 you'll see (this is on GNU/Linux of course):

  $ strace make-4.3 2>&1 | grep clone | wc -l
  3

whereas in 4.4.1 you'll see:

  $ strace make-4.4.1 2>&1 | grep clone | wc -l
  5

Now, two extra clones doesn't seem like much but as you observed, this valu=
e goes up very quickly with the number of exported variables that contain s=
hell calls.  If I add a new line to the makefile:

  export BAZ ?=3D $(shell echo baz)

then the clone() call count in make-4.4.1 goes up to 16!

But we can see that your guess that the problem is that make is still expan=
ding values that are overridden in the environment, is not correct by provi=
ding environment variables:

  $ FOO=3Denv1 BAR=3Denv2 make
  origin FOO =3D environment
  FOO =3D env1
  origin BAR =3D environment
  BAR =3D env2
  echo $FOO $BAR
  env1 env2

and now the clone count:

  $ FOO=3Denv1 BAR=3Denv2 strace make-4.3 2>&1 | grep clone | wc -l
  1

  $ FOO=3Denv1 BAR=3Denv2 strace make-4.4.1 2>&1 | grep clone | wc -l
  1

So, no more work is done here.

The problem you are running into is a result of this change from the NEWS f=
ile:

* WARNING: Backward-incompatibility!
  Previously makefile variables marked as export were not exported to comma=
nds
  started by the $(shell ...) function.  Now, all exported variables are
  exported to $(shell ...).  If this leads to recursion during expansion, t=
hen
  for backward-compatibility the value from the original environment is use=
d.
  To detect this change search for 'shell-export' in the .FEATURES variable=
.

Resolving https://savannah.gnu.org/bugs/?10593


Unfortunately that has the unanticipated side effect that EVERY time $(shel=
l ...) is invoked, EVERY exported variable is expanded.  This gives the beh=
avior you are seeing.

There is a still-open issue we are using to consider how to alleviate this =
but so far no brilliant ideas have surfaced:
https://savannah.gnu.org/bugs/index.php?64746

One thing we did do is add support for a "?:=3D" assignment operator that d=
oes conditional assignment but results in a simple variable assignment, not=
 a recursive variable assignment.  That would fix the issue above (in the n=
ext release of GNU Make) but of course it's not backward-compatible.

> Replacing ?=3D with :=3D (immediate assignment, computed once at top-make
> parse time) avoids the problem at the cost of breaking command-line
> variable override.

There are other ways, of course.  You can do something like:

  default_TOP  :=3D $(shell pwd)
  export TOP   ?=3D $(default_TOP)

That will force the shell to run every time, even when overridden, but it w=
ill only run once (per invocation of make) not multiple times.

Or, you can get very tricksy and use my hack for "deferred simple variable =
expansion":

https://make.mad-scientist.net/deferred-simple-variable-expansion/

Although not discussed in that blog post this works with ?=3D as well.

If you use that trick in my example above and repeat the experiment you'll =
see that make-4.4.1 does the same number of clone() calls as make-4.3, incl=
uding only one when variables are inherited from the environment.

--
Paul D. Smith <[email protected]>            Find some GNU Make tips at:
https://www.gnu.org/                       http://make.mad-scientist.net/
"Please remain calm...I may be mad, but I am a professional." --Mad Scienti=
st



This email may contain confidential and privileged information and is inten=
ded solely for the use of the addressee(s). Unless you are the addressee or=
 are authorized to receive messages for the addressee, you may not use, cop=
y, disseminate, or disclose the information or any attachments to any third=
 party. If you have received this correspondence in error, please notify th=
e sender immediately and delete this email. Your cooperation and understand=
ing are greatly appreciated. Attention Federal Customers: Please note this =
email platform is NOT approved to communicate (send or receive) CUI. For qu=
estions on the approved system to communicate CUI, please contact your desi=
gnated Vcinity Representative.