Re: SETENVI, make "port" and poudriere weirdness

On Sat, 27 Sep 2025 11:06:41 +0200 A FreeBSD User wrote:
> Am Tage des Herren Fri, 26 Sep 2025 17:56:08 +0200
> Tijl Coosemans <[email protected]> schrieb:
> 
>> On Thu, 25 Sep 2025 20:10:45 +0200 A FreeBSD User wrote:  
>>> Am Tage des Herren Wed, 24 Sep 2025 19:17:23 +0200
>>> Tijl Coosemans <[email protected]> schrieb:    
>>>> On Wed, 24 Sep 2025 13:46:07 +0200 A FreeBSD User wrote:      
>>>>> Hello,
>>>>> 
>>>>> running FreeBSD CURRENT and FreeBSD 15-STABLE on several machines, I
>>>>> face some issues with the traditional way of compiling ports and the
>>>>> more resource-eating way with poudriere.
>>>>> 
>>>>> Some ports when compiled via "cd /usr/ports/PORT; make all" do fail
>>>>> with an obscur error reporting
>>>>> 
>>>>> : invalid option -- D
>>>>> 
>>>>> followed by a bunch of gmake options (for instance see PR 277961,
>>>>> x11-wm/libwraster which is still open). Those ports fail in an
>>>>> "unclean natural" environment while compiling in the artificial
>>>>> sterile poudriere environment. In almost all cases the problem is due
>>>>> to "overlapping" environments, targeted by the introduction of
>>>>> SETENVI - see /usr/ports/CHANGES, tag 20240229:.
>>>>> 
>>>>> I try to compile a well maintainded ESP32 toolchain
>>>>> (https://github.com/trombik/xtensa-esp32-elf, here:
>>>>> devel/riscv32-esp-elf and/or devel/xtensa-esp-elf).
>>>>> 
>>>>> The non-official port compiles flawless within recent FreeBSD CURRENT
>>>>> and 15-STABLE with poudriere-devel. 
>>>>> 
>>>>> It fails on all boxes running CURRENT and 15-STABLE  using "make
>>>>> all": It fails in those portions of the make process while "do
>>>>> build" uses ${SETENV} instead of ${SETENVI} [${WRK_ENV}]. After
>>>>> fixing that I'm stuck in the last step of do-build:
>>>>> 
>>>>> I do not want to go into too much details here since I want to point
>>>>> out that the real world "make all" differs in a significant way from
>>>>> the artificial sterile way of doing poudriere (which is to much
>>>>> efford on development/experimental platforms where we change options
>>>>> very often).
>>>>> I'd like to understand the underlying behaviour to fix the
>>>>> misbehaviour in "make all" - this seems to be very hard and I fear
>>>>> that I miss something, some knowledge about the PORTS framework, the
>>>>> role of the environment and so on. I almost changed EVERY posiible
>>>>> permutation of the line (see Makefile of xtensa-esp-elf):
>>>>> 
>>>>> [...]
>>>>> 
>>>>> do-build:
>>>>> ORIG:
>>>>>     cd ${BUILD_WRKSRC} && ${SETENV} ${MAKE_ENV} ${BUILD_ENV} ./ct-ng
>>>>> build VARIANT1:
>>>>> 	cd ${BUILD_WRKSRC} && ${SETENVI} ${MAKE_ENV} ${BUILD_ENV}
>>>>> ./ct-ng build VARIANT2:
>>>>> 	cd ${BUILD_WRKSRC} && ${SETENVI} ${WRK_ENV} ${MAKE_ENV}
>>>>> ${BUILD_ENV} ./ct-ng build
>>>>> 
>>>>> 
>>>>> The great magic here seems to be one needs to understand the
>>>>> difference between poudriere's enviroenment and the environment of my
>>>>> account on several boxes. My (blunt and stupid) approach is to
>>>>> replace on a bail out the first occurence of "${SETENV}" with
>>>>> "${SETENVI} ${WRK_ENV}" as suggested in the above mentioned
>>>>> reference in CHANGES.        
>>>> 
>>>> Is your ports tree up to date because this appears to have been fixed
>>>> in https://cgit.freebsd.org/ports/commit/?id=b8bbb207f55db    
>>> 
>>> Sorry for  late responding.
>>> I made it a habit to keep almost everything on a daily basis up to
>>> date and before compiling a port or something related to the OS I do a
>>> pull (git).
>>> 
>>> The port deve/xtensa-esp-elf is the official port in the ports tree.
>>> The port does not cover ESP32 RISCV32 based and doesn't cover latest
>>> ESP-IDF 5.5.
>>> 
>>> I try to compile  https://github.com/trombik/xtensa-esp32-elf which does
>>> compile fine as long as it is compiled in poudriere (also up to date
>>> with up to date trees and an overlay covering the additional port). As
>>> mentioned, poudriere does the job in its obscure  ways. Replacing the
>>> port devel/xtensa-esp-elf with the port by this Japanese developer
>>> mentioned above in the tree an running either portmaster or make all
>>> does fail as I described above. I refered to some ports and PRs with
>>> similar issues - and in most cases with those ports, the introduction
>>> of SETENVI solves the problem. But not with the reluctant port from
>>> github. When patching the portions of the Makefile responsible for
>>> building, I'm able to get rid of the " : invalid option -- D" issue -
>>> of which the cukprit mutually has been identified but then I run in
>>> seroius trouble at the Makefile's section  "do-build": the process
>>> then complains about missing source file archive (picolibc-esp) -
>>> which is definitely present and which is not complained about by the
>>> poudriere build process. At this point I'm pretty confident that
>>> poudriere and the traditional way are different, q.e.d.
>>> 
>>> I'd like to understand to fix the problem, otherwise I'm drifting
>>> around like a dead man in the water hoping others fix the problem by
>>> accident. That is not an option. I'd like to understand why poudriere
>>> builds the unaltered port without any flaws and the make/portmaster
>>> does not. 
>>> 
>>> Kind regards and thanks,
>>> Oliver    
>> 
>> You'll have to make all the same changes to the overlay that
>> b8bbb207f55db made: add WRK_ENV and replace all SETENV with SETENVI.  
> 
> I do, I did, of course.
> 
> The overlay used in poudriere doesn't have any issue.
> 
> The issue arise using portmaster, then this spooky mixing up of make/gmake occurs and can be
> resolved by addapting ${SETENVI} at places where ${SETENVI} is residing, sometimes adding
> ${WRK_ENV} additionaly.
> 
> In most cases the issue occurs in "build" or "do-build" (of the make framework).
> 
> In the reported case this is complete gone wild, even "make all" starts getting rogue,
> 
> I focussed too much on the "do-build" or "build" part of the Makefile, maybe also the
> "do-configure" and "configure" parts are highly sensitive to this SETENV/SETENVI issue. There
> fore, one has too understand the software the FreeBSD's port system is targetting for - and I
> do not ...
> 
> And, for a reminder, I'm confused by this port (not officially in the ports tree):
> 
> https://github.com/trombik/xtensa-esp32-elf 
> and from there
> devel/xtensa-esp-elf
> devel/riscv32-esp-elf
> 
> both toolchains targetting all available ESP-IDF (5.3, 54 and 5.5 as of today). adapting and
> building with poudriere - no problem. Having a "polluted" ports tree with this specific port,
> even "portmaster" AND "make all" fail 

Add the following line to do-build and comment out the other lines:

	${SETENV} ${BUILD_ENV} env | grep MAKE

Then run 'make build' and see if it prints anything.