Re: FYI, build times for 15.0-RELEASE and 14.4-RELEASE poudriere jail worlds vs. 14.3-RELEASE poudriere jail worlds: an example [timing the 7950X3D doing only the builds done on the WinDevKit2023]

On 4/19/26 07:41, Mark Millard wrote:
> On 4/18/26 17:02, Mark Millard wrote:
>> On 4/18/26 12:26, Mark Millard wrote:
>>> There have been reports of rather large time differences for
>>> 14.4-RELEASE vs. 14.3-RELEASE --as well as prior reports referencing
>>> 14.3-RELEASE  vs. 15.0-RELEASE. This is just a report of from-scratch
>>> building of a particular set of port-packages on a fairly fast machine
>>> (amd64 7950X3D) for each of those.
>>>
>>> Upstream based context installed/used:
>>>
>>> pkgbase GENERIC-NODEBUG main boot kernel
>>> pkgbase debug           main boot world (/etc/malloc.conf -> junk:false)
>>>
>>> pkgbase     15.0-RELEASE poudriere-devel jail world.
>>> ftp-archive 14.4-RELEASE poudriere-devel jail world.
>>> ftp-archive 14.3-RELEASE poudriere-devel jail world.
>>>
>>> (So: None are my builds.)
>>>
>>> # poudriere jail -l
>>> JAILNAME                   VERSION          OSVERSION ARCH  METHOD
>>> TIMESTAMP           PATH
>>> release14p3-amd64          14.3-RELEASE-p10 1403000   amd64 ftp-archive
>>> 2026-04-14 18:00:43 /usr/local/poudriere/jails/release14p3-amd64
>>> . . .
>>> release14-amd64            14.4-RELEASE-p1  1404000   amd64 ftp-archive
>>> 2026-04-14 18:02:42 /usr/local/poudriere/jails/release14-amd64
>>> . . .
>>> release-amd64              15.0-RELEASE-p5  1500068   amd64 pkgbase
>>> 2026-04-17 16:09:49 /usr/local/poudriere/jails/release-amd64
>>> . . .
>>>
>>> ZFS context.

ANother note about my context: the defaults for gcc and llvm are:

lang/gcc15
devel/llvm22

but the 7905X3D also built:

lang/gcc14
devel/llvm19

The port-packages that WinDevKit2023 built did not include lang/gcc14 or
devel/llvm19 .

>>>
>>>
>>> As for the 7950X3D build times:
>>> (maximum ratio: 02:29:18/02:04:06 approx.= 1.2)
>>>
>>> 15.0-RELEASE (run third) took the most time:
>>> (1 less built than for 14.*-RELEASE : 687 vs. 688)
>>>
>>> [02:29:20] [release-amd64-alt] [2026-04-18_08h47m48s] [committing] Time:
>>> 02:29:18
>>>            Queued: 688 Inspected: 0 Ignored: 0 Built: 687 Failed: 1
>>> Skipped: 0 Fetched: 0 Remaining: 0
>>>
>>>
>>> 14.4-RELEASE (run second) took the middle amount of time:
>>>
>>> [02:24:13] [release14-amd64-alt] [2026-04-18_06h23m35s] [committing]
>>> Time: 02:24:11
>>>            Queued: 689 Inspected: 0 Ignored: 0 Built: 688 Failed: 1
>>> Skipped: 0 Fetched: 0 Remaining: 0
> 
> Note: 02:24:11/02:04:05 approx.= 1.16
> Note: max ratio difference: 1.2 - 1.16 = 0.04
> 
>>>
>>>
>>> 14.3-RELEASE (run first) took the least time, by a bigger amount:
>>>
>>> [02:04:06] [release14p3-amd64-alt] [2026-04-17_21h53m02s] [committing]
>>> Time: 02:04:05
>>>            Queued: 689 Inspected: 0 Ignored: 0 Built: 688 Failed: 1
>>> Skipped: 0 Fetched: 0 Remaining: 0
>>>
>>>
>>> So, in the 79050X3D's context, 14.4-RELEASE and 15.0-RELEASE are similar
>>> in how much more time they take than 14.3-RELEASE.
> 
> (Wording above intended as via time ratio comparisons to 14.3.)
> 
> Note: The above 14.4 vs. 15.0 comparison/contrast was not true for the
> WinDevKit20203 (aarch64) of the 3 releases via the ratios, see later
> below. So 15.0 has something rather significant that makes builds take
> more time on the WinDevKit20203 compared to 14.4's ratio.
> 
> (The WinDevKit20203 was tested with a smaller subset of the
> port-packages to be built but still took more than a day to finish the
> sequence of testing 3 poudriere-devel jail releases.)
> 
>>> This suggests that
>>> the biggest difference makers are probably just differences between 14.4
>>> and 14.3 that are also in 15.0, not any extra differences 15.0 may have
>>> vs. 14.3.
> 
> Note: Again, not true for the WinDevKit2023 15.0-RELEASE vs.
> 14.4-RELEASE differences.
> 
>>>
>>> I've never managed to replicate the much larger 14.4 vs. 14.3 time
>>> ratios that have been reported.
> 
> Both platforms were somewhat under 1.25 for the maximum ratio. That is
> much less than one source of build time performance reports that I've seen.
> 
>>>
>>>
>>> For reference:
>>>
>>> # ~/fbsd-based-on-what-commit.sh -C /usr/ports-alt/
>>> b96a271bbc08 (HEAD -> main, freebsd/main, freebsd/HEAD) www/firefox:
>>> update to 150.0 (rc1)
>>> Author:     Christoph Moench-Tegeder <[email protected]>
>>> Commit:     Christoph Moench-Tegeder <[email protected]>
>>> CommitDate: 2026-04-16 17:51:06 +0000
>>> branch: main
>>> merge-base: b96a271bbc0820a058e89001c9ac2309cbf8a60d
>>> merge-base: CommitDate: 2026-04-16 17:51:06 +0000
>>> n742012 (--first-parent --count for merge-base)
>>>
>>> /usr/ports-alt/ is a clean tree.
>>>
>>> My /usr/local/etc/poudriere.d/poudriere.conf is involved and likely unusual.
>>>
>>> 15.0-RELEASE has a newer jemalloc.
>>>
> 
> The below paragraph spans the 3 release tests:
> 
>>> The SWAP space was never put to use. The 1min load average got to be as
>>> high as: 178.27, the 5min: 125.95, the 15min: 76.14 . (Sampled every top
>>> update by a patched top.) There are 16 SMT cores, so 32 FreeBSD cpus.
>>> Implicit 32 builders allowed in parallel.  ALLOW_MAKE_JOBS=yes was in
>>> use but no MAKE_JOBS_NUMBER_LIMIT or the like was in use, so: implicit
>>> 32 per builder allowed. The media in use was (is): PCIe Optane 1.4 TB.
> 
> Note: 192 GiBytes of RAM.
> 
>>>
>>>
>>>
>>> SIDE NOTE:
>>> A different example, aarch64 Windows Dev Kit 2023 context, USB3 media,
>>> UFS, only around 273 port-packages queued/built . . .
> 
> maximum ratio: 09:58:30/08:10:58 approx.= 1.22
> (That was for 15.0-RELEASE)

Later I show use of more significant digits that I did originally.

> 
> But that is a lot more than for 14.4 ratio for WinDevKit2023, unlike
> what was observed for the 7950X3D.
> 
>>>
>>>
>>> 15.0-RELEASE:
>>
>> This was run third and the in-process from-scratch build queued 271 (2
>> less than under 14.*). It will be many hours before there is a final
>> "time" for this.
>>
>>>
>>> UPDATE 15.0-RELEASE NOTES LATER, given how long the WinDevKit2023 builds
>>> take.
> 
> [09:59:09] [release-aarch64-alt] [2026-04-18_15h34m27s] [committing]
> Time: 09:58:30
>            Queued: 271 Inspected: 0 Ignored: 0 Built: 271 Failed: 0
> Skipped: 0 Fetched: 0 Remaining: 0
> 
> This is a lot longer than 14.4 below. So it significantly contributes to
> the maximum ratio vs. 14.3 for the WinDevKit2023 test.
> 
>>>
>>>
>>> 14.4-RELEASE (run first):
>>>
>>> [09:12:00] [release14-aarch64-alt] [2026-04-17_22h10m56s] [committing]
>>> Time: 09:11:36
>>>            Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
>>> Skipped: 0 Fetched: 0 Remaining: 0
> 
> Note: 09:11:36/08:10:58 approx.= 1.12
> Note: max ratio difference: 1.22 - 1.12 = 0.10 (unlike 7950X3D 0.04)
> 
>>>
>>>
>>> 14.3-RELEASE:
>>
>> This was run second.
>>
>>>
>>> UPDATE 14.3-RELEASE NOTES LATER, given how long the WinDevKit2023 builds
>>> take.
>>
>> [08:11:31] [release14p3-aarch64-alt] [2026-04-18_07h22m56s] [committing]
>> Time: 08:10:58
>>            Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
>> Skipped: 0 Fetched: 0 Remaining: 0
>>
>> Note: 09:11:36/08:10:58 approx.= 1.12
> 
> Other notes spanning the 3 release tests:
> 
> 4208Mi was the maximum swap space use observed, there being 32 GiBytes
> of RAM. The 1min load average got to be as high as: 61.29, the 5min:
> 55.65, the 15min: 48.65 . (Sampled every top update by a patched top.)
> There are 8 single-hardware-thread cores, so 8 FreeBSD cpus. Implicit 8
> builders allowed in parallel.  ALLOW_MAKE_JOBS=yes was in use but no
> MAKE_JOBS_NUMBER_LIMIT or the like was in use, so: implicit 8 per
> builder allowed. The media in use was (is): U.2 Optane 1.4 TB via a USB3
> adapter.
> 
> 

In order to make the time ratio comparisons/contrasts more reasonable,
the below is based on the 7950X3D building the same port-packages as the
WinDevKit2023 built:

15.0-RELEASE (built first) build time:

[01:18:43] [release-amd64-alt] [2026-04-19_09h54m55s] [committing] Time:
01:18:41
           Queued: 271 Inspected: 0 Ignored: 0 Built: 271 Failed: 0
Skipped: 0 Fetched: 0 Remaining: 0

15.0 vs. 14.3 time ratio: 01:18:41/01:06:52 approx.= 1.177


14.4-RELEASE (built second) build time:

[01:16:20] [release14-amd64-alt] [2026-04-19_11h13m39s] [committing]
Time: 01:16:18
           Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
Skipped: 0 Fetched: 0 Remaining: 0

14.4 vs. 14.3 time ratio: 01:16:18/01:06:52 approx.= 1.147


14.3-RELEASE (built third) build time:

[01:06:53] [release14p3-amd64-alt] [2026-04-19_12h29m59s] [committing]
Time: 01:06:52
           Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
Skipped: 0 Fetched: 0 Remaining: 0


[Note: overall max observed load averages spanning the 3 7950X3D builds:
1min: 96.84, 5min: 61.24, 15min: 51.75 ]


So, for the comparison/contrast based on the same port-packages being built:

7950X3D       15.0 and 14.4 vs. 14.3 ratios:
15.0: 01:18:41/01:06:52 approx.= 1.177
14.4: 01:16:18/01:06:52 approx.= 1.147
So: 1.177 - 1.147 approx.= 0.030
(so 14.4 is a far bigger ratio increment in the sequence: 0.147)
(sequence: 14.3 -> 14.4 -> 15.0)

WinDevKit2023 15.0 and 14.4 vs. 14.3 ratios:
15.0: 09:58:30/08:10:58 approx.= 1.219
14.4: 09:11:36/08:10:58 approx.= 1.121
1.219 - 1.121 approx.= 0.088
(so 14.4 is a bigger ratio increment in the sequence as well: 0.121)
(sequence: 14.3 -> 14.4 -> 15.0)

In both cases, it looks like focusing on what is different in 14.4 vs.
14.3 that is also different in 15.0 vs. 14.3 would be focusing on the
bigger build-time-taking issues, presuming updating both 15.* and 14.*
for whatever helps.

WITH_LLVM_LINK_STATIC_LIBRARIES could well be involved.


-- 
===
Mark Millard
marklmi at yahoo.com