Re: Still seeing Failed assertion: "p[i] == 0" [ *** i386 chroot on amd64 can get the failure on main 16 too *** ]

[lldb works for that i386 core dump.]

On Nov 20, 2025, at 16:03, Mark Millard <[email protected]> wrote:

> On Nov 20, 2025, at 15:38, Mark Millard <[email protected]> wrote:
> 
>> # cd /usr/src/
>> 
>> # env WITH_META_MODE= make -j10 buildworld
>> . . .
>> Building /usr/obj/usr/src/i386.i386/lib/clang/libclang/CodeGen/CGExprScalar.pico
>> <jemalloc>: /usr/src/contrib/jemalloc/include/jemalloc/internal/ehooks.h:170: Failed assertion: "p[i] == 0"
>> Building /usr/obj/usr/src/i386.i386/lib/clang/libclang/CodeGen/CGGPUBuiltin.pico
>> Abort trap (core dumped)
>> *** [CodeGen/CGBuiltin.pico] Error code 134
>> 
>> The 7950X3D system has 32 FreeBSD cpus (16 cores), 192 GiByTes of RAM,
> 
> [Silly typo fixed: GiBytes]
> 
>> Optane PCIe Storage media.
>> 
>> # uname -apKU
>> FreeBSD 7950X3D-UFS 16.0-CURRENT FreeBSD 16.0-CURRENT main-n281922-4872b48b175c GENERIC-NODEBUG amd64 amd64 1600004 1600004
>> 
>> But in the chroot:
>> 
>> # uname -apKU
>> FreeBSD 7950X3D-UFS 16.0-CURRENT FreeBSD 16.0-CURRENT main-n281922-4872b48b175c GENERIC-NODEBUG i386 i386 1600004 1600004
>> 
>> For that particular system I used:
>> 
>> # grep physmem /boot/loader.conf
>> hw.physmem="5G"
>> 
>> It resulted in:
>> 
>> # dmesg -a | grep 'memory *='
>> real memory  = 137434759168 (131068 MB)
>> avail memory = 2708901888 (2583 MB)
>> 
>> My odd top hack for monitoring and reporting
>> various "Max(imum) Obs(erved)" figures shows:
>> 
>> 1914Mi   MaxObsActive
>> 821116Ki MaxObsWired
>> 2585Mi   MaxObs(Act+Wir+Lndry)
>> 1617Mi   MaxObsSwapUsed,
>> 3488Mi   MaxObs(Act+Lndry+SwapUsed),
>> 4166Mi   MaxObs(A+Wir+L+SU) [at such a time: 4193Mi (A+W+L+SU+InAct)]
>> 
>> I had to build main 16 for the i386 myself, based on the /usr/src/
>> from the official pkgbase system in use: such is no longer available.
>> I did not make changes to allow SIGABRT to core dump.
> 
> Dumb mistake: I *did* make changes to allow SIGABRT to core dump.
> (I had done so earlier when figuring out how to do that. I
> had left the source code change in place on that system and
> it was in the copy of /usr/src/ that I put into the chroot
> area.)
> 
> But I do not have pkg or gdb for the context as stands.
> 
>> -j8 did not fail. -j12 ran out of RAM+SWAP.
>> 
> 
> For reference (white space detail may not be preserved):
> 
> # diff -u /usr/src/contrib/llvm-project/llvm/lib/Support/CrashRecoveryContext.cpp.orig /usr/src/contrib/llvm-project/llvm/lib/Support/CrashRecoveryContext.cpp
> --- /usr/src/contrib/llvm-project/llvm/lib/Support/CrashRecoveryContext.cpp.orig 2025-11-15 07:58:41.000000000 +0000
> +++ /usr/src/contrib/llvm-project/llvm/lib/Support/CrashRecoveryContext.cpp 2025-11-18 01:26:50.661412000 +0000
> @@ -345,7 +345,7 @@
> #include <signal.h>
> 
> static const int Signals[] =
> -    { SIGABRT, SIGBUS, SIGFPE, SIGILL, SIGSEGV, SIGTRAP };
> +    { /*SIGABRT,*/ SIGBUS, SIGFPE, SIGILL, SIGSEGV, SIGTRAP };
> static const unsigned NumSignals = std::size(Signals);
> static struct sigaction PrevActions[NumSignals];
> 
> 
> # diff -u /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc.orig /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc
> --- /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc.orig 2025-11-15 07:58:41.000000000 +0000
> +++ /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc 2025-11-18 15:37:06.502424000 +0000
> @@ -215,7 +215,7 @@
> /// been ordered.
> static const int KillSigs[] = {SIGILL,
>                                SIGTRAP,
> -                               SIGABRT,
> +//                               SIGABRT,
>                                SIGFPE,
>                                SIGBUS,
>                                SIGSEGV,
> 

I'm used to lldb not working for armv7 that I did not
even think to try it for i386 until now. It works for
doing bt for i386. But the ehooks_debug_zero_check
call has addr=<unavailable> :

(lldb) bt
* thread #1, name = 'c++', stop reason = signal SIGABRT
  * frame #0: 0x24bd378b libsys.so.7`__sys_thr_kill at thr_kill.S:4
    frame #1: 0x2ab2231b libc.so.7`__raise(s=6) at raise.c:48:10
    frame #2: 0x2abcc73b libc.so.7`abort at abort.c:61:8
    frame #3: 0x2ac10440 libc.so.7`ehooks_debug_zero_check(addr=<unavailable>, size=<unavailable>) at ehooks.h:0
    frame #4: 0x2ac0ccfa libc.so.7`__je_extent_alloc_wrapper [inlined] ehooks_alloc(tsdn=<unavailable>, ehooks=<unavailable>, new_addr=<unavailable>, size=<unavailable>, alignment=<unavailable>, zero=0xffff917b, commit=<unavailable>) at ehooks.h:208:3
    frame #5: 0x2ac0cc5a libc.so.7`__je_extent_alloc_wrapper(tsdn=0x2adf90a0, pac=0x2b0015e4, ehooks=0x2b000080, new_addr=0x00000000, size=167936, alignment=64, zero=true, commit=0xffff91e3, growing_retained=<unavailable>) at jemalloc_extent.c:1003:15
    frame #6: 0x2ac0c58e libc.so.7`__je_ecache_alloc_grow(tsdn=0x2adf90a0, pac=0x2b0015e4, ehooks=0x2b000080, ecache=0x2b0036d0, expand_edata=0x00000000, size=167936, alignment=64, zero=<unavailable>, guarded=<unavailable>) at jemalloc_extent.c:126:11
    frame #7: 0x2ac3c73f libc.so.7`pac_alloc_impl [inlined] pac_alloc_real(tsdn=<unavailable>, pac=<unavailable>, ehooks=<unavailable>, size=<unavailable>, alignment=64, zero=<unavailable>, guarded=<unavailable>) at jemalloc_pac.c:124:11
    frame #8: 0x2ac3c696 libc.so.7`pac_alloc_impl(tsdn=0x2adf90a0, self=0x2b0015e4, size=167936, alignment=64, zero=<unavailable>, guarded=<unavailable>, frequent_reuse=<unavailable>, deferred_work_generated=<unavailable>) at jemalloc_pac.c:178:11
    frame #9: 0x2ac3afe0 libc.so.7`__je_pa_alloc [inlined] pai_alloc(tsdn=<unavailable>, self=<unavailable>, size=<unavailable>, alignment=<unavailable>, zero=<unavailable>, guarded=<unavailable>, frequent_reuse=<unavailable>, deferred_work_generated=<unavailable>) at pai.h:43:9
    frame #10: 0x2ac3afc9 libc.so.7`__je_pa_alloc(tsdn=0x2adf90a0, shard=0x2b0015d8, size=167936, alignment=64, slab=<unavailable>, szind=49, zero=<unavailable>, guarded=<unavailable>, deferred_work_generated=0xffff92cf) at jemalloc_pa.c:139:11
    frame #11: 0x2abea7a8 libc.so.7`__je_arena_extent_alloc_large(tsdn=0x2adf90a0, arena=0x2b000500, usize=163840, alignment=64, zero=<unavailable>) at jemalloc_arena.c:338:19
    frame #12: 0x2ac117ea libc.so.7`__je_large_palloc(tsdn=0x2adf90a0, arena=<unavailable>, usize=163840, alignment=64, zero=<unavailable>) at jemalloc_large.c:37:42
    frame #13: 0x2ac11342 libc.so.7`__je_large_malloc(tsdn=0x2adf90a0, arena=0x2b000500, usize=163840, zero=<unavailable>) at jemalloc_large.c:17:9
    frame #14: 0x2abed45e libc.so.7`__je_arena_malloc_hard(tsdn=0x2adf90a0, arena=<unavailable>, size=131576, ind=49, zero=<unavailable>) at jemalloc_arena.c:1205:9
    frame #15: 0x2abe1cb0 libc.so.7`iallocztm [inlined] arena_malloc(tsdn=<unavailable>, arena=<unavailable>, size=<unavailable>, ind=<unavailable>, zero=<unavailable>, tcache=<unavailable>, slow_path=<unavailable>) at arena_inlines_b.h:0
    frame #16: 0x2abe1c52 libc.so.7`iallocztm(tsdn=<unavailable>, size=<unavailable>, ind=49, zero=<unavailable>, tcache=0x2adf92f0, is_internal=<unavailable>, arena=0x00000000, slow_path=<unavailable>) at jemalloc_internal_inlines_c.h:55:8
    frame #17: 0x2abe7179 libc.so.7`imalloc_body [inlined] imalloc_no_sample(sopts=0xffff946c, dopts=0xffff944c, tsd=0x2adf90a0, size=131576, usize=163840, ind=49) at jemalloc_jemalloc.c:2402:9
    frame #18: 0x2abe7106 libc.so.7`imalloc_body(sopts=<unavailable>, dopts=<unavailable>, tsd=0x2adf90a0) at jemalloc_jemalloc.c:2577:16
    frame #19: 0x2abdae88 libc.so.7`imalloc(sopts=<unavailable>, dopts=<unavailable>) at tsd.h:0:2
    frame #20: 0x2abdaddc libc.so.7`__je_malloc_default(size=131576) at jemalloc_jemalloc.c:2726:2
    frame #21: 0x2abdb27f libc.so.7`__malloc [inlined] imalloc_fastpath(size=131576, fallback_alloc=<unavailable>) at jemalloc_internal_inlines_c.h:0:2
    frame #22: 0x2abdb149 libc.so.7`__malloc(size=131576) at jemalloc_jemalloc.c:2750:9
    frame #23: 0x28aa2b52 libprivatellvm.so.19`::mallocForGrow() [inlined] safe_malloc at MemAlloc.h:26:18
    frame #24: 0x28aa2b4c libprivatellvm.so.19`::mallocForGrow() at SmallVector.cpp:130:19
    frame #25: 0x2798f6dd libprivatellvm.so.19`::grow() [inlined] mallocForGrow at SmallVector.h:458:48
    frame #26: 0x2798f6c9 libprivatellvm.so.19`::grow() at SmallVector.h:449:16
    frame #27: 0x2798fa76 libprivatellvm.so.19`::resize() [inlined] reserveForParamAndGetAddressImpl<llvm::SmallVectorTemplateBase<std::__1::pair<llvm::PointerUnion<const llvm::TargetRegisterClass *, const llvm::RegisterBank *>, llvm::MachineOperand *>, false> > at SmallVector.h:257:11
    frame #28: 0x2798fa5c libprivatellvm.so.19`::resize() [inlined] reserveForParamAndGetAddress at SmallVector.h:392:12
    frame #29: 0x2798fa5c libprivatellvm.so.19`::resize() [inlined] append at SmallVector.h:707:29
    frame #30: 0x2798fa5c libprivatellvm.so.19`::resize() at SmallVector.h:674:11
    frame #31: 0x2798c39e libprivatellvm.so.19`::createVirtualRegister() [inlined] resize at IndexedMap.h:62:16
    frame #32: 0x2798c393 libprivatellvm.so.19`::createVirtualRegister() [inlined] grow at IndexedMap.h:72:9
    frame #33: 0x2798c390 libprivatellvm.so.19`::createVirtualRegister() [inlined] createIncompleteVirtualRegister at MachineRegisterInfo.cpp:149:12
    frame #34: 0x2798c38d libprivatellvm.so.19`::createVirtualRegister() at MachineRegisterInfo.cpp:166:18
    frame #35: 0x27c07b5f libprivatellvm.so.19`::EmitCopyFromReg() at InstrEmitter.cpp:173:19
    frame #36: 0x27c0e366 libprivatellvm.so.19`::EmitSpecialNode() at InstrEmitter.cpp:1278:5
    frame #37: 0x27d2707e libprivatellvm.so.19`::operator()() [inlined] EmitNode at InstrEmitter.h:147:7
    frame #38: 0x27d27064 libprivatellvm.so.19`::operator()() at ScheduleDAGSDNodes.cpp:874:13
    frame #39: 0x27d266c1 libprivatellvm.so.19`::EmitSchedule() at ScheduleDAGSDNodes.cpp:961:9
    frame #40: 0x27e0ad27 libprivatellvm.so.19`::CodeGenAndEmitDAG() at SelectionDAGISel.cpp:1126:42
    frame #41: 0x27e08f06 libprivatellvm.so.19`::SelectBasicBlock() at SelectionDAGISel.cpp:850:3
    frame #42: 0x27e08310 libprivatellvm.so.19`::SelectAllBasicBlocks() at SelectionDAGISel.cpp:1863:7
    frame #43: 0x27e056de libprivatellvm.so.19`::runOnMachineFunction() at SelectionDAGISel.cpp:631:3
    frame #44: 0x2964affa libprivatellvm.so.19`::runOnMachineFunction() at X86ISelDAGToDAG.cpp:190:32
    frame #45: 0x27e0377d libprivatellvm.so.19`::runOnMachineFunction() at SelectionDAGISel.cpp:374:20
    frame #46: 0x278de5f1 libprivatellvm.so.19`::runOnFunction() at MachineFunctionPass.cpp:94:13
    frame #47: 0x283716f2 libprivatellvm.so.19`::runOnFunction() at LegacyPassManager.cpp:1440:27
    frame #48: 0x28378063 libprivatellvm.so.19`::runOnModule() at LegacyPassManager.cpp:1486:16
    frame #49: 0x28371ee4 libprivatellvm.so.19`::run() [inlined] runOnModule at LegacyPassManager.cpp:1555:27
    frame #50: 0x28371b90 libprivatellvm.so.19`::run() at LegacyPassManager.cpp:541:44
    frame #51: 0x283784ee libprivatellvm.so.19`llvm::legacy::PassManager::run(llvm::Module&) at LegacyPassManager.cpp:1682:14
    frame #52: 0x228e059a libprivateclang.so.19`::EmitBackendOutput() [inlined] RunCodegenPipeline at BackendUtil.cpp:1157:19
    frame #53: 0x228e0589 libprivateclang.so.19`::EmitBackendOutput() [inlined] EmitAssembly at BackendUtil.cpp:1180:3
    frame #54: 0x228e0589 libprivateclang.so.19`::EmitBackendOutput() at BackendUtil.cpp:1341:13
    frame #55: 0x22d84b4d libprivateclang.so.19`::HandleTranslationUnit() at CodeGenAction.cpp:354:3
    frame #56: 0x234f63bd libprivateclang.so.19`::ParseAST() at ParseAST.cpp:184:13
    frame #57: 0x2335bdef libprivateclang.so.19`::ExecuteAction() at FrontendAction.cpp:1192:3
    frame #58: 0x22d8ad1d libprivateclang.so.19`::ExecuteAction() at CodeGenAction.cpp:1144:30
    frame #59: 0x2335b650 libprivateclang.so.19`::Execute() at FrontendAction.cpp:1078:8
    frame #60: 0x232bc3ac libprivateclang.so.19`::ExecuteAction() at CompilerInstance.cpp:1061:33
    frame #61: 0x233f3b7c libprivateclang.so.19`::ExecuteCompilerInvocation() at ExecuteCompilerInvocation.cpp:280:25
    frame #62: 0x0040df8f c++`::cc1_main() at cc1_main.cpp:284:15
    frame #63: 0x0041c927 c++`::ExecuteCC1Tool() at driver.cpp:215:12
    frame #64: 0x22f460a6 libprivateclang.so.19`::callback_fn<(lambda at /usr/src/contrib/llvm-project/clang/lib/Driver/Job.cpp:440:22)>() [inlined] operator() at STLFunctionalExtras.h:68:12
    frame #65: 0x22f46097 libprivateclang.so.19`::callback_fn<(lambda at /usr/src/contrib/llvm-project/clang/lib/Driver/Job.cpp:440:22)>() [inlined] operator() at Job.cpp:440:34
    frame #66: 0x22f46094 libprivateclang.so.19`::callback_fn<(lambda at /usr/src/contrib/llvm-project/clang/lib/Driver/Job.cpp:440:22)>() at STLFunctionalExtras.h:45:12
    frame #67: 0x28a3f834 libprivatellvm.so.19`::RunSafely() [inlined] operator() at STLFunctionalExtras.h:68:12
    frame #68: 0x28a3f82f libprivatellvm.so.19`::RunSafely() at CrashRecoveryContext.cpp:426:3
    frame #69: 0x22f456d4 libprivateclang.so.19`::Execute() at Job.cpp:440:12
    frame #70: 0x22f00688 libprivateclang.so.19`::ExecuteCommand() at Compilation.cpp:199:15
    frame #71: 0x22f0097e libprivateclang.so.19`::ExecuteJobs() at Compilation.cpp:253:19
    frame #72: 0x22f21e9c libprivateclang.so.19`::ExecuteCompilation() at Driver.cpp:1943:5
    frame #73: 0x0041bfdd c++`::clang_main() at driver.cpp:391:21
    frame #74: 0x0041a387 c++`main at clang-driver.cpp:17:10
    frame #75: 0x2aaf8820 libc.so.7`__libc_start1(argc=71, argv=0xffffc2a4, env=0xffffc3c4, cleanup=(ld-elf.so.1`rtld_nop_exit at rtld.c:3602), mainX=(c++`main at clang-driver.cpp:15)) at libc_start1.c:180:7
    frame #76: 0x0040c438 c++`_start at crt1_s.S:84

(I have rarely used lldb. So its commands are not generally
familiar.)

===
Mark Millard
marklmi at yahoo.com