Re: Silent hang in buildworld, was Re: Invoking -v for clang during buildworld
Mark Millard via freebsd-arm <[email protected]>
| Newsgroups | gmane.os.freebsd.devel.arm,gmane.os.freebsd.current |
|---|---|
| Message-ID | <[email protected]> |
On 2021-Jan-18, at 21:12, Mark Millard <marklmi at yahoo.com> wrote:
> On 2021-Jan-18, at 19:19, Mark Millard <marklmi at yahoo.com> wrote:
>
>> . . .
>>> FYI: I re-established my access to a RPi2B V1.1 and made
>>> it report: "maximum recommended amount (468832 pages)"
>>>
>>> (The figure can vary some from release to release.)
>>>
>>> 468832*4096 == 1920335872 or a little over 1831 MiBytes
>>>
>>> For the 4096 Byte pages, that means that the following from
>>> gpart fits without complaint (size is in blocks, not pages):
>>>
>>> 413140992 3686400 da0p2 freebsd-swap (1.8G)
>>>
>>> 3686400*512 is a little over 1.75 GiByte or 1800 MiByte. So
>>> I've left some room below 1831 MiBytes, but not a lot.
>>>
>>> FYI about my build experiment that is running:
>>>
>>> # sysctl hw.physmem
>>> hw.physmem: 979042304
>>>
>>> which, in recent times for armv7, I can (and did) set in
>>> /boot/loader.conf on a faster cortex-A7 SBC (that can boot
>>> the same media but has more RAM).
>>>
>>> So I tried a -j4 build, but with LDFLAGS.lld+= -Wl,--threads=1
>>> in use and my other particular src.conf/make.conf like content
>>> (so the builds do likely differ from yours in various ways).
>>> My build is producing a non-debug build (but with -g symbols).
>>> Somewhat after where your buildworld.log stops, my odd variant
>>> of top was reporting:
>>>
>>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892732Ki MaxObs(Act+Wir)
>>> Swap: . . . , 145832Ki MaxObsUsed
>>>
>>> and top was also showing lots of processes as having "0B" RES
>>> in STATE "wait" or "nanslp" (so, apparently swapped out, not paging).
>>> ("MaxObs" is short for "maximum observed".)
>>>
>>> For comparison, your swapscript.log reported a maximum total of
>>> 346192 KiBytes "Used" for swap, about 98% into the log file.
>>>
>>> (Time goes by . . .)
>>>
>>> It finished with building libllvm and is part way into building
>>> libclang. This is probably well past where your hangup happened,
>>> given that your published buildworldlog file stopped with
>>> libllvm's Target/ARM/ARMMCInstLower.o . My odd top now shows:
>>>
>>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892732Ki MaxObs(Act+Wir)
>>> Swap: . . . , 392328Ki MaxObsUsed
>>>
>>> The build continues to run. I'll let you know how it goes.
>>> . . .
>>
>> Just after libclang finished my odd top showed:
>>
>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 892736Ki MaxObs(Act+Wir)
>> Swap: . . . , 537588Ki MaxObsUsed
>>
>> After liblldb:
>>
>> Mem: . . . , 753672Ki MaxObsActive, 200412Ki MaxObsWired, 899276Ki MaxObs(Act+Wir)
>> Swap: . . . , 537588Ki MaxObsUsed
>>
>> Much later, after the lldb program had been built:
>>
>> Mem: . . . , 765700Ki MaxObsActive, 200412Ki MaxObsWired, 954116Ki MaxObs(Act+Wir)
>> Swap: . . . , 537588Ki MaxObsUsed
>>
>>>>> World build completed on Mon Jan 18 19:10:08 PST 2021
>>>>> World built in 72960 seconds, ncpu: 4, make -j4
>>
>> This was building from scratch what was already installed:
>>
>> # ~/fbsd-based-on-what-freebsd-main.sh
>> merge-base: 818390ce0ca539300dd15d7a817784f1e3f7a9b8
>> merge-base: CommitDate: 2021-01-13 21:27:44 +0000
>> 4180404713ec (HEAD -> mm-src) mm-src snapshot for mm's patched build in git context.
>> 818390ce0ca5 (freebsd/main, freebsd/HEAD, pure-src, main) arm64: fix early devmap assertion
>> FreeBSD OPiP2E_RPi2v11 13.0-CURRENT FreeBSD 13.0-CURRENT mm-src-c255938-g4180404713ec GENERIC-NODBG arm armv7 1300135 1300135
>>
>> This suggests that you should be able to build on the RPi2B v1.1,
>> using -j4, with appropriate configuration for what and how to build.
>>
>>
>> It is now building the matching kernel, my GENERIC-NODBG style.
>
> Done:
>
>>>> Kernel build for GENERIC-NODBG completed on Mon Jan 18 20:33:26 PST 2021
>>>> Kernel(s) GENERIC-NODBG built in 4998 seconds, ncpu: 4, make -j4
>
> So, World+Kernel in somewhat under 22 hours.
>
> The "MaxObs*" figures were unchanged, so:
>
> Mem: . . . , 765700Ki MaxObsActive, 200412Ki MaxObsWired, 954116Ki MaxObs(Act+Wir)
> Swap: . . . , 537588Ki MaxObsUsed
>
> This suggests that, for now, 800 MiByte of swap would be something
> more than 1.5 times what it actually used and 1050 MiBytes would
> be something like 2.0 times what it actually used, so leaving some
> notable margin for variations in peek usage, at least when linker
> threading is avoided.
>
>
>
> As for what I used to control "what and how to build" . . .
>
> # more ~/sys_build_scripts.armv7-host/make_armv7_nodebug_clang_bootstrap-armv7-host.sh
> kldload -n filemon && \
> script ~/sys_typescripts/typescript_make_armv7_nodebug_clang_bootstrap-armv7-host-$(date +%Y-%m-%d:%H:%M:%S) \
> env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" SRC_ENV_CONF="/root/src.configs/src.conf.armv7-clang-bootstrap.armv7-host" \
> WITH_META_MODE=yes \
> WORLD_FLAGS="${WORLD_FLAGS} UBLDR_LOADADDR=0x42000000" \
> MAKEOBJDIRPREFIX="/usr/obj/armv7_clang/arm.armv7" \
> make $*
>
> (In my context, UBLDR_LOADADDR is ignored by anything that
> can not use the figure given. So I've no bothered to be
> more selective about having it in the armv7 builds.)
>
> # more ~/src.configs/make.conf
> LDFLAGS.lld+= -Wl,--threads=1
>
> # more ~/src.configs/src.conf.armv7-clang-bootstrap.armv7-host
> TO_TYPE=armv7
> #
> KERNCONF=GENERIC-NODBG
> TARGET=arm
> .if ${.MAKE.LEVEL} == 0
> TARGET_ARCH=${TO_TYPE}
> .export TARGET_ARCH
> .endif
> #
> #WITH_CROSS_COMPILER=
> WITH_SYSTEM_COMPILER=
> WITH_SYSTEM_LINKER=
> #
> WITH_LIBCPLUSPLUS=
> WITHOUT_BINUTILS_BOOTSTRAP=
> WITH_ELFTOOLCHAIN_BOOTSTRAP=
> #Disables avoiding bootstrap: WITHOUT_LLVM_TARGET_ALL=
> WITHOUT_LLVM_TARGET_AARCH64=
> WITH_LLVM_TARGET_ARM=
> WITHOUT_LLVM_TARGET_MIPS=
> WITHOUT_LLVM_TARGET_POWERPC=
> WITHOUT_LLVM_TARGET_RISCV=
> WITHOUT_LLVM_TARGET_X86=
> WITH_CLANG=
> WITH_CLANG_IS_CC=
> WITH_CLANG_FULL=
> WITH_CLANG_EXTRAS=
> WITH_LLD=
> WITH_LLD_IS_LD=
> WITHOUT_BINUTILS=
> #
> WITH_LLDB=
> #
> WITH_BOOT=
> WITHOUT_LIB32=
> #
> #
> WITHOUT_WERROR=
> #WERROR=
> MALLOC_PRODUCTION=
> WITH_MALLOC_PRODUCTION=
> WITHOUT_ASSERT_DEBUG=
> WITHOUT_LLVM_ASSERTIONS=
> #
> # Avoid stripping but do not control host -g status as well:
> DEBUG_FLAGS+=
> #
> WITH_REPRODUCIBLE_BUILD=
> WITH_DEBUG_FILES=
> #
> # Use of the .clang 's here avoids
> # interfering with other C<?>FLAGS
> # usage, such as ?= usage.
> CFLAGS.clang+= -mcpu=cortex-a7
> CXXFLAGS.clang+= -mcpu=cortex-a7
> CPPFLAGS.clang+= -mcpu=cortex-a7
>
> (I do not claim that you would want WITH_REPRODUCIBLE_BUILD .
> I just happen to have been experimenting with it. You might
> not want to be explicit about the cpu to target. You might
> not want WITH_CLANG_EXTRAS .)
>
> # more /usr/fbsd/mm-src/sys/arm/conf/GENERIC-NODBG
> include "GENERIC"
>
> ident GENERIC-NODBG
>
> makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
>
> options AUDIT # Not enabled by default in armv7/v6 kernels
> # Enabled here to allow kyua test runs to
> # possibly report auditing works.
>
> options ALT_BREAK_TO_DEBUGGER
>
> options KDB # Enable kernel debugger support
>
> # For minimum debugger support (stable branch) use:
> options KDB_TRACE # Print a stack trace for a panic
> options DDB # Enable the kernel debugger
>
> # Extra stuff:
> #options VERBOSE_SYSINIT=0 # Enable verbose sysinit messages
> #options BOOTVERBOSE=1
> #options BOOTHOWTO=RB_VERBOSE
> options ALT_BREAK_TO_DEBUGGER # Enter debugger on keyboard escape sequence
> options KLD_DEBUG
> #options KTR
> #options KTR_MASK=KTR_TRAP
> ##options KTR_CPUMASK=0xF
> #options KTR_VERBOSE
>
> # Disable any extra checking for. . .
> nooptions INVARIANTS # Enable calls of extra sanity checking
> nooptions INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS
> nooptions WITNESS # Enable checks to detect deadlocks and cycles
> nooptions WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed
> nooptions DEADLKRES # Enable the deadlock resolver
> nooptions MALLOC_DEBUG_MAXZONES # Separate malloc(9) zones
> nooptions DIAGNOSTIC
> nooptions BUF_TRACKING
> nooptions FULL_BUF_TRACKING
> nooptions USB_DEBUG
> nooptions USB_REQ_DEBUG
> nooptions USB_VERBOSE
>
> The /boot/loader.conf file and the /etc/sysctl.conf files
> both contained:
>
> vm.pageout_oom_seq=120
> vm.pfault_oom_attempts=-1
>
> (The hw.physmem=979042304 in /boot/loader.conf was very-special,
> to better approximate your environment. I also controlled the
> cpu frequency used via a line in /etc/sysctl.conf . I do not
> bother with such non-default frequency usage [or related settings]
> for RPi*'s with the pre-RPi4B style power connections but do
> control the frequency for the OPi+2E.)
The following had been left implicit about my context and
how it manages memory space use.
I'll note that I do not use tmpfs or other such memory based
file system techniques that could compete for RAM/swap. What
is in use for the only file system involved is just the
root file system:
# df -m
Filesystem 1M-blocks Used Avail Capacity Mounted on
/dev/gpt/BPIM3root 195378 63940 115808 36% /
devfs 0 0 0 100% /dev
It is a USB SSD. The swap partition is also on that same
media. (The BPIM3 based name dates back to before the
BPI-M3 power connection failed and I switched to the
OPi+2E.)
I'll note that I've started a new from-scratch build without
LDFLAGS.lld+= -Wl,--threads=1 . So at some point I'll have
information about how much of a difference (+/-) in swap
usage it actually made for with vs. without, if any.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[email protected]"