Re: PR 252541: Early kernel panic on RPi4B (Too many early devmatch mappings)

On 2021-Jan-11, at 16:19, Mark Millard <marklmi at yahoo.com> wrote:

> 
> 
> On 2021-Jan-11, at 14:23, Mark Millard <marklmi at yahoo.com> wrote:
> 
> 
>> On 2021-Jan-11, at 12:47, Gordon Bergling <gbe at freebsd.org> wrote:
>> 
>> Hi,
>> 
>> I am currently investigating PR 252541 (Too many early devmatch mappings) [1].
>> 
>> The kernel panic happends on the RPi4B. Has anyone successfully booted a rivision
>> on the RPi4B after:
>> 
>> ---------------------------------------------------------------------------------
>> commit e83fdf8bb391579fa422d34663cd8c1f82a00dc0
>> Author:     Chuck Tuffli <[email protected]>
>> AuthorDate: 2021-01-08 22:36:37 +0000
>> Commit:     Chuck Tuffli <[email protected]>
>> CommitDate: 2021-01-08 22:41:45 +0000
>> 
>> fix big-endian platforms after 6733401935f8
>> 
>> The NVMe byte-swap routines for big-endian platforms used memcpy() to
>> move the unaligned 64-bit value into a temp register to byte swap it.
>> Instead of introducing a dependency, manually byte-swap the values in
>> place.
>> ---------------------------------------------------------------------------------
>> 
>> --Gordon
>> 
>> [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252541
>> 
>> I do my own builds but I'm running based on 19cca0b9613d
>> with CommitDate 2021-01-09 16:21:33 -0800 :
>> 
>> # ~/fbsd-based-on-what-freebsd-main.sh mm-src
>> 19cca0b9613d7c3058e41baf0204245119732235
>> CommitDate: 2021-01-09 16:21:33 -0800
>> 5d333ee67ac3 19cca0b9613d (HEAD -> mm-src) mm-src snapshot for mm's patched build in git context.
>> FreeBSD RPi4B 13.0-CURRENT FreeBSD 13.0-CURRENT mm-src-c255807-g5d333ee67ac3 GENERIC-NODBG  arm64 aarch64 1300134 1300134
>> 
>> In other words, the history spanned for e83fdf8bb391 and
>> after is:
>> 
>> * 	aio: fix the tests when ZFS is not available	Alan Somers	46 hours	2	-0/+5
>> * 	linuxkpi: Fix the "error: unknown type name 'u32'" compilation issue when	Neel Chauhan	47 hours	1	-0/+1
>> * 	netmap: vtnet: stop krings during interface reset	Vincenzo Maffione	48 hours	1	-1/+7
>> * 	netmap: refactor netmap_reset	Vincenzo Maffione	2 days	1	-45/+20
>> * 	netmap: iflib: fix asserts in netmap_fl_refill()	Vincenzo Maffione	2 days	1	-1/+2
>> * 	netmap: iflib: stop krings during interface reset	Vincenzo Maffione	2 days	2	-1/+10
>> * 	fileargs: add tests	Mariusz Zaborski	2 days	3	-0/+625
>> * 	tcp: don't use KTLS socket option on listening sockets	Michael Tuexen	4 days	1	-0/+10
>> * 	arm: revert MAXDSIZ change from 202aea9c82ea	Kyle Evans	2 days	1	-1/+1
>> * 	kevent(2): Bugfix for wrong EVFILT_TIMER timeouts	Jan Kokemüller	2 days	1	-1/+1
>> * 	ldd: renumber executable type constants	Ed Maste	2 days	1	-2/+2
>> * 	diff: honour flags with -q	Ed Maste	2 days	2	-1/+13
>> * 	sysctl: improve debug.kdb.panic_str description	Warner Losh	2 days	1	-1/+1
>> * 	last(1): Add EXAMPLES section	Fernando Apesteguía	2 days	1	-4/+22
>> * 	man(1): Bump .Dd	Fernando Apesteguía	2 days	1	-1/+1
>> * 	man(1): Add EXAMPLES section	Fernando Apesteguía	2 days	1	-0/+35
>> * 	mvneta: Acquire the softc lock before clearing the MIB	Mark Johnston	2 days	1	-0/+2
>> * 	Add fib lookup testing module.	Alexander V. Chernikov	2 days	2	-0/+548
>> * 	Bring DPDK route lookups to FreeBSD.	Alexander V. Chernikov	2 days	17	-0/+6030
>> * 	Fix LINT kernel build after 01f2e864f79584c0cd250a8e7cfb501a9985768a.	Hans Petter Selasky	3 days	1	-1/+4
>> * 	certctl: factor out certname resolution	Kyle Evans	3 days	1	-2/+17
>> * 	certctl: replace hardcoded uses of /usr/local	Kyle Evans	3 days	1	-2/+3
>> * 	fix big-endian platforms after 6733401935f8	Chuck Tuffli	3 days	1	-5/+9
>> 
>> The RPi4B is a 8 GiByte one, booted directly from a USB3 SSD,
>> no microsd card involved. I can boot either u-boot style or
>> UEFI/ACPI style from the same media, just switching config.txt
>> content.
>> 
>> I do not have MMCCAM or the like:
>> 
>> # more /usr/fbsd/mm-src/sys/arm64/conf/GENERIC-NODBG 
>> #
>> # GENERIC -- Custom configuration for the arm64/aarch64
>> #
>> 
>> include "GENERIC"
>> 
>> ident   GENERIC-NODBG
>> 
>> makeoptions     DEBUG=-g                # Build kernel with gdb(1) debug symbols
>> 
>> options         ALT_BREAK_TO_DEBUGGER
>> 
>> options         KDB                     # Enable kernel debugger support
>> 
>> # For minimum debugger support (stable branch) use:
>> #options        KDB_TRACE               # Print a stack trace for a panic
>> options         DDB                     # Enable the kernel debugger
>> 
>> # Extra stuff:
>> #options        VERBOSE_SYSINIT=0       # Enable verbose sysinit messages
>> #options        BOOTVERBOSE=1
>> #options        BOOTHOWTO=RB_VERBOSE
>> #options        KTR
>> #options        KTR_MASK=KTR_TRAP
>> ##options       KTR_CPUMASK=0xF
>> #options        KTR_VERBOSE
>> 
>> # Disable any extra checking for. . .
>> nooptions       DEADLKRES               # Enable the deadlock resolver
>> nooptions       INVARIANTS              # Enable calls of extra sanity checking
>> nooptions       INVARIANT_SUPPORT       # Extra sanity checks of internal structures, required by INVARIANTS
>> nooptions       WITNESS                 # Enable checks to detect deadlocks and cycles
>> nooptions       WITNESS_SKIPSPIN        # Don't run witness on spinlocks for speed
>> nooptions       DIAGNOSTIC
>> nooptions       MALLOC_DEBUG_MAXZONES   # Separate malloc(9) zones
>> nooptions       BUF_TRACKING
>> nooptions       FULL_BUF_TRACKING
>> 
> 
> Looks like the message is from a KASSERT that does nothing
> unless INVARIANTS/INVARIANT_SUPPORT is enabled.
> 
> Unfortunately, arftifacts.ci.freebsd.org has not started
> getting git-based main builds yet. Normally I'd support
> an official debug kernel from there and see if I could
> repeat the problem.
> 
> So I've made my own debug kernel build for things as they
> are in my context and it reproduced the problem:
> 
> Hit [Enter] to boot immediately, or any other key for command prompt.
> Booting [/boot/kernel/kernel]...               
> Using DTB provided by EFI at 0x7ef0000.
> EFI framebuffer information:
> addr, size     0x3e2fe000, 0x7e9000
> dimensions     1920 x 1080
> stride         1920
> masks          0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000
> ---<<BOOT>>---
> panic: Too many early devmap mappings
> cpuid = 0
> time = 1
> KDB: stack backtrace:
> (null)() at 0xffff000000116980
>         pc = 0xffff000000772af4  lr = 0xffff000000116980
>         sp = 0xffff0000011f1320  fp = 0xffff0000011f1520
> 
> (null)() at 0xffff000000464710
>         pc = 0xffff000000116980  lr = 0xffff000000464710
>         sp = 0xffff0000011f1530  fp = 0xffff0000011f1590
> 
> (null)() at 0xffff0000004644b4
>         pc = 0xffff000000464710  lr = 0xffff0000004644b4
>         sp = 0xffff0000011f15a0  fp = 0xffff0000011f1650
> 
> (null)() at 0xffff0000007e9838
>         pc = 0xffff0000004644b4  lr = 0xffff0000007e9838
>         sp = 0xffff0000011f1660  fp = 0xffff0000011f1660
> 
> (null)() at 0xffff00000076f744
>         pc = 0xffff0000007e9838  lr = 0xffff00000076f744
>         sp = 0xffff0000011f1670  fp = 0xffff0000011f1690
> 
> (null)() at 0xffff000000782904
>         pc = 0xffff00000076f744  lr = 0xffff000000782904
>         sp = 0xffff0000011f16a0  fp = 0xffff0000011f16c0
> 
> (null)() at 0xffff0000002896b0
>         pc = 0xffff000000782904  lr = 0xffff0000002896b0
>         sp = 0xffff0000011f16d0  fp = 0xffff0000011f1790
> 
> (null)() at 0xffff0000007d9bb0
>         pc = 0xffff0000002896b0  lr = 0xffff0000007d9bb0
>         sp = 0xffff0000011f17a0  fp = 0xffff0000011f1820
> 
> (null)() at 0xffff00000028b814
>         pc = 0xffff0000007d9bb0  lr = 0xffff00000028b814
>         sp = 0xffff0000011f1830  fp = 0xffff0000011f1840
> 
> (null)() at 0xffff00000039e448
>         pc = 0xffff00000028b814  lr = 0xffff00000039e448
>         sp = 0xffff0000011f1850  fp = 0xffff0000011f1870
> 
> (null)() at 0xffff0000004af2ac
>         pc = 0xffff00000039e448  lr = 0xffff0000004af2ac
>         sp = 0xffff0000011f1880  fp = 0xffff0000011f18b0
> 
> (null)() at 0xffff00000077ef90
>         pc = 0xffff0000004af2ac  lr = 0xffff00000077ef90
>         sp = 0xffff0000011f18c0  fp = 0xffff0000011f1a00
> 
> (null)() at 0xffff00000000089c
>         pc = 0xffff00000077ef90  lr = 0xffff00000000089c
>         sp = 0xffff0000011f1a10  fp = 0x0000000000000000
> 
> KDB: enter: panic
> [ thread pid 0 tid 0 ]
> Stopped at      0xffff0000004aeeb4
> db> dump
> Cannot dump: no dump device specified.
> db> 


I stuck in some printf's showing figures in hexadecimal:

. . .
---<<BOOT>>---
pmap_mapdev early_boot: akva_devmap_vaddr: ffff007fff816000 size: 1000
pmap_mapdev early_boot: va: ffff007fff815000 VM_MAX_KERNEL_ADDRESS: ffff008000000000 L2_SIZE: 200000
panic: Too many early devmap mappings
cpuid = 0
. . .

For reference:

#if defined(__aarch64__) || defined(__riscv)
        if (early_boot) {
printf("pmap_mapdev early_boot: akva_devmap_vaddr: %jx size: %jx\n",
(uintmax_t) akva_devmap_vaddr, (uintmax_t) size);
                akva_devmap_vaddr = trunc_page(akva_devmap_vaddr - size);
                va = akva_devmap_vaddr;
printf("pmap_mapdev early_boot: va: %jx VM_MAX_KERNEL_ADDRESS: %jx L2_SIZE: %jx\n",
(uintmax_t) va, (uintmax_t) VM_MAX_KERNEL_ADDRESS, (uintmax_t) L2_SIZE);
                KASSERT(va >= VM_MAX_KERNEL_ADDRESS - L2_SIZE,
                    ("Too many early devmap mappings"));
        } else
#endif

So (hexadecimal):

VM_MAX_KERNEL_ADDRESS - L2_SIZE == ffff007fffe00000

and so va < VM_MAX_KERNEL_ADDRESS - L2_SIZE:

ffff007fff815000 < ffff007fffe00000

by:

ffff007fffe00000-ffff007fff815000 == 5eb000

I've not done anything to track down a relationship to
e83fdf8bb391 .

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[email protected]"