Re: Running armv7 on aarch64

On 2022-Oct-18, at 09:53, Kristof Provost <[email protected]> wrote:

> I’ve recently discovered that I can no longer run an armv7 binary on my aarch64 FreeBSD machine.
> 
> $ /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id
> ELF binary type "9" not known.
> /bin/sh: /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id: Exec format error
> $ file /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id
> /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id: ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, for FreeBSD 14.0 (1400066), stripped
> $ readelf -e /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id
> 
> File: /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id
> ELF Header:
>   Magic:   7f 45 4c 46 01 01 01 09 00 00 00 00 00 00 00 00
>   Class:                             ELF32
>   Data:                              2's complement, little endian
>   Version:                           1 (current)
>   OS/ABI:                            FreeBSD
>   ABI Version:                       0
>   Type:                              EXEC (Executable file)
>   Machine:                           ARM
>   Version:                           0x1
>   Entry point address:               0x20ef0
>   Start of program headers:          52 (bytes into file)
>   Start of section headers:          8912 (bytes into file)
>   Flags:                             0x5000400, Version5 EABI, VFP
>   Size of this header:               52 (bytes)
>   Size of program headers:           32 (bytes)
>   Number of program headers:         11
>   Size of section headers:           40 (bytes)
>   Number of section headers:         28
>   Section header string table index: 27
> 
> Elf file type is EXEC (Executable file)
> Entry point 0x20ef0
> There are 11 program headers, starting at offset 52
> 
> Program Headers:
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   PHDR           0x000034 0x00010034 0x00010034 0x00160 0x00160 R   0x4
>   INTERP         0x000194 0x00010194 0x00010194 0x00015 0x00015 R   0x1
>       [Requesting program interpreter: /libexec/ld-elf.so.1]
>   LOAD           0x000000 0x00010000 0x00010000 0x00ccc 0x00ccc R   0x10000
>   LOAD           0x000ccc 0x00020ccc 0x00020ccc 0x01294 0x01294 R E 0x10000
>   LOAD           0x001f60 0x00031f60 0x00031f60 0x000c8 0x000c8 RW  0x10000
>   LOAD           0x002028 0x00042028 0x00042028 0x000a0 0x00138 RW  0x10000
>   DYNAMIC        0x001f68 0x00031f68 0x00031f68 0x000c0 0x000c0 RW  0x4
>   GNU_RELRO      0x001f60 0x00031f60 0x00031f60 0x000c8 0x000c8 R   0x1
>   GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0
>   NOTE           0x0001ac 0x000101ac 0x000101ac 0x00064 0x00064 R   0x4
>   ARM_EXIDX      0x00090c 0x0001090c 0x0001090c 0x00068 0x00068 R   0x4
> 
>  Section to Segment mapping:
>   Segment Sections...
>    00
>    01     .interp
>    02     .interp .note.tag .dynsym .gnu.version .gnu.version_r .gnu.hash .hash .dynstr .rel.dyn .ARM.exidx .rel.plt .rodata .ARM.extab
>    03     .text .init .fini .plt
>    04     .jcr .init_array .dynamic
>    05     .data .got.plt .bss
>    06     .dynamic
>    07     .jcr .init_array .dynamic
>    08
>    09     .note.tag
>    10     .ARM.exidx
> There are 28 section headers, starting at offset 0x22d0:
> 
> Section Headers:
>   [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
>   [ 0]                   NULL            00000000 000000 000000 00      0   0  0
>   [ 1] .interp           PROGBITS        00010194 000194 000015 00   A  0   0  1
>   [ 2] .note.tag         NOTE            000101ac 0001ac 000064 00   A  0   0  4
>   [ 3] .dynsym           DYNSYM          00010210 000210 0002c0 10   A  8   1  4
>   [ 4] .gnu.version      SUNW_versym     000104d0 0004d0 000058 02   A  3   0  2
>   [ 5] .gnu.version_r    SUNW_verneed    00010528 000528 000050 00   A  8   2  4
>   [ 6] .gnu.hash         GNU_HASH        00010578 000578 000030 00   A  3   0  4
>   [ 7] .hash             HASH            000105a8 0005a8 000168 04   A  3   0  4
>   [ 8] .dynstr           STRTAB          00010710 000710 0001e3 00   A  0   0  1
>   [ 9] .rel.dyn          REL             000108f4 0008f4 000018 08  AI  3   0  4
>   [10] .ARM.exidx        ARM_EXIDX       0001090c 00090c 000068 00   A 14   0  4
>   [11] .rel.plt          REL             00010974 000974 000120 08  AI  3  22  4
>   [12] .rodata           PROGBITS        00010a94 000a94 00021f 01 AMS  0   0  1
>   [13] .ARM.extab        PROGBITS        00010cb4 000cb4 000018 00   A  0   0  4
>   [14] .text             PROGBITS        00020ccc 000ccc 000ff0 00  AX  0   0  4
>   [15] .init             PROGBITS        00021cc0 001cc0 000014 00  AX  0   0 16
>   [16] .fini             PROGBITS        00021ce0 001ce0 000014 00  AX  0   0 16
>   [17] .plt              PROGBITS        00021d00 001d00 000260 00  AX  0   0 16
>   [18] .jcr              PROGBITS        00031f60 001f60 000004 00  WA  0   0  4
>   [19] .init_array       INIT_ARRAY      00031f64 001f64 000004 00  WA  0   0  4
>   [20] .dynamic          DYNAMIC         00031f68 001f68 0000c0 08  WA  8   0  4
>   [21] .data             PROGBITS        00042028 002028 000004 00  WA  0   0  4
>   [22] .got.plt          PROGBITS        0004202c 00202c 00009c 00  WA  0   0  4
>   [23] .bss              NOBITS          00042100 0020c8 000060 00  WA  0   0 64
>   [24] .comment          PROGBITS        00000000 0020c8 0000b6 01  MS  0   0  1
>   [25] .ARM.attributes   ARM_ATTRIBUTES  00000000 00217e 000049 00      0   0  1
>   [26] .gnu_debuglink    PROGBITS        00000000 0021c7 000010 00      0   0  1
>   [27] .shstrtab         STRTAB          00000000 0021d7 0000f7 00      0   0  1
> Key to Flags:
>   W (write), A (alloc), X (execute), M (merge), S (strings)
>   I (info), L (link order), G (group), x (unknown)
>   O (extra OS processing required) o (OS specific), p (processor specific)
> 
> It’s not quite clear to me how this is supposed to work (now). On amd64 there’s a separate /libexec/ld-elf32.so.1, which we don’t have on aarch64. Is it supposed to be built?
> 
> It’s broken on ab9293239c7d and e03b7883e97c at the very least.

[I'm ignoring qemu, which I do not use. The below is from
a Cortex-A72 aarch64 context that can execute Cortex-A7
armv7 code as well. Have you been using qemu?]

Historically I've only been able to execute armv7 FreeBSD
code on aarch64 FreeBSD via using the likes of, say,
chroot'ing into an installed armv7 world in a directory
tree that I created for such. (I manually split some
liong-lineouptut for readabilty.)

# uname -apKU
FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #63
main-n258610-ba7319e9091b-dirty: Fri Oct 14 14:29:14 PDT 2022
root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72
arm64 aarch64 1400072 1400072

# chroot /usr/obj/DESTDIRs/main-CA7-chroot/

# uname -apKU
FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #63
main-n258610-ba7319e9091b-dirty: Fri Oct 14 14:29:14 PDT 2022
root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72
arm armv7 1400072 1400072

# which date
/bin/date

# file /bin/date
/bin/date: ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, for FreeBSD 14.0 (1400072), not stripped

# date
Tue Oct 18 17:28:13 UTC 2022


Direct execution attempts from an aarch64 world without such a
chroot (or equivalent for the purpose) to a armv7 world have
always produced the likes of:

# /usr/obj/DESTDIRs/main-CA7-chroot/bin/date
ELF interpreter /libexec/ld-elf.so.1 not found, error 8
Abort trap



===
Mark Millard
marklmi at yahoo.com