Poor I/O performance and errors on SPARC S7-2

Hello,

I am trying OpenBSD 7.8 in a logical domain (LDOM) on a SPARC S7-2
server. The good news is, OpenBSD is the only BSD that boots, installs,
and gets me to a usable shell on the sun4v platform. The bad news is,
I'm experiencing severe performance problems and I/O errors. To
eliminate the LDOM virtual devices -- in particular the virtual disk
server via the vdsk driver -- I booted OpenBSD on the primary domain and
installed to a physical hard disk, and the performance is still well
below what I'd expect.

I'm curious if others have run into this, or if there are some options I
should be tuning on this hardware.

Summary
-------

After installing, the sign of trouble was when I tried to extract the
src.tar.gz archive into /usr/src so that I could apply and build the
errata. It took approximately 30 minutes, and I got the following
errors:

$ tar xzf /home/mwilson/src.tar.gz                                                
tar: Failed write to file gnu/usr.bin/binutils/gdb/ChangeLog-1999: Input/output error
tar: Failed write to file gnu/usr.bin/gcc/gcc/config/arm/arm.md: Input/output error
tar: Failed write to file gnu/usr.bin/gcc/gcc/config/sh/sh.md: Input/output error

Sometimes on reboots I've even gotten I/O-related errors relinking
libraries:

# cat /usr/share/relink/kernel/GENERIC.MP/relink.log
(SHA256) /bsd: OK
LD="ld" sh makegap.sh 0x00000000
ld -n -T ld.script --warn-common -nopie -o newbsd ${SYSTEM_HEAD} vers.o ${OBJS}
ld: final link failed: Input/output error

Curious if this was a problem with the virtual disk driver, I ended up
putting a spare hard drive in the server, booting the installer on the
"bare metal" (i.e. the primary domain that owns all of the hardware),
installing to the hard disk, and booting OpenBSD from that. I haven't
received any I/O errors in that configuration yet, but the performance
is still poor: extracting the src tarball took 22 minutes.

I have tried both the SP and MP kernels, and there is no appreciable
difference between them.

On an OpenBSD amd64 VM I have, the same tarball extracts in 30 seconds.
In a Linux LDOM on the same SPARC S7-2 server, the tarball extracts in
13 seconds.

Additional Details
------------------

My LDOM configuration for OpenBSD has 7 CPU cores (56 vCPUs with the
8-way SMT of the SPARC S7 processor), 128GB of RAM, and is backed by a
64GB zvol via the virtual disk server running in the primary (Solaris
11.4) domain.

The server has the latest firmware installed, 9.10.10.a.

I installed from the install78.iso CD for sparc64.

On the default multiprocessor kernel,

$ time tar xzf ../src.tar.gz                                                                
    30m44.82s real     0m07.53s user     0m12.53s system

On the single processor kernel,

$ time tar xzf ../src.tar.gz
    30m51.25s real     0m06.56s user     0m09.31s system

For comparison, I have Linux installed in an LDOM with the exact same
configuration with the same backing storage, and the same tarball
extracts in seconds:

linux$ /usr/bin/time -p tar xzf ../src.tar.gz 
real 12.69
user 11.09
sys 7.30

Also for comparison, I have an OpenBSD x86 VM on different hardware and
the performance is acceptable:

amd64$ time tar xzf src.tar.gz                                                           
     0m31.56s real     0m06.70s user     0m12.68s system

I don't know whether to attribute the performance problem to the vdsk
driver or something more fundamental with the kernel running on the S7
processor, so I abandonded the LDOM configuration and booted the
install78.iso CD on the "bare metal" and installed to a physical hard
disk in the system. While the performance is better than in the LDOM,
it's still far from expected:

baremetal$ time tar xzf ../src.tar.gz                                          
    22m06.32s real     0m07.30s user     0m12.05s system

I have not yet had any I/O errors like in the LDOM with the vdsk driver,
but I haven't used it as much in this configuration.

(Very) Summarized dmesg from OpenBSD booting in the LDOM, mostly just
the initial few messages and then the devices loading:

OpenBSD 7.8 (GENERIC.MP) #83: Mon Oct 13 02:26:40 MDT 2025
     [email protected]:/usr/src/sys/arch/sparc64/compile/GENERIC.MP
real mem = 137438953472 (131072MB)
avail mem = 135013384192 (128758MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root: SPARC S7-2
cpu0 at mainbus0: SPARC-S7 (rev 0.0) @ 4266.560 MHz
...
cpu55 at mainbus0: SPARC-S7 (rev 0.0) @ 4266.560 MHz
vbus0 at mainbus0
cbus0 at vbus0
vnet0 at cbus0 chan 0x0: ivec 0x0, 0x1, address 00:14:4f:fb:ee:52
vdsk0 at cbus0 chan 0x2: ivec 0x4, 0x5
scsibus1 at vdsk0: 1 targets
sd0 at scsibus1 targ 0 lun 0: <SUN, Virtual Disk, 1.1>
sd0: 65536MB, 512 bytes/sector, 134217728 sectors
bootpath: /virtual-devices@100,0/channel-devices@200,0/disk@0,0
root on sd0a (22be429df50b8a81.a) swap on sd0b dump on sd0b

At this point, I'm not sure what to look at next to isolate where the
I/O performance problems (and errors, in the case of the couple I see in
the LDOM installation) are coming from.

If anyone has any ideas or insights, I'd appreciate them. Otherwise, at
least let this be informationl data about the current state of the S7
processor.

Thank you,
Matthew