Insights from successful, yet painful, install attempt.

Hi port-amiga,

I installed netbsd 9.2 last night. It wasn't smooth. Here's what I learned.

*** hardware
- Amiga 1200.
- Blizzard 1230mkIV with 68030@50MHz, 68882@50MHz, 64MB RAM.
- ~120GB 2.5 HDD.
- 3Com 3c589 PCMCIA NIC.
- Null-modem (RX<>TX / GND--GND) cable, attached to another computer
running tio.

*** loadbsd & serial console
Going by the install guide, there's more than a few install methods.
But I had to discard most of them, because they require the netbsd
partition to be below the 4GB point in the disk.

What was left is
/pub/NetBSD/NetBSD-9.2/amiga/installation/misc/loadbsd. When I looked
at the 9.2 manpage for loadbsd, a dedicated section down at boot(8),
there seemed to be a convenient -C option to tell the loaded kernel to
use the serial port as console. Except loadbsd wouldn't take it.

After some research on IRC (with mlelstv), it turned out that loadbsd
hasn't been built in a very long time, because the netbsd build
process is not able to build AmigaOS binaries. The source code can be
found at /src/sys/arch/amiga/stand/loadbsd/. It includes a README with
some instructions by what likely was the last person who built it.

Thus, having a working amigaos crossdev toolchain, I went ahead and
tried to build it using these instructions as a starting point.
Predictably, it didn't work immediately, as netbsd has changed a lot
in the years after loadbsd was last built. Amusingly, loadbsd.c itself
was found broken; The string long help text printed with -h parameter
uses multiple likes of C sources without \n\ at the end of each line.
Thus this source has been changed over the years with no attempt to
build it.

Surprisingly, booting the netbsd install kernel with the binary I
built worked right away, but the install kernel couldn't find the
installation image (miniroot.fs) in wd0b, despite I had theoretically
written it earlier, due to an issue with xstreamtodev elaborated
below.

The result of this work is here, including a binary:
https://github.com/rvalles/netbsd_amiga_loadbsd_pleasebuild

Providing a current binary, be it mine or one built by an actual
maintainer, would not hurt. But loadbsd, as it is, isn't maintainable.
Some sort of decision needs to be made on the direction to take to
ensure a future for this tool.

*** xstreamtodev & TD64|NSD
As it turns out, the xstreamtodev tool, also found at
installation/misc/, does not support disk offsets above 4GB.

Tool seems to be able to take a bunch of parameters, so I couldn't
find a manpage for the tool anywhere, so I had just followed the
install guide instructions. The partition name was easily found with
rdbinfo, from the same misc dir, xstreamtodev ran, and its output
suggested it worked. But it didn't.

What happened instead is, as discussed with mlelstv on IRC, somewhere
in the first 4GB of the disk, data that has nothing to do with netbsd
got clobbered i.e. destroyed. xstreamtodev does not support "large
disks", which can be done through implementing NSD or TD64
specifications. Note that xstreamtodev uses scsi.device directly,
which has no concept of partitions. The whole disk is addressed by a
32bit offset, thus it will overflow and clobber/destroy data when the
32bit uint does overflow.

Some info on the 4GB problem here:
http://thomas-rapp.homepage.t-online.de/4gb_faq.html

TD64 spec is found here: http://aminet.net/package/dev/misc/trackdisk64

xstreamtodev source code here:
http://ftp.nluug.nl/os/BSD/NetBSD/arch/amiga/utils/

I couldn't find any dd-like tools for AmigaOS, which suggests I should
try and implement some level of support for doing this on my
pyamigadebug (library behind AmigaXfer) in the future. But, for now, I
had to settle with booting an old Linux install and using dd.

The summary version is: The netbsd INSTALL guide is instructing users
to unwittingly destroy their data. I was lucky not to be keeping
anything important there. As it is, I would suggest the xstreamtodev
tool be relocated into a "danger" directory, including the sources and
a README including a warning about its current state / 4GB issue. The
INSTALL guide should be updated accordingly.

*** Install script annoyances
By now, with some help from the new loadbsd binary from above, and an
old Linux install to use dd and get around xstreamtodev issue above, I
had managed to start the Netbsd install script, which I interacted
with through the serial port.

This is a script that installs netbsd, very different from the
installer found in some other architectures. I do not know why Amiga
port uses a separate script (although I'm curious). This script was a
little annoying in places. I had to restart the whole process a few
times due to issues with it.

After setting up the network (including manual IP addresses; there's
no option to use dhcp), the script offers the user to run a shell to
do further manual network configuration if necessary. It should
probably suggest testing the network connectivity there, as this
cannot be fixed later. Fortunately, I was able to get the network
working on the first try.

When choosing install source (ftp), the user is asked for the IP
address of the FTP server. Not the DNS name, but the IP address. I do
not know (I have not tried) if entering a name there would work, so I
went with the safer option of entering a public netbsd mirror IP. And
I messed up, by using an ip address of a mirror that doesn't have a
FTP server. Later, connection failed, but the install script wouldn't
allow me to pick a different address anymore, only backtracking a
step, to picking a directory path to the install sets. This forced me
to restart the process.

Second attempt, I put the right mirror address, but when the time came
to download the sets, the install process blew up. Apparently, there
was a '@' character in the password, and the script did absolutely not
like that. Note this is the common way of doing anonymous FTP
connections: 'anonymous' as user, and an email address as the
password. Again, forced to restart the script as I couldn't backtrack
far enough. Nasty.

Finally, I was able to go through the full script and netbsd got
installed. But not without some eerie messages while unpacking the
sets:

Extracting comp.tgz:
pax: Invalid header, starting valid header search.
pax: Header checksum 15606 does not match 15612
pax: Header checksum 15211 does not match 15216

No idea what happened there, and the install process continued, but I
have low confidence in the installed compiler set.

*** First boot annoyances
I booted the netbsd-GENERIC kernel with the loadbsd I built above,
again using the serial port with '-C'. First boot took a very long
time, with the fontconfig cache step taking a whole hour, before
getting to a login prompt.

Later, a process updating man db ran in the background for over two
hours. I was working on something else, so I just glanced at a
terminal with the serial console running 'top' once in a while.

Once I got back to it, I manually started sshd via '/etc/rc.d/sshd
onestart' as the install script did not enable it or ask whether to
enable it. Not a big deal. Generating the keys took a long time as
expected.

So, I finally ssh'd into the amiga, only to find it wouldn't work,
because the install script did not set up ptyfs to automount in fstab.
I solved this with 'cat /usr/share/examples/fstab/fstab.pseudo
>>/etc/fstab and mounted it manually for now, but the install script
should probably be modified to do this much automatically.

And thus, I was finally able to get in via ssh.

*** X11
If I used the Amiga's own console, X would be a problem, as the Amiga
chipset's graphics aren't supported with current Xorg. This is a known
issue and I also know from list discussion that steps are being taken
to, so it just gets a passing mention.

*** Conclusions
It is possible to install the Netbsd Amiga port. It is far from a
smooth experience, including data-destroying bear traps, and requires
an unrealistic amount of expertise and patience. A bunch of flaws have
been highlighted above, along with specific suggestions on how to
possibly tackle some of them, in order to make the install experience
less painful.

Best Regards,
Roc