Re: RFC: ENOSYS vs SIGSYS for FreeBSD ABIs

On 18 Apr 2026, at 18:04, John Baldwin <[email protected]> wrote:
>=20
> FreeBSD has a longstanding behavior of sending a SIGSYS signal to a =
thread
> that invokes an invalid system call.  This behavior was inherited from =
BSD
> from a commit from Mike Karels whose log message is "POSIX changes"
> =
(https://svnweb.freebsd.org/csrg/sys/kern/kern_sig.c?revision=3D39513&view=
=3Dmarkup).
>=20
> =46rom what I can tell though, while POSIX documents both ENOSYS and =
SIGSYS,
> it does not document the semantics of these (and the one reference I =
could
> find to SIGSYS specifically noted that the semantics aren't documented =
since
> there aren't any consistent semantics across systems).
>=20
> Linux does not send SIGSYS for an invalid system call, but instead =
returns
> ENOSYS which motivated several changes from kib@ a few years ago =
which,
> amony other things, makes FreeBSD's behavior as there is now a sysctl =
to
> determine if the FreeBSD ABIs raise SIGSYS or just return ENOSYS.
>=20
> In my experience to date, I have generally found SIGSYS a pain.  While =
we
> don't generally support forwards compatibility (newer world on older
> kernel), it sometimes happens, and it's really annoying to have init, =
etc.
> all die on boot (resulting in an instant panic in the case of init).
> This is especially true if one is adding a new "advisory" system call =
for
> which the program can still run fine if the syscall fails.  However, =
there
> is not a really easy way to deal with this, particularly in library =
code.
> Installing a handler for SIGSYS (or just ignoring it), means a library
> has to stomp on process state that typically should be managed by the
> main binary.  The other alternative is to try to do extra checks to =
determine
> if a syscall might work before you invoke it, whether that's
> kern.feature.foo sysctls (but that does mean invoking =
yet-another-syscall
> before the one you actually care about), or AT_BSDFLAGS, etc.
>=20
> On the other side, I have _never_ had SIGSYS be an aid.  It's only =
gotten
> in the way.
>=20
> Over in CheriBSD land we just got bitten by this (again) for a system =
call
> that is purely advisory (permits adding a label to memory regions that =
you
> can see in procstat -v, etc., but isn't needed for functionality) due =
to
> some of the cross-threading we have between kernels and worlds which =
is why
> this is top of my mind again.
>=20
> Both OpenBSD and NetBSD still send SIGSYS.  To deal with SIGSYS in =
/sbin/init,
> all of the BSDs imported a change from BSDI after 4.4BSD that ignored =
the
> first 25 errors, but both FreeBSD and OpenBSD have removed that =
(NetBSD still
> has it).  The log message for our removal claims that relying on =
ENOSYS vs
> SIGSYS is harmful, but I'm less convinced that that is true.  You have =
to
> have code invoking a system call that doesn't check for errors for =
SIGSYS
> to actually be useful.
>=20
> My (long-winded) proposal is to change the default setting of the =
sysctl kib@
> added a few years ago such that we only return ENOSYS by default.  =
Folks who
> really want SIGSYS could still opt-in via the sysctl/loader tunable, =
but I
> feel that ENOSYS really is the better default behavior instead of =
SIGSYS.



I have had SIGSYS be an aid.

SIGSYS stops a process that attempts to use a system call that does not =
exist, without requiring a test. ENOSYS requires a user mode process to =
check every system call return value correctly, and then correctly =
decide what to do. Correctness on rare paths is not always there.

Using SIGSYS by default eliminates the class of failure when the user =
mode process failed to check and handle ENOSYS correctly. I think that =
is the core idea behind the statement =E2=80=9Crelying on ENOSYS vs =
SIGSYS is harmful.=E2=80=9D

Assume ENOSYS: What do you do if the system call you want doesn=E2=80=99t =
exist? Code a fallback strategy, like =E2=80=9Crename(2) isn=E2=80=99t =
there, I=E2=80=99ll try link(2) and then unlink(2)=E2=80=9D? Just say =
"it didn=E2=80=99t matter anyway=E2=80=9D? Or just kill the process as =
if SIGSYS was sent?

The arguments I read here are about special cases as a FreeBSD =
developer. People messing around with CURRENT are knowledgeable enough =
to change a sysctl without changing the default for everyone.

If I deploy a binary to a system that uses an unimplemented system call, =
I want that process to die quickly. Fail-fast. Explicitly handling these =
cases is not worth thinking about if you can just get SIGSYS.


Jan M.