Re: RFC: ENOSYS vs SIGSYS for FreeBSD ABIs

On Sun, Apr 19, 2026 at 10:55:22AM -0600, Warner Losh wrote:
> On Sat, Apr 18, 2026 at 2:02???PM Poul-Henning Kamp <[email protected]>
> wrote:
> 
> > --------
> > John Baldwin writes:
> >
> > > From what I can tell though, while POSIX documents both ENOSYS and
> > SIGSYS,
> > > it does not document the semantics of these (and the one reference I
> > could
> > > find to SIGSYS specifically noted that the semantics aren't documented
> > since
> > > there aren't any consistent semantics across systems).
> >
> > The explanation I heard many years ago, is that SIGSYS was supposed
> > to make it possible to emulate system-calls in userland, with some
> > hand-wavy mumbling about pre-shlib binaries not having to know if
> > you had bought a STREAMS based or socket(2) based network-add-on
> > product.
> >
> > To me that sounded like precisely the kind of thing the commercial
> > UNIX vendors agree on in the late 1980ies, so it satisfied my curiosity.
> >
> > Any of Sam, Kirk, Eric or TUHS would be a more credible source.
> >
> 
> Yea, I was going to do a deep dive in into V7, talk about how SIGEMT
> allowed RT11 binaries to run, etc, but I'm highly skeptical that a pre-load
> shim could work since many programs reset segnal handlers to default
> just to be safe.
> 
> I thought the reason was more prosaic: we don't trust programs to error
> check
> so when a bad system call is made, we kill them. Bad bit of policy, but I
> suspect that's the real reason

I think the main reason to keep it is if you add new syscalls that
allower permissions to be dropped and don't check the results, you can
end up with nasty security holes.  We've now potentially got tooling like
__result_use_check to make this harder to get wrong, but we're not using
them consistently (e.g., setcred(2) should probably have had this
annotation in its decleration).

> > > On the other side, I have _never_ had SIGSYS be an aid.  It's only gotten
> > > in the way.
> >
> > +1
> >
> 
> Same.  Though with the caveat that I'm not sure how our massive system call
> changes would have gone in the past. I think that some would be fine, other
> would just fail entirely depending on how critical the new system call was.
> Hopefully,
> it would have been 'fine enough' to recover, much like using libfoo.so.10
> when
> libfoo.so.11 comes out.

As a developer I've never found SIGSYS helpful.  That being said, I
don't think a more nuanced approach would be all that hard.  One option
I've pondered would to automatially flip kern.signosys to 0 (and
probably kern.lognosys to > 0) if init gets a SIGSYS (maybe limited to
before it forks successfully).  You'd probably want some pretty shouty
logging to go with that, but it would let you survive a kernel roll-back
with new syscalls in csu code.

-- Brooks