kern/60638: AF_UNIX SOCK_SEQPACKET recvmsg truncates messages sent w/ MSG_EOR
"Eric Wong via gnats" <[email protected]>
| Newsgroups | gmane.os.netbsd.bugs |
|---|---|
| Message-ID | <[email protected]> |
>Number: 60638 >Category: kern >Synopsis: AF_UNIX SOCK_SEQPACKET recvmsg truncates messages sent w/ MSG_EOR >Confidential: no >Severity: serious >Priority: medium >Responsible: kern-bug-people >State: open >Class: sw-bug >Submitter-Id: net >Arrival-Date: Mon Aug 24 20:40:00 +0000 2026 >Originator: Eric Wong <[email protected]> >Release: NetBSD 11.0 >Organization: self >Environment: System: NetBSD netbsd110-amd64 11.0 NetBSD 11.0 (GENERIC) #0: Thu Jul 30 15:23:12 UTC 2026 [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC amd64 Architecture: x86_64 Machine: amd64 >Description: AF_UNIX SOCK_SEQPACKET messages sent with MSG_EOR are truncated upon recvmsg small values under 2K even when receiver supplies adequately large buffers. - While I have workaround for sending large buffers by putting the payload in an extra FD via SCM_RIGHTS, 1972 bytes seems like an unusually small value for a buffer (smaller than _SC_PAGE_SIZE or PIPE_BUF) and the workaround costs extra FDs. - The problem doesn't appear to manifest w/o MSG_EOR, but MSG_EOR seems like the right thing to demarcate individual messages when using SOCK_SEQPACKET with local Unix sockets. - OpenBSD fixed a similar issue several years ago (see code comments). MSG_EOR seems required for my code to work with FreeBSD 15.x+ (it worked fine w/o MSG_EOR in FreeBSD <=14.x). I've used Linux with and without MSG_EOR since 2013 for AF_UNIX+SOCK_SEQPACKET and haven't noticed unusually small truncation lengths. >How-To-Repeat: /* * compile + run following code: * cc -o /path/to/run /this/file.c -Wall && /path/to/run * * Using sendmsg(..., MSG_EOR) on an AF_UNIX + SOCK_SEQPACKET socket * appears to cause the recvmsg() caller to receive truncated data * (and set MSG_TRUNC) on NetBSD 11.0. * * OpenBSD had a similar bug fixed in 2023: * <https://marc.info/[email protected]> * * Feel free to use this as test case under what ever license fits * best (CC-0, MIT, BSD-[23], ISC). * I extracted this test case from a codebase which I've been * using on both FreeBSD 12.x+ and Linux since 2021. * I normally send data between processes, but this truncation * happens within the same process, too. */ #include <sys/socket.h> #include <sys/types.h> #include <sys/uio.h> #include <unistd.h> #include <stdlib.h> #include <errno.h> #include <stdio.h> #include <string.h> #include <assert.h> #include <err.h> #define NSEND 2048 #define NRECV (4096 * 33) // the most FDs I ever intend to send: #define SEND_FD_CAPA 10 #define SEND_FD_SPACE (SEND_FD_CAPA * sizeof(int)) union my_cmsg { struct cmsghdr hdr; char pad[sizeof(struct cmsghdr) + 16 + SEND_FD_SPACE]; }; static void do_sendmsg(int fd) { /* * FreeBSD 15.x seems to require MSG_EOR nowadays for proper * message boundaries (didn't need it before 14.x). Linux * doesn't seem to care... */ int sflags = MSG_EOR; /* * I normally have several FDs to send, but the * discrepancy happens even with nfds == 0 */ int i, nfds = 4; struct msghdr msg = { 0 }; union my_cmsg cmsg = { 0 }; char to_send[NSEND]; struct iovec iov; int *fdp; ssize_t sent; memset(to_send, 'a', sizeof(to_send)); to_send[NSEND - 1] = 'b'; // for kdump || strace output iov.iov_base = to_send; iov.iov_len = sizeof(to_send); msg.msg_iov = &iov; msg.msg_iovlen = 1; msg.msg_control = &cmsg.hdr; msg.msg_controllen = CMSG_SPACE(nfds * sizeof(int)); cmsg.hdr.cmsg_level = SOL_SOCKET; cmsg.hdr.cmsg_type = SCM_RIGHTS; cmsg.hdr.cmsg_len = CMSG_LEN(nfds * sizeof(int)); fdp = (int *)CMSG_DATA(&cmsg.hdr); for (i = 0; i < nfds; i++) *fdp++ = i; sent = sendmsg(fd, &msg, sflags); fprintf(stderr, "%d sent: %zd of %zu\n", (int)getpid(), sent, iov.iov_len); } static void do_recvmsg(int fd) { union my_cmsg cmsg = { 0 }; struct msghdr msg = { 0 }; struct iovec iov; ssize_t r; char recvbuf[NRECV]; iov.iov_base = recvbuf; iov.iov_len = sizeof(recvbuf); msg.msg_iov = &iov; msg.msg_iovlen = 1; msg.msg_control = &cmsg.hdr; msg.msg_controllen = CMSG_SPACE(SEND_FD_SPACE); r = recvmsg(fd, &msg, 0); /* * FreeBSD and Linux receive everything sent, here, but * NetBSD only gets 1972 bytes and sets MSG_TRUNC here: */ fprintf(stderr, "%d recv: %zd of %zu " "EOR=%d TRUNC=%d CTRUNC=%d OOB=%d\n", getpid(), r, iov.iov_len, msg.msg_flags & MSG_EOR, msg.msg_flags & MSG_TRUNC, msg.msg_flags & MSG_CTRUNC, msg.msg_flags & MSG_OOB); if (r > 0 && cmsg.hdr.cmsg_level == SOL_SOCKET && cmsg.hdr.cmsg_type == SCM_RIGHTS) { size_t len = cmsg.hdr.cmsg_len; int *fdp = (int *)CMSG_DATA(&cmsg.hdr); size_t i; for (i = 0; CMSG_LEN((i + 1) * sizeof(int)) <= len; i++) fprintf(stderr, "%d recv fd=%d\n", (int)getpid(), *fdp++); } if (msg.msg_flags & (MSG_TRUNC|MSG_CTRUNC)) errx(EXIT_FAILURE, "truncated (BAD)"); else warnx("no truncation (GOOD)"); } int main(void) { int fds[2]; int rc = socketpair(AF_UNIX, SOCK_SEQPACKET, 0, fds); assert(rc == 0); do_sendmsg(fds[0]); do_recvmsg(fds[1]); return 0; } >Fix: <how to correct or work around the problem, if known (multiple lines)> >Unformatted: <Please check that the above is correct for the bug being reported,> <and append source date of snapshot, if applicable (one line).>