SOCK_RAW and in_pcb.c

  Hi,

the context and motivation of this email is that I'm working on improving
parallelism on connect(2) on TCP sockets.  UDP will be also affected as a side
effect :) The goal is to get rid of two fat locks that lock the inpcb database
for a given protocol - the inp_info_lock and inp_hash_lock.  In my working
branch I haved achieved getting rid of inp_info_lock and pushing
INP_HASH_WLOCK() out of TCP and UDP files into in_pcb.c:

https://github.com/freebsd/freebsd-src/compare/main...glebius:FreeBSD:inpcb

The following step would be to decompose this fat lock into per-slot locks. But
then I stumbled on SOCK_RAW.  The netinet/raw_ip.c is the last out of in_pcb.c
consumer of INP_HASH_WLOCK().

TLDR version: I want to divorce in_pcb.c and AF_INET/SOCK_RAW and
AF_INET6/SOCK_RAW.

Why did I come to such decision?  First, let's refresh on what SOCK_RAW input
routine is supposed to do.  It is supposed to deliver an incoming packet to
multiple destinations.  The best description of it is given in Stevens's UNIX
Network Programming (Section 28.4 in the last edition). Second, let's look at
the historical implementation of rip_input():

https://github.com/sergev/4.4BSD-Lite2/blob/master/usr/src/sys/netinet/raw_ip.c#L84

It is simple as 2 cents.  Now, let's take a look into our modern
raw_ip6.c:rip6_input().  Algorithmically it does the same.  However, a ton of
complexity is provided underneath - an SMR-protected UMA zone, port(!) hash and
other stuff that in_pcb.c does for TCP and UDP.  But we still end up with
scanning all sockets.  All of the complex in_pcb.c machinery was designed to
speed up the fast path in_pcblookup(), that finds a single inpcb that matches a
connection.  But in_pcblookup() is not applicable to SOCK_RAW, as there will
never be a single pcb that answers the question where "shall a datagram go?".

For IPv4's rip_input() there is one difference - it first scans a hash slot
with raw sockets that are protocol bound and also did bind(2) and did
connect(2), and only then scans all unbound or loosely bound sockets.  With
loosely bound I mean sockets that did either bind(2) or connect(2).  This
difference between rip_input() and rip6_input() originates from 9ed324c9a588f,
and this revision provides the only practical case when there are a lot of raw
sockets in a system - a VPN concentrator.  A better solution for the problem
solved by 9ed324c9a588f would be a kernel module that registers into
ip_protox[] and does its own optimized PPTP processing.  However, I won't call
9ed324c9a588f a wrong change.  The mpd5 port is still alive and with the modern
internet the popularity of different VPNs that run over different IP protocols
is growing. Although any specific VPN will benefit from a specific kernel
module, we still want to improve a case when we got thousands of tightely bound
raw sockets.  We should also extend improvements of 9ed324c9a588f to IPv6.

One other point I'd like to note is that speculatively the de2d47842e88 (SMR
for inpcbs) was a negative performance change for SOCK_RAW.  The inpcb
iteration with help of inp_next() supposedly is more expensive than historic
INP_INFO_RLOCK(); LIST_FOREACH(..., cause we lock every single inpcb.  This is
fine for TCP and UDP where iteration never happens in the fast path, rather we
are focused on iteration to never have affect on the fast path and we accept
that iteratation is more expensive.  But for SOCK_RAW iteration is required on
the fast path.

Here is the plan:
- a shared code between IPv4 and IPv6
- rawpcb database per VNET that has: a list of unbound pcbs and tree of
  src-bound pcbs.  We could have also a tree of dst-bound and src+dst-bound,
  but I would speculate that practical hosts/routers that have a large
  (local IP address count * raw socket count) product do not exist.  The
  current hash that serves only the src+dst+proto-bound pcbs, IMHO, is too
  restrictive and supposedly only mpd5 benefits from it.  I'd speculate that
  src-bound lookup database will not degrade mpd5 performance, but will
  provide improvements for other raw users.
- the database is protected by rwlock, that is rlocked on the fast path and
  wlocked by socket(2), bind(2) and connect(2).
- setsockopt(2) needs to take care to not use raw's so_pcb as inpcb any more

Any comments?

--
Gleb Smirnoff