[NETLINK]: Fix two socket hashing bugs.

Linux Kernel Mailing List <[email protected]> Mon, 4 Jul 2005 23:06:00 -0700
Newsgroups gmane.linux.kernel.commits.2-4
Message-ID <[email protected]>
tree fa23b92189554d42087d85b33913fb4b18995ad7
parent bb6c40830e2f66b33c22275829a730ed078e430a
author David S. Miller <[email protected]> Sun, 26 Jun 2005 14:20:15 -0700
committer David S. Miller <[email protected]> Sun, 26 Jun 2005 14:20:15 -0700

[NETLINK]: Fix two socket hashing bugs.

1) netlink_release() should only decrement the hash entry
   count if the socket was actually hashed.

   This was causing hash->entries to underflow, which
   resulting in all kinds of troubles.

   On 64-bit systems, this would cause the following
   conditional to erroneously trigger:

	err = -ENOMEM;
	if (BITS_PER_LONG > 32 && unlikely(hash->entries >= UINT_MAX))
		goto err;

2) netlink_autobind() needs to propagate the error return from
   netlink_insert().  Otherwise, callers will not see the error
   as they should and thus try to operate on a socket with a zero pid,
   which is very bad.

   So bug #1 above, combined with this one, resulted in hangs
   on netlink_sendmsg() calls to the rtnetlink socket.  We'd try
   to do the user sendmsg() with the socket's pid set to zero,
   later we do a socket lookup using that pid (via the value we
   stashed away in NETLINK_CB(skb).pid), but that won't give us the
   user socket, it will give us the rtnetlink socket.  So when we
   try to wake up the receive queue, we dive back into rtnetlink_rcv()
   which tries to recursively take the rtnetlink semaphore.

Thanks to Jakub Jelink for providing backtraces, and Herbert Xu for
debugging patches to help track this down.

Signed-off-by: David S. Miller <[email protected]>

 net/netlink/af_netlink.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -330,9 +330,9 @@ static void netlink_remove(struct sock *
 	u32 pid = nlk_sk(sk)->pid;
 
 	netlink_table_grab();
-	hash->entries--;
 	for (skp = nl_pid_hashfn(hash, pid); *skp; skp = &((*skp)->next)) {
 		if (*skp == sk) {
+			hash->entries--;
 			*skp = sk->next;
 			__sock_put(sk);
 			break;
@@ -450,7 +450,7 @@ retry:
 	err = netlink_insert(sk, pid);
 	if (err == -EADDRINUSE)
 		goto retry;
-	return 0;
+	return err;
 }
 
 static inline int netlink_capable(struct socket *sock, unsigned int flag)