Re: [Pound Mailing List] Pound 2.7 and TProxy

Scott McKeown <scott-Oteo70RhwZ77w/[email protected]>
Newsgroups gmane.comp.web.pound.general
Message-ID <[email protected]>
Hi Everyone,

Right we finally got this working so anyone that is also wanting to use
TProxy with Pound v2.7 the patch file is attached.


patch -i pound_2.7_with_tproxy.patch --verbose --ignore-whitespace




On 20 October 2015 at 16:19, Scott McKeown <scott-Oteo70RhwZ77w/[email protected]> wrote:

> Hi Joe,
>
> Thanks for the reply.
>
> Yes that TPROXY=1 is set when running the make command
>
> make TPROXY=1
>
> However, we 'think' we may have found the issue now after some more
> digging around looks like I'm missing a section from within the config.c
> file to allow the TProxy directive to be understood but I'm going to have
> another look tomorrow now.
>
> Likewise I'll do a gitpull on your branch and give the 'IPTransparent'
> option ago.
>
>
> ~Scott
>
> On 20 October 2015 at 16:04, Joe Gooch <[email protected]> wrote:
>
>> Did you change TPROXY=1 in the Makefile? Looks like unless you do that,
>> it's not included.
>>
>> The other option I see is to use my version of a similar thing:
>>
>> https://github.com/goochjj/pound/commit/65e14aa8b52f9170f513399bfe430a1c66a9e34b
>>
>> Your kernel headers need to include the IP options, and then transproxy
>> works for ipv6 listeners, freebind for ipv4 listeners.
>> --
>> Joe
>>
>> Confidentiality Notice: This e-mail transmission may contain confidential
>> and legally privileged information that is intended only for the individual
>> named in the e-mail address. If you are not the intended recipient, you are
>> hereby notified that any disclosure, copying, distribution, or reliance
>> upon the contents of this e-mail message is strictly prohibited. If you
>> have received this e-mail transmission in error, please reply to the
>> sender, so that proper delivery can be arranged, and please delete the
>> message from your mail box.
>>
>>
>>
>> Joseph Gooch
>>
>> www.sapphirek12.org | office: (866) 366-9540
>>
>>
>>
>> *CONFIDENTIALITY STATEMENT*
>>
>> *The documents and communication included in this email transmission may
>> contain confidential information.  All information is intended only for the
>> use of the above named recipient(s).  If you are not the named recipient,
>> you are NOT authorized to read, disclose, copy, distribute, or take any
>> action on the information and any action other than immediate delivery to
>> the named recipient is strictly prohibited. If you have received this email
>> in error, do NOT read the information and please immediately notify sender
>> by telephone and email and immediately delete this email.  If you are the
>> named recipient, you are NOT authorized to reveal any of this information
>> to any unauthorized person and are hereby instructed to delete this email
>> when no longer needed. *
>>
>> From: Scott McKeown <scott-Oteo70RhwZ77w/[email protected]>
>> Reply-To: "[email protected]" <[email protected]>
>> Date: Tuesday, October 20, 2015 at 5:25 AM
>> To: Pound Mailing List <[email protected]>
>> Subject: [Pound Mailing List] Pound 2.7 and TProxy
>>
>> Hello Everyone,
>>
>> I've finally got to admit defeat on getting this patch to work and I was
>> wondering if anyone out there could please give me a hand getting this to
>> work.
>>
>> I've attached the patch file for anyone to play with but what I'm trying
>> to do is get Unix TProxy to work with pound 2.7 again, the original version
>> of this patch has been working since pound v2.4.
>>
>> Anyhow, the patch compiles into the latest version but if you add "TProxy
>> 1" into the configuration you get a 'unknown directive' error
>>
>>
>> *Config File:*
>> # cat /etc/pound/pound.cfg
>> # Pound configuration file generated by loadbalancer.org appliance
>> User            "nobody"
>> Group           "nobody"
>> LogLevel        0
>> Client          30
>> Timeout         60
>> Threads         250
>> TProxy          1
>>
>> ListenHTTPS
>>         # Label: pound1
>>         Address 192.168.63.59
>>         Port    443
>>         Cert    "/etc/loadbalancer.org/certs/server.pem"
>>         xHTTP   3
>>         ReWriteLocation 1
>>         Ciphers
>> "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS"
>>         SSLHonorCipherOrder 1
>>         SSLAllowClientRenegotiation 0
>>         Disable SSLv2
>>         Disable SSLv3
>>         Service
>>                 BackEnd
>>                         Address 192.168.64.254
>>                         Port    80
>>                         TProxy  1
>>                 End
>>         End
>> End
>>
>>
>> *Start Up:*
>> # /usr/local/sbin/pound
>> starting...
>> detect_tproxy(): tproxy is is detected
>> tproxy: available
>> /etc/pound/pound.cfg line 8: unknown directive - aborted
>>
>>
>>
>>
>> --
>> With Kind Regards.
>>
>> Scott McKeown
>> Loadbalancer.org
>> http://www.loadbalancer.org
>> Tel (UK) - +44 (0) 3303801064 (24x7)
>> Tel (US) - +1 888.867.9504 (Toll Free)(24x7)
>>
>
>
>
> --
> With Kind Regards.
>
> Scott McKeown
> Loadbalancer.org
> http://www.loadbalancer.org
> Tel (UK) - +44 (0) 3303801064 (24x7)
> Tel (US) - +1 888.867.9504 (Toll Free)(24x7)
>



-- 
With Kind Regards.

Scott McKeown
Loadbalancer.org
http://www.loadbalancer.org
Tel (UK) - +44 (0) 3303801064 (24x7)
Tel (US) - +1 888.867.9504 (Toll Free)(24x7)
pound_2.7_with_tproxy.patch (application/octet-stream, 14.3 KB)
diff -urN config.c config.c
--- config.c
+++ config.c
@@ -81,6 +81,9 @@
 static regex_t  ClientCert, AddHeader, DisableProto, SSLAllowClientRenegotiation, SSLHonorCipherOrder, Ciphers;
 static regex_t  CAlist, VerifyList, CRLlist, NoHTTPS11, Grace, Include, ConnTO, IgnoreCase, HTTPS;
 static regex_t  Disabled, Threads, CNName, Anonymise, ECDHCurve;
+#ifdef TPROXY_ENABLE
+static regex_t  TProxy;
+#endif
 
 static regmatch_t   matches[5];
 
@@ -250,6 +250,9 @@
     res->url = NULL;
     res->next = NULL;
     has_addr = has_port = 0;
+#ifdef TPROXY_ENABLE
+    res->tp_enabled = 0;
+#endif
     pthread_mutex_init(&res->mut, NULL);
     while(conf_fgets(lin, MAXBUF)) {
         if(strlen(lin) > 0 && lin[strlen(lin) - 1] == '\n')
@@ -415,6 +415,10 @@
             if((res->addr.ai_family == AF_INET || res->addr.ai_family == AF_INET6) && !has_port)
                 conf_err("BackEnd missing Port - aborted");
             return res;
+#ifdef TPROXY_ENABLE
+        } else if(!regexec(&TProxy, lin, 4, matches, 0)) {
+	    if (enable_tproxy && have_tproxy) res->tp_enabled = atoi(lin + matches[1].rm_so);
+#endif
         } else {
             conf_err("unknown directive");
         }
@@ -1282,6 +1282,13 @@
 #endif
             }
             return res;
+#ifdef TPROXY_ENABLE
+        } else if(!regexec(&TProxy, lin, 4, matches, 0)) {
+	    if (have_tproxy)
+                enable_tproxy = atoi(lin + matches[1].rm_so);
+	    else
+		enable_tproxy = 0;
+#endif
         } else {
             conf_err("unknown directive");
         }
@@ -1434,7 +1434,16 @@
             }
         } else if(!regexec(&Anonymise, lin, 4, matches, 0)) {
             anonymise = 1;
-        } else {
+#ifdef TPROXY_ENABLE
+        } else if(!regexec(&TProxy, lin, 4, matches, 0)) {
+            if (have_tproxy)
+                enable_tproxy = atoi(lin + matches[1].rm_so);
+            else
+                enable_tproxy = 0;
+#endif
+
+
+       } else {
             conf_err("unknown directive - aborted");
         }
     }
@@ -1492,6 +1492,9 @@
     || regcomp(&VerifyList, "^[ \t]*VerifyList[ \t]+\"(.+)\"[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
     || regcomp(&CRLlist, "^[ \t]*CRLlist[ \t]+\"(.+)\"[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
     || regcomp(&NoHTTPS11, "^[ \t]*NoHTTPS11[ \t]+([0-2])[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
+#ifdef TPROXY_ENABLE
+    || regcomp(&TProxy, "^[ \t]*TProxy[ \t]+([01])[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
+#endif
     || regcomp(&Include, "^[ \t]*Include[ \t]+\"(.+)\"[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
     || regcomp(&ConnTO, "^[ \t]*ConnTO[ \t]+([1-9][0-9]*)[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
     || regcomp(&IgnoreCase, "^[ \t]*IgnoreCase[ \t]+([01])[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
@@ -1662,6 +1662,9 @@
     regfree(&VerifyList);
     regfree(&CRLlist);
     regfree(&NoHTTPS11);
+#ifdef TPROXY_ENABLE
+    regfree(&TProxy);
+#endif
     regfree(&Include);
     regfree(&ConnTO);
     regfree(&IgnoreCase);

diff -urN http.c http.c
--- http.c
+++ http.c
@@ -855,7 +855,12 @@
                 clean_all();
                 return;
             }
-            if(connect_nb(sock, &backend->addr, backend->conn_to) < 0) {
+
+#ifdef TPROXY_ENABLE
+	    if(connect_nb(sock, &backend->addr, backend->to, backend->tp_enabled ? &from_host : NULL) < 0) {
+#else
+	    if(connect_nb(sock, &backend->addr, backend->to) < 0) {
+#endif
                 str_be(buf, MAXBUF - 1, backend);
                 logmsg(LOG_WARNING, "(%lx) backend %s connect: %s", pthread_self(), buf, strerror(errno));
                 shutdown(sock, 2);

--- Makefile.in
+++ Makefile.in
@@ -25,9 +25,11 @@

 CC=@PTHREAD_CC@

+C_TPROXY=-DTPROXY_ENABLE
+
 CFLAGS=-DF_CONF=\"@sysconfdir@/pound.cfg\" -DVERSION=\"@PACKAGE_VERSION@\" -DC_SSL=\"@C_SSL@\" -DC_T_RSA=\"@C_T_RSA@\" \
        -DC_DH_LEN=\"@C_DH_LEN@\" -DC_MAXBUF=\"@C_MAXBUF@\" -DC_OWNER=\"@C_OWNER@\" -DC_GROUP=\"@C_GROUP@\" \
-    -DC_SUPER=\"@C_SUPER@\" -DC_CERT1L=\"@C_CERT1L@\" @CFLAGS@ @PTHREAD_CFLAGS@ @CPPFLAGS@
+    -DC_SUPER=\"@C_SUPER@\" -DC_CERT1L=\"@C_CERT1L@\" @CFLAGS@ @PTHREAD_CFLAGS@ @CPPFLAGS@ ${C_TPROXY}
 LIBS=@LIBS@ @PTHREAD_LIBS@

 prefix=@prefix@

diff -urN pound.c pound.c
--- pound.c
+++ pound.c
@@ -41,6 +41,10 @@
             print_log,          /* print log messages to stdout/stderr */
             grace,              /* grace period before shutdown */
             control_sock;       /* control socket */
+#ifdef TPROXY_ENABLE
+int	    have_tproxy,	/* is transparent proxy available */
+	    enable_tproxy;	/* is tproxy enabled */
+#endif
 
 SERVICE     *services;          /* global services (if any) */

@@ -246,11 +246,23 @@
 #ifndef SOL_TCP
     struct protoent     *pe;
 #endif
+#ifdef TPROXY_ENABLE
+    typedef struct __user_cap_header_struct	tp_cap_header_t;
+    typedef struct __user_cap_data_struct	tp_cap_data_t;
+    tp_cap_header_t	tp_cap_header;
+    tp_cap_data_t	tp_cap_data;
+#endif
 
     print_log = 0;
     (void)umask(077);
     control_sock = -1;
     log_facility = -1;
+#ifdef TPROXY_ENABLE
+    have_tproxy = 0;
+    tp_cap_header.version = _LINUX_CAPABILITY_VERSION;
+    tp_cap_header.pid = 0;
+    tp_cap_data.effective = tp_cap_data.permitted = tp_cap_data.inheritable = 0;
+#endif
     logmsg(LOG_NOTICE, "starting...");
 
     signal(SIGHUP, h_shut);
@@ -320,6 +320,19 @@
     }
     SOL_TCP = pe->p_proto;
 #endif
+ 
+#ifdef TPROXY_ENABLE
+    if (! detect_tproxy()) {
+	if (geteuid() == 0) {
+	    have_tproxy = 1;
+	    logmsg(LOG_INFO, "tproxy: available");
+	} else {
+	    logmsg(LOG_ERR, "tproxy: disabled. you must start pound with root privileges. later it will drop privileges.");
+	}
+    } else {
+	logmsg(LOG_INFO, "tproxy: not supported or not enough privileges");
+    }
+#endif
 
     /* read config */
     config_parse(argc, argv);
@@ -450,6 +450,36 @@
         }
     }
 
+#ifdef TPROXY_ENABLE
+    if (enable_tproxy)
+    {
+        if (capget(&tp_cap_header, &tp_cap_data) != 0)
+	{
+            logmsg(LOG_ERR, "capabilities: can't get capabilities");
+            exit(1);
+	}
+	if (prctl(PR_SET_KEEPCAPS, 1) < 0)
+	{
+            logmsg(LOG_ERR, "capabilities: can't enable keep capabilities");
+            exit(1);
+	}
+	
+	tp_cap_data.effective = tp_cap_data.permitted = tp_cap_data.inheritable = 0;
+	tp_cap_data.effective |= (1 << CAP_NET_ADMIN);
+	tp_cap_data.effective |= (1 << CAP_SETUID);
+	tp_cap_data.effective |= (1 << CAP_SETGID);
+	tp_cap_data.permitted |= (1 << CAP_NET_ADMIN);
+	tp_cap_data.permitted |= (1 << CAP_SETUID);
+	tp_cap_data.permitted |= (1 << CAP_SETGID);
+
+	if (capset(&tp_cap_header, &tp_cap_data) != 0)
+	{
+            logmsg(LOG_ERR, "capabilities: can't set capabilities");
+            exit(1);
+	}
+    }
+#endif
+
     if(group)
         if(setgid(group_id) || setegid(group_id)) {
             logmsg(LOG_ERR, "setgid: %s - aborted", strerror(errno));
@@ -491,6 +491,32 @@
             exit(1);
         }
 
+#ifdef TPROXY_ENABLE
+    if (enable_tproxy)
+    {
+        if (capget(&tp_cap_header, &tp_cap_data) != 0)
+	{
+            logmsg(LOG_ERR, "user capabilities: can't get capabilities");
+            exit(1);
+	}
+	if (prctl(PR_SET_KEEPCAPS, 1) < 0)
+	{
+            logmsg(LOG_ERR, "user capabilities: can't enable keep capabilities");
+            exit(1);
+	}
+
+	tp_cap_data.effective = tp_cap_data.permitted = tp_cap_data.inheritable = 0;
+	tp_cap_data.effective |= (1 << CAP_NET_ADMIN);
+	tp_cap_data.permitted |= (1 << CAP_NET_ADMIN);
+
+	if (capset(&tp_cap_header, &tp_cap_data) != 0)
+	{
+            logmsg(LOG_ERR, "user capabilities: can't set capabilities");
+            exit(1);
+	}
+    }
+#endif
+
     /* split off into monitor and working process if necessary */
     for(;;) {
 #ifdef  UPER

diff -urN pound.h pound.h
--- pound.h
+++ pound.h
@@ -241,6 +241,14 @@
 #define const
 #endif

+#ifdef TPROXY_ENABLE
+#include <sys/prctl.h>
+#include <linux/capability.h>
+#ifndef IP_TRANSPARENT
+#define IP_TRANSPARENT 19
+#endif
+#endif
+
 #ifdef  HAVE_LONG_LONG_INT
 #define LONG    long long
 #define L0      0LL
@@ -275,6 +275,11 @@
             grace,              /* grace period before shutdown */
             control_sock;       /* control socket */

+#ifdef TPROXY_ENABLE
+extern int  have_tproxy,       /* is tproxy available */
+           enable_tproxy;      /* is tproxy enabled */
+#endif
+
 extern regex_t  HEADER,     /* Allowed header */
                 CHUNK_HEAD, /* chunk header line */
                 RESP_SKIP,  /* responses for which we skip response */
@@ -320,6 +320,9 @@
     int                 to;         /* read/write time-out */
     int                 conn_to;    /* connection time-out */
     struct addrinfo     ha_addr;    /* HA address/port */
+#ifdef TPROXY_ENABLE
+    int                        tp_enabled; /* TProxy is enabled */
+#endif
     char                *url;       /* for redirectors */
     int                 redir_req;  /* the redirect should include the request path */
     SSL_CTX             *ctx;       /* CTX for SSL connections */
@@ -579,7 +579,12 @@
  * Non-blocking version of connect(2). Does the same as connect(2) but
  * ensures it will time-out after a much shorter time period CONN_TO.
  */
+#ifdef TPROXY_ENABLE
+extern int  detect_tproxy(void);
+extern int  connect_nb(const int, const struct addrinfo *, const int, const struct addrinfo *);
+#else
 extern int  connect_nb(const int, const struct addrinfo *, const int);
+#endif

 /*
  * Parse arguments/config file

diff -urN svc.c svc.c
--- svc.c
+++ svc.c
@@ -372,6 +372,40 @@
     return res - kp_res;
 }

+#ifdef TPROXY_ENABLE
+/*
+ * detect_tproxy
+ * check for TPROXY
+ *
+ * Thanks to Loadbalancer.org, Inc. especially to Malcolm Turnbull for sponsorship
+ * Thanks to Sianet Business Hosting especially to Rodrigo L. L. Jorge for sponsorship
+ *
+ */
+int
+detect_tproxy(void)
+{
+    int sock, tp_val;
+
+    tp_val = 1;
+
+    sock = socket(AF_INET, SOCK_STREAM, 0);
+    if (sock < 0) {
+        logmsg(LOG_WARNING, "detect_tproxy(): socket SOCK_STREAM failed: %s", strerror(errno));
+        return 1;
+    }
+    if (setsockopt(sock, SOL_IP, IP_TRANSPARENT, &tp_val, sizeof(tp_val)) < 0) {
+        logmsg(LOG_INFO, "detect_tproxy(): tproxy is not available");
+        close(sock);
+        return 1;
+    }
+
+    close(sock);
+    logmsg(LOG_INFO, "detect_tproxy(): tproxy is is detected");
+
+    return 0;
+}
+#endif
+
 /*
  * Parse a header
  * return a code and possibly content in the arg
@@ -922,12 +922,26 @@
  * Non-blocking connect(). Does the same as connect(2) but ensures
  * it will time-out after a much shorter time period SERVER_TO
  */
+#ifdef TPROXY_ENABLE
 int
-connect_nb(const int sockfd, const struct addrinfo *serv_addr, const int to)
-{
+connect_nb(const int sockfd, const struct addrinfo *serv_addr, const int to, const struct addrinfo *tp_addr) {
+#else
+int
+connect_nb(const int sockfd, const struct addrinfo *serv_addr, const int to) {
+#endif
     int             flags, res, error;
     socklen_t       len;
     struct pollfd   p;
+#ifdef TPROXY_ENABLE
+    int tp_val = 1;
+    struct sockaddr_in in;
+#endif
+
+    if (! serv_addr->ai_addr)
+    {
+       logmsg(LOG_ERR, "cconnect_nb: wrong serv_addr->ai_addr");
+       return -1;
+    }

     if((flags = fcntl(sockfd, F_GETFL, 0)) < 0) {
         logmsg(LOG_WARNING, "(%lx) connect_nb: fcntl GETFL failed: %s", pthread_self(), strerror(errno));
@@ -937,6 +937,24 @@
         logmsg(LOG_WARNING, "(%lx) connect_nb: fcntl SETFL failed: %s", pthread_self(), strerror(errno));
         return -1;
     }
+
+#ifdef TPROXY_ENABLE
+    if (tp_addr) {
+       memcpy(&in, tp_addr->ai_addr, sizeof(in));
+       in.sin_port = 0;
+       if (setsockopt(sockfd, SOL_IP, IP_TRANSPARENT, &tp_val, sizeof(tp_val)) < 0) {
+           logmsg(LOG_ERR, "(%lx) connect_nb: cannot set TProxy IP_TRANSPARENT socket option. Error: %d, %s.", pthread_self(),
+                            error, strerror(error));
+           return -1;
+       }
+
+       if (bind(sockfd, (struct sockaddr *) &in, sizeof(in)) < 0) {
+           logmsg(LOG_ERR, "(%lx) connect_nb: cannot bind to TProxy IP. Error: %d, %s.", pthread_self(),
+                            error, strerror(error));
+           return -1;
+       }
+    }
+#endif

     error = 0;
     if((res = connect(sockfd, serv_addr->ai_addr, serv_addr->ai_addrlen)) < 0)
@@ -1036,7 +1036,12 @@
         default:
             continue;
         }
-        if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0) {
+#ifdef TPROXY_ENABLE
+        if(connect_nb(sock, &be->ha_addr, be->conn_to, NULL) != 0)
+#else
+        if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0)
+#endif
+       {
             kill_be(svc, be, BE_KILL);
             str_be(buf, MAXBUF - 1, be);
             logmsg(LOG_NOTICE, "BackEnd %s is dead (HA)", buf);
@@ -1072,7 +1072,12 @@
         default:
             continue;
         }
-        if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0) {
+#ifdef TPROXY_ENABLE
+        if(connect_nb(sock, &be->ha_addr, be->conn_to, NULL) != 0)
+#else
+        if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0)
+#endif
+       {
             kill_be(svc, be, BE_KILL);
             str_be(buf, MAXBUF - 1, be);
             logmsg(LOG_NOTICE, "BackEnd %s is dead (HA)", buf);
@@ -1127,7 +1127,12 @@
                 }
                 addr = &be->ha_addr;
             }
-            if(connect_nb(sock, addr, be->conn_to) == 0) {
+#ifdef TPROXY_ENABLE
+           if(connect_nb(sock, addr, be->conn_to, NULL) == 0)
+#else
+           if(connect_nb(sock, addr, be->conn_to) == 0)
+#endif
+           {
                 be->resurrect = 1;
                 modified = 1;
             }
@@ -1196,7 +1196,12 @@
                 }
                 addr = &be->ha_addr;
             }
-            if(connect_nb(sock, addr, be->conn_to) == 0) {
+#ifdef TPROXY_ENABLE
+           if(connect_nb(sock, addr, be->conn_to, NULL) != 0)
+#else
+            if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0)
+#endif
+           {
                 be->resurrect = 1;
                 modified = 1;
             }
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.