Re: [Pound Mailing List] Pound 2.7 and TProxy
Scott McKeown <scott-Oteo70RhwZ77w/[email protected]>
| Newsgroups | gmane.comp.web.pound.general |
|---|---|
| Message-ID | <[email protected]> |
Hi Everyone, Right we finally got this working so anyone that is also wanting to use TProxy with Pound v2.7 the patch file is attached. patch -i pound_2.7_with_tproxy.patch --verbose --ignore-whitespace On 20 October 2015 at 16:19, Scott McKeown <scott-Oteo70RhwZ77w/[email protected]> wrote: > Hi Joe, > > Thanks for the reply. > > Yes that TPROXY=1 is set when running the make command > > make TPROXY=1 > > However, we 'think' we may have found the issue now after some more > digging around looks like I'm missing a section from within the config.c > file to allow the TProxy directive to be understood but I'm going to have > another look tomorrow now. > > Likewise I'll do a gitpull on your branch and give the 'IPTransparent' > option ago. > > > ~Scott > > On 20 October 2015 at 16:04, Joe Gooch <[email protected]> wrote: > >> Did you change TPROXY=1 in the Makefile? Looks like unless you do that, >> it's not included. >> >> The other option I see is to use my version of a similar thing: >> >> https://github.com/goochjj/pound/commit/65e14aa8b52f9170f513399bfe430a1c66a9e34b >> >> Your kernel headers need to include the IP options, and then transproxy >> works for ipv6 listeners, freebind for ipv4 listeners. >> -- >> Joe >> >> Confidentiality Notice: This e-mail transmission may contain confidential >> and legally privileged information that is intended only for the individual >> named in the e-mail address. If you are not the intended recipient, you are >> hereby notified that any disclosure, copying, distribution, or reliance >> upon the contents of this e-mail message is strictly prohibited. If you >> have received this e-mail transmission in error, please reply to the >> sender, so that proper delivery can be arranged, and please delete the >> message from your mail box. >> >> >> >> Joseph Gooch >> >> www.sapphirek12.org | office: (866) 366-9540 >> >> >> >> *CONFIDENTIALITY STATEMENT* >> >> *The documents and communication included in this email transmission may >> contain confidential information. All information is intended only for the >> use of the above named recipient(s). If you are not the named recipient, >> you are NOT authorized to read, disclose, copy, distribute, or take any >> action on the information and any action other than immediate delivery to >> the named recipient is strictly prohibited. If you have received this email >> in error, do NOT read the information and please immediately notify sender >> by telephone and email and immediately delete this email. If you are the >> named recipient, you are NOT authorized to reveal any of this information >> to any unauthorized person and are hereby instructed to delete this email >> when no longer needed. * >> >> From: Scott McKeown <scott-Oteo70RhwZ77w/[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Tuesday, October 20, 2015 at 5:25 AM >> To: Pound Mailing List <[email protected]> >> Subject: [Pound Mailing List] Pound 2.7 and TProxy >> >> Hello Everyone, >> >> I've finally got to admit defeat on getting this patch to work and I was >> wondering if anyone out there could please give me a hand getting this to >> work. >> >> I've attached the patch file for anyone to play with but what I'm trying >> to do is get Unix TProxy to work with pound 2.7 again, the original version >> of this patch has been working since pound v2.4. >> >> Anyhow, the patch compiles into the latest version but if you add "TProxy >> 1" into the configuration you get a 'unknown directive' error >> >> >> *Config File:* >> # cat /etc/pound/pound.cfg >> # Pound configuration file generated by loadbalancer.org appliance >> User "nobody" >> Group "nobody" >> LogLevel 0 >> Client 30 >> Timeout 60 >> Threads 250 >> TProxy 1 >> >> ListenHTTPS >> # Label: pound1 >> Address 192.168.63.59 >> Port 443 >> Cert "/etc/loadbalancer.org/certs/server.pem" >> xHTTP 3 >> ReWriteLocation 1 >> Ciphers >> "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS" >> SSLHonorCipherOrder 1 >> SSLAllowClientRenegotiation 0 >> Disable SSLv2 >> Disable SSLv3 >> Service >> BackEnd >> Address 192.168.64.254 >> Port 80 >> TProxy 1 >> End >> End >> End >> >> >> *Start Up:* >> # /usr/local/sbin/pound >> starting... >> detect_tproxy(): tproxy is is detected >> tproxy: available >> /etc/pound/pound.cfg line 8: unknown directive - aborted >> >> >> >> >> -- >> With Kind Regards. >> >> Scott McKeown >> Loadbalancer.org >> http://www.loadbalancer.org >> Tel (UK) - +44 (0) 3303801064 (24x7) >> Tel (US) - +1 888.867.9504 (Toll Free)(24x7) >> > > > > -- > With Kind Regards. > > Scott McKeown > Loadbalancer.org > http://www.loadbalancer.org > Tel (UK) - +44 (0) 3303801064 (24x7) > Tel (US) - +1 888.867.9504 (Toll Free)(24x7) > -- With Kind Regards. Scott McKeown Loadbalancer.org http://www.loadbalancer.org Tel (UK) - +44 (0) 3303801064 (24x7) Tel (US) - +1 888.867.9504 (Toll Free)(24x7)
pound_2.7_with_tproxy.patch
(application/octet-stream, 14.3 KB)
diff -urN config.c config.c
--- config.c
+++ config.c
@@ -81,6 +81,9 @@
static regex_t ClientCert, AddHeader, DisableProto, SSLAllowClientRenegotiation, SSLHonorCipherOrder, Ciphers;
static regex_t CAlist, VerifyList, CRLlist, NoHTTPS11, Grace, Include, ConnTO, IgnoreCase, HTTPS;
static regex_t Disabled, Threads, CNName, Anonymise, ECDHCurve;
+#ifdef TPROXY_ENABLE
+static regex_t TProxy;
+#endif
static regmatch_t matches[5];
@@ -250,6 +250,9 @@
res->url = NULL;
res->next = NULL;
has_addr = has_port = 0;
+#ifdef TPROXY_ENABLE
+ res->tp_enabled = 0;
+#endif
pthread_mutex_init(&res->mut, NULL);
while(conf_fgets(lin, MAXBUF)) {
if(strlen(lin) > 0 && lin[strlen(lin) - 1] == '\n')
@@ -415,6 +415,10 @@
if((res->addr.ai_family == AF_INET || res->addr.ai_family == AF_INET6) && !has_port)
conf_err("BackEnd missing Port - aborted");
return res;
+#ifdef TPROXY_ENABLE
+ } else if(!regexec(&TProxy, lin, 4, matches, 0)) {
+ if (enable_tproxy && have_tproxy) res->tp_enabled = atoi(lin + matches[1].rm_so);
+#endif
} else {
conf_err("unknown directive");
}
@@ -1282,6 +1282,13 @@
#endif
}
return res;
+#ifdef TPROXY_ENABLE
+ } else if(!regexec(&TProxy, lin, 4, matches, 0)) {
+ if (have_tproxy)
+ enable_tproxy = atoi(lin + matches[1].rm_so);
+ else
+ enable_tproxy = 0;
+#endif
} else {
conf_err("unknown directive");
}
@@ -1434,7 +1434,16 @@
}
} else if(!regexec(&Anonymise, lin, 4, matches, 0)) {
anonymise = 1;
- } else {
+#ifdef TPROXY_ENABLE
+ } else if(!regexec(&TProxy, lin, 4, matches, 0)) {
+ if (have_tproxy)
+ enable_tproxy = atoi(lin + matches[1].rm_so);
+ else
+ enable_tproxy = 0;
+#endif
+
+
+ } else {
conf_err("unknown directive - aborted");
}
}
@@ -1492,6 +1492,9 @@
|| regcomp(&VerifyList, "^[ \t]*VerifyList[ \t]+\"(.+)\"[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
|| regcomp(&CRLlist, "^[ \t]*CRLlist[ \t]+\"(.+)\"[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
|| regcomp(&NoHTTPS11, "^[ \t]*NoHTTPS11[ \t]+([0-2])[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
+#ifdef TPROXY_ENABLE
+ || regcomp(&TProxy, "^[ \t]*TProxy[ \t]+([01])[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
+#endif
|| regcomp(&Include, "^[ \t]*Include[ \t]+\"(.+)\"[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
|| regcomp(&ConnTO, "^[ \t]*ConnTO[ \t]+([1-9][0-9]*)[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
|| regcomp(&IgnoreCase, "^[ \t]*IgnoreCase[ \t]+([01])[ \t]*$", REG_ICASE | REG_NEWLINE | REG_EXTENDED)
@@ -1662,6 +1662,9 @@
regfree(&VerifyList);
regfree(&CRLlist);
regfree(&NoHTTPS11);
+#ifdef TPROXY_ENABLE
+ regfree(&TProxy);
+#endif
regfree(&Include);
regfree(&ConnTO);
regfree(&IgnoreCase);
diff -urN http.c http.c
--- http.c
+++ http.c
@@ -855,7 +855,12 @@
clean_all();
return;
}
- if(connect_nb(sock, &backend->addr, backend->conn_to) < 0) {
+
+#ifdef TPROXY_ENABLE
+ if(connect_nb(sock, &backend->addr, backend->to, backend->tp_enabled ? &from_host : NULL) < 0) {
+#else
+ if(connect_nb(sock, &backend->addr, backend->to) < 0) {
+#endif
str_be(buf, MAXBUF - 1, backend);
logmsg(LOG_WARNING, "(%lx) backend %s connect: %s", pthread_self(), buf, strerror(errno));
shutdown(sock, 2);
--- Makefile.in
+++ Makefile.in
@@ -25,9 +25,11 @@
CC=@PTHREAD_CC@
+C_TPROXY=-DTPROXY_ENABLE
+
CFLAGS=-DF_CONF=\"@sysconfdir@/pound.cfg\" -DVERSION=\"@PACKAGE_VERSION@\" -DC_SSL=\"@C_SSL@\" -DC_T_RSA=\"@C_T_RSA@\" \
-DC_DH_LEN=\"@C_DH_LEN@\" -DC_MAXBUF=\"@C_MAXBUF@\" -DC_OWNER=\"@C_OWNER@\" -DC_GROUP=\"@C_GROUP@\" \
- -DC_SUPER=\"@C_SUPER@\" -DC_CERT1L=\"@C_CERT1L@\" @CFLAGS@ @PTHREAD_CFLAGS@ @CPPFLAGS@
+ -DC_SUPER=\"@C_SUPER@\" -DC_CERT1L=\"@C_CERT1L@\" @CFLAGS@ @PTHREAD_CFLAGS@ @CPPFLAGS@ ${C_TPROXY}
LIBS=@LIBS@ @PTHREAD_LIBS@
prefix=@prefix@
diff -urN pound.c pound.c
--- pound.c
+++ pound.c
@@ -41,6 +41,10 @@
print_log, /* print log messages to stdout/stderr */
grace, /* grace period before shutdown */
control_sock; /* control socket */
+#ifdef TPROXY_ENABLE
+int have_tproxy, /* is transparent proxy available */
+ enable_tproxy; /* is tproxy enabled */
+#endif
SERVICE *services; /* global services (if any) */
@@ -246,11 +246,23 @@
#ifndef SOL_TCP
struct protoent *pe;
#endif
+#ifdef TPROXY_ENABLE
+ typedef struct __user_cap_header_struct tp_cap_header_t;
+ typedef struct __user_cap_data_struct tp_cap_data_t;
+ tp_cap_header_t tp_cap_header;
+ tp_cap_data_t tp_cap_data;
+#endif
print_log = 0;
(void)umask(077);
control_sock = -1;
log_facility = -1;
+#ifdef TPROXY_ENABLE
+ have_tproxy = 0;
+ tp_cap_header.version = _LINUX_CAPABILITY_VERSION;
+ tp_cap_header.pid = 0;
+ tp_cap_data.effective = tp_cap_data.permitted = tp_cap_data.inheritable = 0;
+#endif
logmsg(LOG_NOTICE, "starting...");
signal(SIGHUP, h_shut);
@@ -320,6 +320,19 @@
}
SOL_TCP = pe->p_proto;
#endif
+
+#ifdef TPROXY_ENABLE
+ if (! detect_tproxy()) {
+ if (geteuid() == 0) {
+ have_tproxy = 1;
+ logmsg(LOG_INFO, "tproxy: available");
+ } else {
+ logmsg(LOG_ERR, "tproxy: disabled. you must start pound with root privileges. later it will drop privileges.");
+ }
+ } else {
+ logmsg(LOG_INFO, "tproxy: not supported or not enough privileges");
+ }
+#endif
/* read config */
config_parse(argc, argv);
@@ -450,6 +450,36 @@
}
}
+#ifdef TPROXY_ENABLE
+ if (enable_tproxy)
+ {
+ if (capget(&tp_cap_header, &tp_cap_data) != 0)
+ {
+ logmsg(LOG_ERR, "capabilities: can't get capabilities");
+ exit(1);
+ }
+ if (prctl(PR_SET_KEEPCAPS, 1) < 0)
+ {
+ logmsg(LOG_ERR, "capabilities: can't enable keep capabilities");
+ exit(1);
+ }
+
+ tp_cap_data.effective = tp_cap_data.permitted = tp_cap_data.inheritable = 0;
+ tp_cap_data.effective |= (1 << CAP_NET_ADMIN);
+ tp_cap_data.effective |= (1 << CAP_SETUID);
+ tp_cap_data.effective |= (1 << CAP_SETGID);
+ tp_cap_data.permitted |= (1 << CAP_NET_ADMIN);
+ tp_cap_data.permitted |= (1 << CAP_SETUID);
+ tp_cap_data.permitted |= (1 << CAP_SETGID);
+
+ if (capset(&tp_cap_header, &tp_cap_data) != 0)
+ {
+ logmsg(LOG_ERR, "capabilities: can't set capabilities");
+ exit(1);
+ }
+ }
+#endif
+
if(group)
if(setgid(group_id) || setegid(group_id)) {
logmsg(LOG_ERR, "setgid: %s - aborted", strerror(errno));
@@ -491,6 +491,32 @@
exit(1);
}
+#ifdef TPROXY_ENABLE
+ if (enable_tproxy)
+ {
+ if (capget(&tp_cap_header, &tp_cap_data) != 0)
+ {
+ logmsg(LOG_ERR, "user capabilities: can't get capabilities");
+ exit(1);
+ }
+ if (prctl(PR_SET_KEEPCAPS, 1) < 0)
+ {
+ logmsg(LOG_ERR, "user capabilities: can't enable keep capabilities");
+ exit(1);
+ }
+
+ tp_cap_data.effective = tp_cap_data.permitted = tp_cap_data.inheritable = 0;
+ tp_cap_data.effective |= (1 << CAP_NET_ADMIN);
+ tp_cap_data.permitted |= (1 << CAP_NET_ADMIN);
+
+ if (capset(&tp_cap_header, &tp_cap_data) != 0)
+ {
+ logmsg(LOG_ERR, "user capabilities: can't set capabilities");
+ exit(1);
+ }
+ }
+#endif
+
/* split off into monitor and working process if necessary */
for(;;) {
#ifdef UPER
diff -urN pound.h pound.h
--- pound.h
+++ pound.h
@@ -241,6 +241,14 @@
#define const
#endif
+#ifdef TPROXY_ENABLE
+#include <sys/prctl.h>
+#include <linux/capability.h>
+#ifndef IP_TRANSPARENT
+#define IP_TRANSPARENT 19
+#endif
+#endif
+
#ifdef HAVE_LONG_LONG_INT
#define LONG long long
#define L0 0LL
@@ -275,6 +275,11 @@
grace, /* grace period before shutdown */
control_sock; /* control socket */
+#ifdef TPROXY_ENABLE
+extern int have_tproxy, /* is tproxy available */
+ enable_tproxy; /* is tproxy enabled */
+#endif
+
extern regex_t HEADER, /* Allowed header */
CHUNK_HEAD, /* chunk header line */
RESP_SKIP, /* responses for which we skip response */
@@ -320,6 +320,9 @@
int to; /* read/write time-out */
int conn_to; /* connection time-out */
struct addrinfo ha_addr; /* HA address/port */
+#ifdef TPROXY_ENABLE
+ int tp_enabled; /* TProxy is enabled */
+#endif
char *url; /* for redirectors */
int redir_req; /* the redirect should include the request path */
SSL_CTX *ctx; /* CTX for SSL connections */
@@ -579,7 +579,12 @@
* Non-blocking version of connect(2). Does the same as connect(2) but
* ensures it will time-out after a much shorter time period CONN_TO.
*/
+#ifdef TPROXY_ENABLE
+extern int detect_tproxy(void);
+extern int connect_nb(const int, const struct addrinfo *, const int, const struct addrinfo *);
+#else
extern int connect_nb(const int, const struct addrinfo *, const int);
+#endif
/*
* Parse arguments/config file
diff -urN svc.c svc.c
--- svc.c
+++ svc.c
@@ -372,6 +372,40 @@
return res - kp_res;
}
+#ifdef TPROXY_ENABLE
+/*
+ * detect_tproxy
+ * check for TPROXY
+ *
+ * Thanks to Loadbalancer.org, Inc. especially to Malcolm Turnbull for sponsorship
+ * Thanks to Sianet Business Hosting especially to Rodrigo L. L. Jorge for sponsorship
+ *
+ */
+int
+detect_tproxy(void)
+{
+ int sock, tp_val;
+
+ tp_val = 1;
+
+ sock = socket(AF_INET, SOCK_STREAM, 0);
+ if (sock < 0) {
+ logmsg(LOG_WARNING, "detect_tproxy(): socket SOCK_STREAM failed: %s", strerror(errno));
+ return 1;
+ }
+ if (setsockopt(sock, SOL_IP, IP_TRANSPARENT, &tp_val, sizeof(tp_val)) < 0) {
+ logmsg(LOG_INFO, "detect_tproxy(): tproxy is not available");
+ close(sock);
+ return 1;
+ }
+
+ close(sock);
+ logmsg(LOG_INFO, "detect_tproxy(): tproxy is is detected");
+
+ return 0;
+}
+#endif
+
/*
* Parse a header
* return a code and possibly content in the arg
@@ -922,12 +922,26 @@
* Non-blocking connect(). Does the same as connect(2) but ensures
* it will time-out after a much shorter time period SERVER_TO
*/
+#ifdef TPROXY_ENABLE
int
-connect_nb(const int sockfd, const struct addrinfo *serv_addr, const int to)
-{
+connect_nb(const int sockfd, const struct addrinfo *serv_addr, const int to, const struct addrinfo *tp_addr) {
+#else
+int
+connect_nb(const int sockfd, const struct addrinfo *serv_addr, const int to) {
+#endif
int flags, res, error;
socklen_t len;
struct pollfd p;
+#ifdef TPROXY_ENABLE
+ int tp_val = 1;
+ struct sockaddr_in in;
+#endif
+
+ if (! serv_addr->ai_addr)
+ {
+ logmsg(LOG_ERR, "cconnect_nb: wrong serv_addr->ai_addr");
+ return -1;
+ }
if((flags = fcntl(sockfd, F_GETFL, 0)) < 0) {
logmsg(LOG_WARNING, "(%lx) connect_nb: fcntl GETFL failed: %s", pthread_self(), strerror(errno));
@@ -937,6 +937,24 @@
logmsg(LOG_WARNING, "(%lx) connect_nb: fcntl SETFL failed: %s", pthread_self(), strerror(errno));
return -1;
}
+
+#ifdef TPROXY_ENABLE
+ if (tp_addr) {
+ memcpy(&in, tp_addr->ai_addr, sizeof(in));
+ in.sin_port = 0;
+ if (setsockopt(sockfd, SOL_IP, IP_TRANSPARENT, &tp_val, sizeof(tp_val)) < 0) {
+ logmsg(LOG_ERR, "(%lx) connect_nb: cannot set TProxy IP_TRANSPARENT socket option. Error: %d, %s.", pthread_self(),
+ error, strerror(error));
+ return -1;
+ }
+
+ if (bind(sockfd, (struct sockaddr *) &in, sizeof(in)) < 0) {
+ logmsg(LOG_ERR, "(%lx) connect_nb: cannot bind to TProxy IP. Error: %d, %s.", pthread_self(),
+ error, strerror(error));
+ return -1;
+ }
+ }
+#endif
error = 0;
if((res = connect(sockfd, serv_addr->ai_addr, serv_addr->ai_addrlen)) < 0)
@@ -1036,7 +1036,12 @@
default:
continue;
}
- if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0) {
+#ifdef TPROXY_ENABLE
+ if(connect_nb(sock, &be->ha_addr, be->conn_to, NULL) != 0)
+#else
+ if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0)
+#endif
+ {
kill_be(svc, be, BE_KILL);
str_be(buf, MAXBUF - 1, be);
logmsg(LOG_NOTICE, "BackEnd %s is dead (HA)", buf);
@@ -1072,7 +1072,12 @@
default:
continue;
}
- if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0) {
+#ifdef TPROXY_ENABLE
+ if(connect_nb(sock, &be->ha_addr, be->conn_to, NULL) != 0)
+#else
+ if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0)
+#endif
+ {
kill_be(svc, be, BE_KILL);
str_be(buf, MAXBUF - 1, be);
logmsg(LOG_NOTICE, "BackEnd %s is dead (HA)", buf);
@@ -1127,7 +1127,12 @@
}
addr = &be->ha_addr;
}
- if(connect_nb(sock, addr, be->conn_to) == 0) {
+#ifdef TPROXY_ENABLE
+ if(connect_nb(sock, addr, be->conn_to, NULL) == 0)
+#else
+ if(connect_nb(sock, addr, be->conn_to) == 0)
+#endif
+ {
be->resurrect = 1;
modified = 1;
}
@@ -1196,7 +1196,12 @@
}
addr = &be->ha_addr;
}
- if(connect_nb(sock, addr, be->conn_to) == 0) {
+#ifdef TPROXY_ENABLE
+ if(connect_nb(sock, addr, be->conn_to, NULL) != 0)
+#else
+ if(connect_nb(sock, &be->ha_addr, be->conn_to) != 0)
+#endif
+ {
be->resurrect = 1;
modified = 1;
}