Re: how to let keepalived skip ip:port that is not configured as virtual listener

sorry I missed to mention I am using a custom ipvs in kernel and I
found out the there is custom code drop the request if the ip:port
does not match any virtual listener, yes, the issue is in kernel, not
keepalived.

by the way, the default upstream kernel ipvs should pass through the
request to local process if no virtual listener match. so should be no
issue.

static int
tcp_conn_schedule(int af, struct sk_buff *skb, struct ip_vs_proto_data *pd,
                  int *verdict, struct ip_vs_conn **cpp,
                  struct ip_vs_iphdr *iph)
{
        struct net *net;
        struct ip_vs_service *svc;
        struct tcphdr _tcph, *th;
        struct netns_ipvs *ipvs;

        th = skb_header_pointer(skb, iph->len, sizeof(_tcph), &_tcph);
        if (th == NULL) {
                *verdict = NF_DROP;
                return 0;
        }
        net = skb_net(skb);
        ipvs = net_ipvs(net);
        /* No !th->ack check to allow scheduling on SYN+ACK for Active FTP */
        rcu_read_lock();
        if ((th->syn || sysctl_sloppy_tcp(ipvs)) && !th->rst &&
            (svc = ip_vs_service_find(net, af, skb->mark, iph->protocol,
                                      &iph->daddr, th->dest))) {
                int ignored;

......................CUT.....................

        }
             /* NF_ACCEPT */
        return 1;
}


On Wed, Nov 12, 2014 at 11:32 PM, Alexey Andriyanov <[email protected]> wrote:
> Does the issue reproduce if you configure IPVS manually using ipvsadm and do not use keepalived?
> Looks like it is not a keepalived issue.
>
> 11.11.2014 23:14, Vincent Li wrote:
>> Hi,
>>
>> I have both LVS+keepalived and nginx running on same box and came
>> across this issue, configuration below:
>>
>> keepalived.conf: (So LVS should handle 10.1.72.36:80)
>>
>>
>> vrrp_instance VI_1 {
>>     state MASTER
>> .................CUT..........
>> ...
>>     virtual_ipaddress {
>>         10.1.72.38/16 dev eth2
>>         10.1.72.36/16 dev eth2 #<========
>>     }
>> }
>>
>> virtual_server 10.1.72.36 80 { <===========
>>     delay_loop 90
>>     lb_algo rr
>> #    persistence_timeout 50
>> .................CUT........
>> }
>>
>> nginx.conf  (nginx should handle 10.1.72.36:8080)
>>
>>     server {
>>         listen       10.1.72.36:8080;
>>
>>         location / {
>>             root   html;
>>             index  index.html index.htm;
>>
>>         }
>>
>>     }
>>
>> so virtual 10.1.72.36:80 is handled fine by LVS as expected.
>>
>> but when I access 10.1.72.36:8080, since there is no virtual
>> configuration for 10.1.72.36:8080, I am hopping nginx will handle it,
>> but it appears keepalived think 10.1.72.36:8080 should still be
>> handled by LVS and client got no SYN+ACK back, tried to turn on more
>> verbose keepalived debug, no debug message tells why.
>>
>> LVS log:
>>
>> Nov 11 11:25:53 vincent-centos64-2 kernel: IPVS: lookup service: fwm 0
>> TCP 10.1.72.36:8080 not hit
>>
>> Nov 11 11:25:53 vincent-centos64-2 kernel: IPVS: lookup TCP
>> 10.1.72.6:36220->10.1.72.36:8080 not hit
>>
>> Nov 11 11:25:53 vincent-centos64-2 kernel: IPVS: lookup TCP
>> 10.1.72.6:36220->10.1.72.36:8080 not hit
>>
>> if I stop keepalived, 10.1.72.36:8080 is handled by ngnix correctly,
>> or if I use uniq ip address as listener in keepalived and nginx, it
>> works as expected, for example 10.1.72.38:80 in keepalived,
>> 10.1.72.36:8080 in nginx, it works.
>>
>> so it looks I can't use same IP address in LVS and nginx as listener,
>> even with different port.
>>
>> is there anyway to skip keepalived/LVS handling when there is no
>> 10.1.72.36:8080 in keepalived configuration? I thought this might be
>> related to how keepalived communicated to LVS regarding virtual
>> listener definition, any clue?
>>
>> Vincent
>
>
> --
> Best regards,
> Alexey

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk