Re: Intermittent TLS inbound failure behind HAProxy, Kamailio opens new TCP to HAProxy ephemeral port and gets RST

Hi Han,

Thanks for your reply.

Following up with additional findings from debugging. 
I've identified what's happening but not why, and I'm hoping someone can help explain the root cause.

I placed 10 mobile devices on a table, all registered through the same setup, and left them idle for a few hours. 
After that period, 1 out of the 10 could no longer receive inbound calls, while still being registered and able to place outbound calls. 
After enabling DEBUG logs, I compared _tcpconn_find() behavior:
For 9 out of 10 connections: _tcpconn_find(): found connection by peer address (id: X) appears consistently. INVITEs to these users succeed.
For connection 137 (the failing one): zero occurrences of "found connection by peer address". INVITEs fail with tcp_send(): no open tcp connection found, opening new one, followed by connect failed (RST) Connection refused.
At the same time, I checked and the connection is alive, healthy (CONN_OK), timeout is refreshing, and the client continues to re-REGISTER with incrementing CSeq over this connection.
The kamcmd core.tcp_list, tls.list, and ul.dump output for this connection looks identical before and after the problem appears. 
The connection's src_port: 48996 and Tcpconn-Id: 137 remain unchanged. 
Everything looks normal from the outside.

The issue is that the peer-address alias entry (the mapping from 10.42.106.41:48996 to connection ID 137 in the internal TCP alias hash table) is missing for this connection, while it exists for all others.
What I don't understand (root cause) is why does this specific connection lose its peer-address alias (or never get one) while the other 9 connections are fine?
Under what conditions are TCP connection aliases created and removed in the core? Is there a known scenario where an alias can be lost while the connection itself remains active?
Could there be a hash collision or race condition in the alias hash table that causes some entries to be evicted?

Workaround I'm testing:
I found that lookup() in the registrar module already sets msg->otcpid to the contact's tcpconn_id. 
The TM module will use dst.id = msg->otcpid in t_fwd.c but only if the FL_USE_OTCPID flag is set on the message. 
When dst.id is set, tcp_send() finds the connection by ID directly, bypassing the alias hash table entirely.
Adding tcp_set_otcpid_flag(1) from the tcpops module after lookup() completes this chain.
I'm checking now if this works around the problem by skipping the alias lookup, but it doesn't explain why the alias goes missing in the first place.

Any insights would be greatly appreciated.

Thanks,
Joey
__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions -- [email protected]
To unsubscribe send an email to [email protected]
Important: keep the mailing list in the recipients, do not reply only to the sender!