Re: Getting intermittent mod_jk errors on 1 / 3 web servers

Am 31.10.25 um 09:04 schrieb Mark Thomas:
> On 30/10/2025 14:06, Christopher Schultz wrote:
> 
> <snip/>
> 
>> That symptom, plus the "this is the only server using a NAT gateway" 
>> would surely point to one place: the NAT gateway is killing 
>> connections that are idle and surprising both stunnel and mod_jk. I 
>> can also see a graph of non-zero numbers of "Idle Timeouts" on the NAT 
>> gateway. It doesn't tell me more details about those timeouts, but 
>> they are almost certainly outgoing AJP/stunnel connections.
> 
> Your reasoning above looks sound to me.
> 
>> But.
>>
>> Here is my mod_jk workers configuration:
>>
>> # Template worker
>> worker.template.type=ajp13
>> worker.template.host=localhost
>> worker.template.connection_pool_timeout=60
>> worker.template.socket_timeout=300
>> worker.template.max_packet_size=65536
>>
>> worker.node1.reference=worker.template
>> worker.node1.port=7015
>> worker.node1.route=node1
>>
>> My expectation is that connection_pool_timeout of 60 (seconds) will 
>> close connections which have been idle for 60 seconds. If mod_jk 
>> closes a connection, stunnel will also close that connection. (Note: I 
>> have no explicit connectionTimeout or keepAliveTimeout on the Tomcat 
>> side. But this doesn't seem to be any problem for the other two web 
>> servers.)
>>
>> Checking my configuration for the NAT gateway, it has a fixed idle 
>> timeout of 350 seconds, which is much longer than the 60 seconds I 
>> (believe I) have set for idle AJP connections.
>>
>> I do not use servlet async or Websocket for anything in my 
>> application, so I do not expect long-lasting connections between 
>> client and server.
>>
>> Is there anything I haven't checked at this point?
> 
> You might want to check how connection_pool_timeout interacts with 
> connection_pool_size and connection_pool_minsize.
> 
> I am wondering if the current size is at minsize will an idle connection 
> be closed if it exceeds the timeout?
> 
> I think the code that handles this is here:
> 
> https://github.com/apache/tomcat-connectors/blob/main/native/common/ 
> jk_ajp_common.c#L3510
> 
> and I think it only closes idle connections until the pool reaches minsize.
> 
> Which isn't what the docs suggest. Which in turn might explain why you 
> are seeing connections open longer than the NAT gateway timeout.
> 
> Rainer understands this code far better than I do. It would be good to 
> get his view on this. If I am right, I think we either need to update 
> the docs or we need to fix the code so idle connections below minsize 
> are closed and then re-opened to refresh them.
> 
> Mark

It has been quite a long time that I touched this code. But Mark is 
right, idle connections will not get closed below the min pool size.

Most of the code default values for configuration never reflected the 
code progress and were kept to keep old behavior in place. I typically 
suggest to start with the config from here:

https://github.com/apache/tomcat-connectors/blob/main/conf/workers.properties

That should have better default params (like min pool size 0) and 
comments. See also the other files in that directory.

I like most of our jk timeouts and the cping/cpong, but I generally do 
not like the socket_timeout, because it is a complex to understand low 
level timeout which can kick in in any communication context.

In addition when doing observation/monitoring keep in mind, that the 
connection pools are per child process, not per Apache httpd instance.

Hth a bit.

Rainer