Re: [Pound Mailing List] How to count active threads

Hi List,

just for the sake of completeness, here is a basic outline of how we 
solved this issue for us (e.g. how to determine how many of the 
allocated threads are actually busy):

Basically, we run multiple pound instances on a single server that deal 
with multiple backends on separate servers. As the script itself is 
closely tied to our setup, I will try to explain what we are doing, 
rather than just pasting some random code.


We created a simple bash script, that
- knows a way to enumerate all pound instances on the local system
- checks if each one is running
- if so, it checks the syscall info from /proc for all sub-threads of 
the given pound:

  /proc/POUND_PROCESS_ID/task/*/syscall

- it seems that you can determine idle threads because their last 
syscall was 202 (which equals getegid() )
- active threads usually show 7 (waitpid()) or 35 (ftime())
- with these information, you can count (or rather guesstimate) active + 
inactive pounds

In our case, we use the bash script to write the numbers out to a 
temporary file, which is then read by a snmpd perl extension upon 
request that exports all numbers as some sort of snmp table. This table 
is being read by cacti plugins which in turn create simple graphs for 
each instance that is running on the system. You can always use the 
cacti GUI to rediscover all running pound instances if there are new 
ones and have cacti create new graphs for those as well (without doing 
any changes to the bash script, the snmp extension or the cacti templates).

The SNMP approach is somewhat generic - it just needs a file with all 
the relevant information in it. How those numbers are actually 
calculated, that is up to the bash script running as a cronjob in the 
background. That way, we can always change the way of calculating the 
numbers without breaking the snmp extension or the cacti graphs/templates.

This is an example line from our stats file:

[POUND]: some-pound-instance-name, [DEFINED_THREADS]: 128, [THREADS]: 
130, [ADDITIONAL_THREADS]: 2, [ACTIVE]: 2, [INACTIVE]: 128, [UNKNOWN]: 
0, [USAGE]: 1

You see an idle pound instance that has 128 threads configured in its 
config file. 130 Threads are actually running (these two additional 
threads are not connection-worker-threads but pound-internal threads).

"Unknown" threads are those that have a syscall other than the ones 
mentioned above. This should always be zero or can otherwise be used to 
improve the accuracy of the script.


This is not in production yet - we have only used it on our testbed. 
Cacti uses these numbers to produce nice graphs, Icinga checks if any 
pound instance reaches a critical amount of active threads. That way we 
can always increase our threads limit before running into trouble (or 
decrease, if necessary).

Please let us know what you think. If anyone is interested, we might be 
able to publish the snmp extension along with the cacti configuration 
and the icinga check command and some code snippets that might help 
implementing the process/thread checker on the server itself.

On 10/16/2013 06:38 AM, Leo wrote:
> Hi,
>
> I have already complained about the new threading model in version 2.6
> (http://www.apsis.ch/pound/pound_list/archive/2012/2012-01/1326878271000#1326878271000)
> and some of my suggestions have been added to the feature list for Pound 2.7
> (http://www.apsis.ch/pound/pound_list/archive/2012/2012-02/1328374222000#1328374222000).
>
> Current beta version of 2.7 shows the length of the request queue
> (http://www.apsis.ch/pound/pound_list/archive/2012/2012-04/1333979077000#1333979077000).
> But I suspect that the old (dynamic) threading model will not come back.
>
> Regarding a release date for version 2.7 you have to ask Robert Segall ;-)
>
> Hope this helps ...
>
> Regards,
> Leo
>
>
> On 10/15/2013 04:27 PM, Rudolph Bott wrote:
>> Hello List,
>>
>> we are currently switching from pound 2.5 to 2.6, which introduces us to the static thread model. Of course it is pretty easy to count the current threads on our pound processes to find a suitable number of threads to configure. But how would you actually monitor this in the future so that you can
>> readjust the number of threads if needed? Since there are always N threads allocated and there is no information regarding threads available via the pound socket - how would you actually count the usage?
>>
>>
>> Besides that, is there a release date for pound 2.7? Comparing the debian changelog to the 2.7 feature list that was discussed here earlier, some of the new features seem to be already integrated by debian. However, we would welcome a way to use dynamic threads again OR a way to monitor threads so
>> that we know when to readjust settings.
>>
>>
>
>
> --
> To unsubscribe send an email with subject unsubscribe to [email protected].
> Please contact [email protected] for questions.
>


-- 
Mit freundlichen Grüßen / With kind regards
   Rudolph Bott

--
To unsubscribe send an email with subject unsubscribe to [email protected]
Please contact [email protected] for questions.