Fw: [ITM6] Monitoring Agent for Microsoft SQL Server / Problem Running Multiple Agents

EPHEMERAL:Y is the way to go. You can run as many agents as you want 
because in the mode there is *NO* explicit agent listening port.

The historical data collection is influenced and you have three choices.

1) no historical data
2) Configure historical data collection on the TEMS
3) Configure historical data collection on the TEMS and run WPA on the 
same system as the TEMS

Several large customer have assured me that (3) is the very best way to 
get scalability and reliability - with or without EPHEMERAL:Y.

I really don't think a complicated set of  SKIPs and COUNTs is a good way 
to proceed, it is too fragile.... one thing goes wrong and they call fall 
down.

Incidently, you can just put one or two agents on EPHEMERAL:Y and that 
frees up listening ports.

Another interesting way to go is if you have two separate interfaces [ip 
addresses] on the server.

You can have up to 16 agents on each interface. In that case you need to 
do a

KDEB_INTERFACELIST=!<ip_address or name>

for the agents running on that interface.  Depending on the platform you 
may also need

KDEB_INTERFACELIST_IPV6=-

to remove conflicts with the IPV6 interfaces,

Anyway - lots of options.

I used to think the customer would have to be crazy to run so many agents. 
Then I worked with a customer who had two LARGE unix boxes that typically 
ran 20,000 to 25,000 processes in production.

Regards, 

  



John Alvord - Ph: 1-720-396-2788    Cell: none 
Customer Support - Tivoli Software - jalvord-r/[email protected]
Advisory Engineer - Tivoli Monitoring - ITM infrastructure
Follow us on Twitter! @Tivolisupport and Facebook
Personalize your support needs with the IBM Support Portal
Use Service Request to get assistance!
For emails regarding a PMR, copy [email protected]
For secure browser uploads: https://www.ecurep.ibm.com/app/upload
For Customer Support guidance: IBM Software Support Handbook






----- Forwarded by John Alvord/Menlo Park/IBM on 02/23/2013 06:49 AM -----

From:   "[email protected]" <[email protected]>
To:     "Discussion list for Tivoli product and Tivoli Ready products." 
<[email protected]>, 
Date:   02/22/2013 07:29 PM
Subject:        Re: [TME10] [ITM6] Monitoring Agent for Microsoft SQL 
Server /        Problem Running Multiple Agents
Sent by:        [email protected]



Martin: You have your logical bases covered on this, and I think you're 
correct about the root cause being socket contention.  The easy solution 
would be to setup "EPHEMERAL:Y" in the KDC_FAMILIES string in each agent 
instance's koqenv file--this parameter gets around the max-15-agent 
limitation. Thus, if that really is the issue, this would address it. You 
could also just specify "EPHEMERAL:inbound" just once in the *ms*config 
file of the TEMS these agents connect to.  However, given that there are 
some additional configuration items to be wary of using this parameter 
(you'll either need to locate the  Warehouse Proxy agent on the TEMS these 
agents connect to, or if not, you'll need to use the KDE Gateway feature 
to route data exports correctly), I think the following is 
preferable--plus it's more interesting from a investigative perspective.

a) Edit each instance's %CANDLE_HOME%\tmaitm6\koqenv file (might be under 
..\tmaitm6_x64) to hardcode the KDC_FAMILIES variable going sequentially 
up the SKIP chain and using "COUNT:1" in each as well just to ensure they 
only ever try RPC bindings on the one port--we'll place the COUNT/SKIP 
parameters to make them apply to IP.SPIPE first since that's your 
preferred protocol:

NOTE1: SKIP:1 is the default so we don't need to add it in the first one
NOTE2: In the below lines, both PIPE and SPIPE are enabled, but SPIPE is 
always used first to talk to the TEMS

KOQ Agent Instance 1
KDC_FAMILIES=IP.PIPE PORT:1918 IP use:n SNA use:n IP.SPIPE COUNT:1 
PORT:3660

KOQ Agent Instance 2
KDC_FAMILIES=IP.PIPE PORT:1918 IP use:n SNA use:n IP.SPIPE COUNT:1 SKIP:2 
PORT:3660

KOQ Agent Instance 3
KDC_FAMILIES=IP.PIPE PORT:1918 IP use:n SNA use:n IP.SPIPE COUNT:1 SKIP:3 
PORT:3660
....
KOQ Agent Instance 12
KDC_FAMILIES=IP.PIPE PORT:1918 IP use:n SNA use:n IP.SPIPE COUNT:1 SKIP:12 
PORT:3660

b) Once complete ensure that not only all KOQ agents, but also any 
ITM-related agents/components on the Windows host are stopped.

c) Run the following command from a command prompt and make certain no 
lines are returned--we want to be sure nothing is listening on any of the 
IP.SPIPE ports that the 12 KOQ agents might occupy.  NOTE: when you pass 
'findstr' a series of text in quotation marks, it doesn't treat it as a 
literal string but instead treats each space-separate string as a distinct 
item to search for, basically OR'ing the whole thing:

netstat -abn | findstr LISTEN | findstr "7756 11852 15948 20044 24140 
28236 32332 36428 40524 44620 48716 52812 56908 61004 65100"

d) Beginning with one KOQ agent, start it up from the MTEMS UI.  Wait up 
to 10 seconds and repeat the command from step c) and ensure you see 7756 
as a LISTENER.  It's okay if you see it twice bound to both "0.0.0.0" and 
"[::}"

e) Repeat step d) 11 more times for the remaining KOQ agent instances, 
ensuring you continue to see the subsequent higher LISTENER port show up 
in the netstat output.

f) Login to the TEPS and see whether they have all come online and that 
you can view real-time data for each by navigating to a workspace 
underneath the particular agent in the navigator tree.

This should work, but if the problem is something other than socket 
contention, then we'll just have to goto the koq RAS1 logs for a look.  
You can then start the KNT and custom agents without problem as well, but 
you might also want to edit their KDC_FAMILIES lines to add in "COUNT:1 
SKIP:13" and "COUNT:1 SKIP:14" to allow their startup to proceed more 
quickly, otherwise they'll try each of the first 12 ports before finally 
binding to 56908 and 61004, respectively.

Good luck and let us know the result...

Toben

On Fri, Feb 22, 2013 at 5:26 PM, Martin Butler <[email protected]> wrote:

I have a Windows 2003 (64 bit) server with three varieties of MSSQL
running, and 17 unique instances, all of which have to be monitored.

I have a [standard] base OS agent and a custom OS agent installed on this
server.

After remotely pushing the SQL agent to 12 instances (SQL 2005 & 2008), 
all
of which worked perfectly, the thirteenth installation completed but never
showed up on the TEMS/TEPS despite running locally on the server.
The only way to get this (and any subsequent) agents to connect to the hub
was to change the protocol to ip.pipe  (I want to use ip.spipe for all
agents if possible)

I think I am hitting a network port contention (baseport + N*4096) 
although
I expcected that on agent number 17 (not 15).

I have since tried stopping/starting agents using the MTEMS and the 
Service
Control Panel with varying results.  In some cases, changing the protocol
from ip.spipe to ip.pipe (and back again) resolved my problem. In other
cases, this had no bearing.

Of the 12 instance agents that I originally had working, I can now only 
run
7 fully functional.

Is it possible that I have locked ports up ?  If so, for how long ?

I have read about SKIP and COUNT modifiers to control the port search
algorithm.  Has anyone had success using this on a similar scale ?


Target server:
Operating System:  Windows 2003 (64 bit)
SQL Server 2000 (32 bit)
SQL Server 2005 (32 & 64 bit)
SQL Server 2008 (64 bit)


Agents
Monitoring Agent for Windows OS  / 06.22.04.00  (64 bit)
Monitoring Agent for Microsoft SQL Server  / 06.30.00.00


Appreciate any help.







   Martin                                         410 Albert Street       
  (Embedded
   (M.A.)                                                             
 image moved to
   Butler                                                                 
      file:
                                                                        
pic00325.gif)

   Senior IT                                      Waterloo, Ontario
   Services                                      N2L 3V3
   Professional
   - Tivoli on
   Demand
   Services

   T7Z                                            Canada

   DM

   Phone:       +1-519-880-2920

   e-mail:      [email protected]
















_______________________________________________
TME10 mailing list
[email protected]
Unsubscribe:[email protected]




-- 
"Sometimes I think that's the only right thing to do:
To dream. to live in the world of dreams.
But it doesn't last forever--wakefulness always comes to take me back..." 
_______________________________________________
TME10 mailing list
[email protected]
Unsubscribe:[email protected]

_______________________________________________
TME10 mailing list
[email protected]
Unsubscribe:[email protected]