RE: [ITM] Out of sync HUB TEMS?
John Alvord <jalvord-r/[email protected]> Thu, 31 Oct 2013 13:35:37 -0700
| Newsgroups | gmane.comp.sysutils.tivoli.general |
|---|---|
| Message-ID | <OFF1402332.50DEF91F-ON88257C15.0070F9FD-88257C15.00712012__49723.2380592779$1383253798$gmane$org@us.ibm.com> |
This is a multipart message in MIME format. --===============1774178307== Content-Type: multipart/alternative; boundary="=_alternative 0071201188257C15_=" This is a multipart message in MIME format. --=_alternative 0071201188257C15_= Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable So... you did a tacmd cleanms -a... then stopped the hub TEMS, waited for=20 the backup hub tems to become the primary, did the tacmd cleanms -a=20 again,,, then stopped the TEMS and brought up the primary hub TEMS... and=20 are still seeing unexpected offline agents? Regards,=20 =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F John Alvord Advisory Engineer and ITM Historian IBM Tivoli Monitoring 6 Platform Customer Support=20 Office: 1-720-396-2788=20 Mobile: none=20 jalvord-r/[email protected] Cloud & Smarter Infrastructure IBM Support Portal Open a New Service Request Update your PMR via Email Escalate your PMR My Blog: Wonderful World of Situations Secure upload to ECUREP From: "Liu, Jun Feng" <[email protected]> To: "Discussion list for Tivoli product and Tivoli Ready products."=20 <[email protected]>,=20 Date: 10/31/2013 01:24 PM Subject: RE: [TME10] [ITM] Out of sync HUB TEMS? Sent by: [email protected] We have the same problem, even I clear offline, they will be back when the = failover happen again. =20 =20 =20 Thanks =20 Jun Feng Liu =20 =20 =20 From: [email protected] [ mailto:[email protected]] On Behalf Of John Alvord Sent: Thursday, October 31, 2013 12:12 PM To: Discussion list for Tivoli product and Tivoli Ready products. Subject: Re: [TME10] [ITM] Out of sync HUB TEMS? =20 The internal equivalent of the INODESTS [node status table - TNODESAV] is=20 not synchronized between the primary hub TEMS and the backup hub TEMS. You = probably need to do a=20 ./tacmd cleanms -a =20 to remove all the offline nodes. Regards,=20 =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=20 =20 John Alvord Advisory Engineer and ITM Historian IBM Tivoli Monitoring 6 Platform Customer Support=20 Office: 1-720-396-2788=20 Mobile: none=20 jalvord-r/[email protected] Cloud & Smarter Infrastructure=20 IBM Support Portal Open a New Service Request Update your PMR via Email Escalate your PMR My Blog: Wonderful World of Situations Secure upload to ECUREP=20 From: "Battista, Joe" <[email protected]>=20 To: "[email protected]" <[email protected]>,=20 Date: 10/31/2013 07:35 AM=20 Subject: [TME10] [ITM] Out of sync HUB TEMS?=20 Sent by: [email protected]=20 We recently upgraded to ITM 6.2.3 FP03. Our environment consists of a pair = of HUB TEMS, (primary and hot-standby) and four Remote TEMS for our=20 agents. We shut down our primary HUB which forced the failover to the=20 hot-standby. Within a period of time we started receiving agent offline=20 alerts which we sort of expected since our Remote TEMS were failing over=20 to the hot-standby HUB. It?s been a while since we left our hot-standby=20 HUB run as the primary.=20 =20 After closer review, we noticed some of the offline agent alerts were from = servers that had been removed from our environment some time ago.=20 I see references to the servers that no longer exist in our environment in = the QA1*.DB files on the hot-standby HUB but not on the primary HUB,=20 specifically tables QA1CSTSC.DB, QA1CSTSH.DB and QA1DNSAV.DB.=20 =20 I assume this means in some way our HUB TEMS are out-of-sync? Is there an=20 easy way to get our hot-standby cleaned up and in-sync with the primary=20 HUB?=20 =20 =20 Thanks,=20 Joe =20 =20 =20 ***************************************************************************= ****=20 Unauthorized interception of this communication could be a violation of=20 Federal and State Law. This communication and any files transmitted with=20 it are confidential and may contain protected health information. This=20 communication is solely for the use of the person or entity to whom it was = addressed. If you are not the intended recipient, any use, distribution,=20 printing or acting in reliance on the contents of this message is strictly = prohibited. If you have received this message in error, please notify the=20 sender and destroy any and all copies. Thank you..=20 ***************************************************************************= ****=20 =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F TME10 mailing list [email protected] Unsubscribe:[email protected] NOTICE: Confidential message which may be privileged. Unauthorized=20 use/disclosure prohibited. If received in error, please go to=20 www.td.com/legal for instructions. AVIS : Message confidentiel dont le contenu peut =EAtre privil=E9gi=E9.=20 Utilisation/divulgation interdites sans permission. Si re=E7u par erreur,=20 pri=E8re d'aller au www.td.com/francais/avis=5Fjuridique pour des=20 instructions.=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F TME10 mailing list [email protected] Unsubscribe:[email protected] --=_alternative 0071201188257C15_= Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable <font size=3D2 face=3D"sans-serif">So... you did a tacmd cleanms -a... then stopped the hub TEMS, waited for the backup hub tems to become the primary, did the tacmd cleanms -a again,,, then stopped the TEMS and brought up the primary hub TEMS... and are still seeing unexpected offline agents?<br> </font><font size=3D3><br> Regards, <br> =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F</font> <table width=3D433 style=3D"border-collapse:collapse;"> <tr height=3D8> <td width=3D433 style=3D"border-style:solid;border-color:#000000;border-wid= th:0px 0px 0px 0px;padding:0px 0px;"><font size=3D1><br> </font> <table width=3D433 style=3D"border-collapse:collapse;"> <tr height=3D8> <td width=3D221 style=3D"border-style:solid;border-color:#000000;border-wid= th:0px 0px 0px 0px;padding:0px 0px;"><font size=3D1 face=3D"Arial"><b>John Alvord</b><br> Advisory Engineer and ITM Historian<br> IBM Tivoli Monitoring 6 Platform<b><br> Customer Support </b><br> Office: 1-720-396-2788 <br> Mobile: none </font><font size=3D1 color=3Dblue face=3D"Arial"><u><br> </u></font><a href=3Dmailto:jalvord-r/[email protected]><font size=3D1 color=3Dblue = face=3D"Arial"><u>jalvord-r/[email protected]</u></font></a><font size=3D3><br> </font><font size=3D1 color=3Dblue face=3D"Arial"><b><u><br> </u></b></font><a href=3Dhttp://www.ibm.com/software/tivoli/><font size=3D1= color=3Dblue face=3D"Arial"><b><u>Cloud & Smarter Infrastructure</u></b></font></a> <td width=3D212 style=3D"border-style:solid;border-color:#000000;border-wid= th:0px 0px 0px 0px;padding:0px 0px;"><a href=3Dhttps://www.ibm.com/support/= ><font size=3D1 color=3Dblue face=3D"Arial"><u>IBM Support Portal</u></font></a><font size=3D1 color=3Dblue face=3D"Arial"><u>= <br> </u></font><a href=3Dhttp://www.ibm.com/support/entry/portal/Open=5Fservice= =5Frequest/><font size=3D1 color=3Dblue face=3D"Arial"><u>Open a New Service Request</u></font></a><font size=3D1 color=3Dblue face=3D"Ari= al"><u><br> </u></font><a href=3D"https://www-304.ibm.com/jct05001c/de/support/ecurep/s= end=5Femail.html#naming"><font size=3D1 color=3Dblue face=3D"Arial"><u>Upda= te your PMR via Email</u></font></a><font size=3D1 color=3Dblue face=3D"Arial"= ><u><br> </u></font><a href=3D"http://www-304.ibm.com/support/customercare/sas/f/han= dbook/getsupport.html#9"><font size=3D1 color=3Dblue face=3D"Arial"><u>Esca= late your PMR</u></font></a><font size=3D1 color=3Dblue face=3D"Arial"><u><br> </u></font><a href=3Dhttps://www.ibm.com/developerworks/mydeveloperworks/bl= ogs/jalvord><font size=3D1 color=3Dblue face=3D"Arial"><u>My Blog: Wonderful World of Situations</u></font></a><font size=3D1 color=3Dbl= ue face=3D"Arial"><u><br> </u></font><a href=3Dhttps://www.ecurep.ibm.com/app/upload><font size=3D1 c= olor=3Dblue face=3D"Arial"><u>Secure upload to ECUREP</u></font></a> <p><a href=3Dhttps://www.facebook.com/IBMTivolisupport></a><a href=3Dhttps:= //twitter.com/TivoliSupport></a><a href=3Dhttps://www.ibm.com/developerwork= s/servicemanagement/index.html></a></table> <br> <tr height=3D8> <td width=3D433 style=3D"border-style:solid;border-color:#000000;border-wid= th:0px 0px 0px 0px;padding:0px 0px;"><a href=3Dhttp://www.ibm.com/software/= tivoli// target=3D=5Fnew></a></table> <br> <br> <br> <br> <br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">From: = </font><font size=3D1 face=3D"sans-serif">"Liu, Jun Feng" <[email protected]></font> <br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">To: &n= bsp; </font><font size=3D1 face=3D"sans-serif">"Discussion list for Tivoli product and Tivoli Ready products." <[email protected]= com>, </font> <br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">Date: = </font><font size=3D1 face=3D"sans-serif">10/31/2013 01:24 PM</font> <br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">Subject: &nbs= p; </font><font size=3D1 face=3D"sans-serif">RE: [TME10] [ITM] Out of sync HUB TEMS?</font> <br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">Sent by: &nbs= p; </font><font size=3D1 face=3D"sans-serif">tme10-bounces@lists.= ca.ibm.com</font> <br> <hr noshade> <br> <br> <br><font size=3D2 color=3D#004080 face=3D"Calibri">We have the same proble= m, even I clear offline, they will be back when the failover happen again.</fo= nt> <br><font size=3D2 color=3D#004080 face=3D"Calibri"> </font> <br><font size=3D2 color=3D#004080 face=3D"Calibri"> </font> <br><font size=3D2 color=3D#004080 face=3D"Calibri"> </font> <br><font size=3D2 color=3D#2200a1 face=3D"Arial">Thanks</font> <br><font size=3D2 color=3D#2200a1 face=3D"Calibri"> </font> <br><font size=3D2 color=3D#2200a1 face=3D"Arial">Jun Feng Liu</font> <br><font size=3D2 color=3Dblue face=3D"Arial"> </font> <br><font size=3D2 color=3D#004080 face=3D"Calibri"> </font> <br><font size=3D2 color=3D#004080 face=3D"Calibri"> </font> <br><font size=3D2 face=3D"Tahoma"><b>From:</b> [email protected]= com [</font><a href=3D"mailto:[email protected]"><font size=3D2 fa= ce=3D"Tahoma">mailto:[email protected]</font></a><font size=3D= 2 face=3D"Tahoma">] <b>On Behalf Of </b>John Alvord<b><br> Sent:</b> Thursday, October 31, 2013 12:12 PM<b><br> To:</b> Discussion list for Tivoli product and Tivoli Ready products.<b><br> Subject:</b> Re: [TME10] [ITM] Out of sync HUB TEMS?</font> <br><font size=3D3 face=3D"Times New Roman"> </font> <br><font size=3D2 face=3D"Arial">The internal equivalent of the INODESTS [= node status table - TNODESAV] is not synchronized between the primary hub TEMS and the backup hub TEMS. You probably need to do a </font><font size=3D3 fa= ce=3D"Times New Roman"><br> </font><font size=3D2 face=3D"Arial"><br> ./tacmd cleanms -a </font><font size=3D3 face=3D"Times New Roma= n"> <br> </font><font size=3D2 face=3D"Arial"><br> to remove all the offline nodes.</font><font size=3D3 face=3D"Times New Rom= an"><br> <br> Regards, <br> =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F </font> <p> <table width=3D434 style=3D"border-collapse:collapse;"> <tr height=3D8> <td width=3D432 style=3D"border-style:solid;border-color:#000000;border-wid= th:1px 1px 3px 1px;padding:0px 0px;"><font size=3D3 face=3D"Times New Roman= "> </font> <p> <br> <table width=3D434 style=3D"border-collapse:collapse;"> <tr height=3D8> <td width=3D220 style=3D"border-style:solid;border-color:#000000;border-wid= th:1px 3px 1px 1px;padding:0px 0px;"><font size=3D1 face=3D"Arial"><b>John Alvord</b><br> Advisory Engineer and ITM Historian<br> IBM Tivoli Monitoring 6 Platform<b><br> Customer Support </b><br> Office: 1-720-396-2788 <br> Mobile: none </font><font size=3D3 color=3Dblue face=3D"Times New Roman"><u= ><br> </u></font><a href=3Dmailto:jalvord-r/[email protected]><font size=3D1 color=3Dblue = face=3D"Arial"><u>jalvord-r/[email protected]</u></font></a><font size=3D1 color=3Db= lue face=3D"Arial"><b><u><br> </u></b></font><font size=3D3 color=3Dblue face=3D"Times New Roman"><u><br> </u></font><a href=3Dhttp://www.ibm.com/software/tivoli/><font size=3D1 col= or=3Dblue face=3D"Arial"><b><u>Cloud & Smarter Infrastructure</u></b></font></a><font size=3D3 face=3D"Times= New Roman"> </font> <td width=3D209 style=3D"border-style:solid;border-color:#000000;border-wid= th:1px 1px 1px 3px;padding:0px 0px;"><a href=3Dhttps://www.ibm.com/support/= ><font size=3D1 color=3Dblue face=3D"Arial"><u>IBM Support Portal</u></font></a><font size=3D3 color=3Dblue face=3D"Times New = Roman"><u><br> </u></font><a href=3Dhttp://www.ibm.com/support/entry/portal/Open=5Fservice= =5Frequest/><font size=3D1 color=3Dblue face=3D"Arial"><u>Open a New Service Request</u></font></a><font size=3D3 color=3Dblue face=3D"Tim= es New Roman"><u><br> </u></font><a href=3D"https://www-304.ibm.com/jct05001c/de/support/ecurep/s= end=5Femail.html#naming"><font size=3D1 color=3Dblue face=3D"Arial"><u>Upda= te your PMR via Email</u></font></a><font size=3D3 color=3Dblue face=3D"Times = New Roman"><u><br> </u></font><a href=3D"http://www-304.ibm.com/support/customercare/sas/f/han= dbook/getsupport.html#9"><font size=3D1 color=3Dblue face=3D"Arial"><u>Esca= late your PMR</u></font></a><font size=3D3 color=3Dblue face=3D"Times New Roman"= ><u><br> </u></font><a href=3Dhttps://www.ibm.com/developerworks/mydeveloperworks/bl= ogs/jalvord><font size=3D1 color=3Dblue face=3D"Arial"><u>My Blog: Wonderful World of Situations</u></font></a><font size=3D3 color=3Dbl= ue face=3D"Times New Roman"><u><br> </u></font><a href=3Dhttps://www.ecurep.ibm.com/app/upload><font size=3D1 c= olor=3Dblue face=3D"Arial"><u>Secure upload to ECUREP</u></font></a><font size=3D3 face=3D"Times New Roman"> </f= ont></table> <br> <tr height=3D8> <td width=3D432 style=3D"border-style:solid;border-color:#000000;border-wid= th:3px 1px 1px 1px;padding:0px 0px;"></table> <br><font size=3D3 face=3D"Times New Roman"><br> <br> <br> <br> </font><font size=3D1 color=3D#5f5f5f face=3D"Arial"><br> From: </font><font size=3D1 face=3D"Arial">"= ;Battista, Joe" <</font><a href=3Dmailto:[email protected]><font size= =3D1 color=3Dblue face=3D"Arial"><u>[email protected]</u></font></= a><font size=3D1 face=3D"Arial">></font><font size=3D3 face=3D"Times New= Roman"> </font><font size=3D1 color=3D#5f5f5f face=3D"Arial"><br> To: </font><font size=3D1 face=3D"Arial">"<= /font><a href=3Dmailto:[email protected]><font size=3D1 color=3Dblue f= ace=3D"Arial"><u>[email protected]</u></font></a><font size=3D1 face= =3D"Arial">" <</font><a href=3Dmailto:[email protected]><font size=3D1 color=3Db= lue face=3D"Arial"><u>[email protected]</u></font></a><font size=3D1 f= ace=3D"Arial">>, </font><font size=3D1 color=3D#5f5f5f face=3D"Arial"><br> Date: </font><font size=3D1 face=3D"Arial">10/31= /2013 07:35 AM</font><font size=3D3 face=3D"Times New Roman"> </font><font size= =3D1 color=3D#5f5f5f face=3D"Arial"><br> Subject: </font><font size=3D1 face=3D"Arial">[T= ME10] [ITM] Out of sync HUB TEMS?</font><font size=3D3 face=3D"Times New Roman"> </font><font size=3D1 color=3D#5f5f5f face=3D"Arial"><br> Sent by: </font><a href=3D"mailto:tme10-bounces@= lists.ca.ibm.com"><font size=3D1 color=3Dblue face=3D"Arial"><u>tme10-bounc= [email protected]</u></font></a><font size=3D3 face=3D"Times New Roman"> </font> <div align=3Dcenter> <hr noshade></div> <br><font size=3D3 face=3D"Times New Roman"><br> <br> </font><font size=3D2 color=3D#004080 face=3D"Calibri"><br> We recently upgraded to ITM 6.2.3 FP03. Our environment consists of a pair of HUB TEMS, (primary and hot-standby) and four Remote TEMS for our agents. We shut down our primary HUB which forced the failover to the hot-standby. Within a period of time we started receiving agent offline alerts which we sort of expected since our Remote TEMS were failing over to the hot-standby HUB. It’s been a while since we left our hot-standby HUB run as the primary. <br> </font><font size=3D3 face=3D"Times New Roman"> </font><font size=3D2= color=3D#004080 face=3D"Calibri"><br> After closer review, we noticed some of the offline agent alerts were from servers that had been removed from our environment some time ago. <br> I see references to the servers that no longer exist in our environment in the QA1*.DB files on the hot-standby HUB but not on the primary HUB, specifically tables QA1CSTSC.DB, QA1CSTSH.DB and QA1DNSAV.DB. <br> </font><font size=3D3 face=3D"Times New Roman"> </font><font size=3D2= color=3D#004080 face=3D"Calibri"><br> I assume this means in some way our HUB TEMS are out-of-sync? Is there an easy way to get our hot-standby cleaned up and in-sync with the primary HUB?</font><font size=3D3 face=3D"Times New Roman"> </font><font size=3D2 c= olor=3D#004080 face=3D"Calibri"><br> </font><font size=3D3 face=3D"Times New Roman"> </font><font size=3D2= color=3D#004080 face=3D"Calibri"><br> </font><font size=3D3 face=3D"Times New Roman"> </font><font size=3D2= color=3D#004080 face=3D"Calibri"><br> Thanks, <br> Joe </font><font size=3D3 face=3D"Times New Roman"> </font><font size= =3D2 color=3D#004080 face=3D"Calibri"><br> </font><font size=3D3 face=3D"Times New Roman"> </font><font size=3D2= face=3D"Calibri"><br> </font><font size=3D3 face=3D"Times New Roman"> </font> <p><font size=3D3 face=3D"Times New Roman"><br> ***************************************************************************= **** <br> Unauthorized interception of this communication could be a violation of Federal and State Law. This communication and any files transmitted with it are confidential and may contain protected health information. This communication is solely for the use of the person or entity to whom it was addressed. If you are not the intended recipient, any use, distribution, printing or acting in reliance on the contents of this message is strictly prohibited. If you have received this message in error, please notify the sender and destroy any and all copies. Thank you.. <br> ***************************************************************************= **** </font><font size=3D2 face=3D"Courier New">=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F<br> TME10 mailing list</font><font size=3D2 color=3Dblue face=3D"Courier New"><= u><br> </u></font><a href=3Dmailto:[email protected]><font size=3D2 color=3Db= lue face=3D"Courier New"><u>[email protected]</u></font></a><font size= =3D2 face=3D"Courier New"><br> Unsubscribe:[email protected]</font> <p><font size=3D1 face=3D"Arial"><br> NOTICE: Confidential message which may be privileged. Unauthorized use/disc= losure prohibited. If received in error, please go to </font><a href=3Dwww.td.com/= legal><font size=3D1 face=3D"Arial">www.td.com/legal</font></a><font size= =3D1 face=3D"Arial"> for instructions.<br> AVIS : Message confidentiel dont le contenu peut =EAtre privil=E9gi=E9. Uti= lisation/divulgation interdites sans permission. Si re=E7u par erreur, pri=E8re d'aller au </fon= t><a href=3Dwww.td.com/francais/avis=5Fjuridique><font size=3D1 face=3D"Ari= al">www.td.com/francais/avis=5Fjuridique</font></a><font size=3D1 face=3D"A= rial"> pour des instructions.</font><tt><font size=3D2>=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F<br> TME10 mailing list<br> [email protected]<br> Unsubscribe:[email protected]<br> </font></tt> <p> --=_alternative 0071201188257C15_=-- --===============1774178307== Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline _______________________________________________ TME10 mailing list [email protected] Unsubscribe:[email protected] --===============1774178307==--