Re: [jgroups-users] JDBC - Cloning DB

With all due respect, I don't think MERGE2 or MERGE3 is the root cause, 
it is only used to fix the root cause...

On 21/06/18 12:55, Questions/problems related to using JGroups wrote:
> Hi Bela,
> 
> And indeed the "problem" manifest itself only if MERGE2 is used, while it disappears when upgrading to MERGE3.
> 
> Thanks (again)
> P.
> 
> 
>> -----Original Message-----
>> From: Questions/problems related to using JGroups [mailto:javagroups-
>> [email protected]]
>> Sent: 20 June 2018 15:28
>> To: [email protected]
>> Subject: Re: [jgroups-users] JDBC - Cloning DB
>>
>> The config you show below suggest you're using an old version of JGroups
>> (e.g. UNICAST2, MuxRequestCorrelator, TCP's shared transport etc), can
>> you try this with the latest 4.x or 3.6.x release?
>>
>> Comments below
>>
>> On 19/06/18 19:21, Questions/problems related to using JGroups wrote:
>>> Hello,
>>>
>>> The application I am working with makes use of JDBC_PING for discovery
>> and I've recently stumbled upon a scenario which involves the following stack
>> configuration :
>>>
>>> <config xmlns="urn:org:jgroups"
>>>           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>           xsi:schemaLocation="urn:org:jgroups file:schema/JGroups-3.2.xsd">
>>>      <TCP    [..]
>>>      />
>>>
>>>      <JDBC_PING connection_url="jdbc:mysql://localhost:3306/jgroups"
>>>                 connection_username="test"
>>>                 connection_password="test"
>>>                 connection_driver="com.mysql.jdbc.Driver"
>>>      />
>>>      <MERGE2 max_interval="30000" min_interval="10000"/>
>>>      <FD_SOCK/>
>>>      <FD timeout="3000" max_tries="3"/>
>>>      <VERIFY_SUSPECT timeout="1500"/>
>>>      <pbcast.NAKACK2 use_mcast_xmit="false"
>>>                     [...]
>>>      />
>>>      <UNICAST2 stable_interval="5000"
>>>                [..]
>>>      />
>>>      <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000"
>> max_bytes="1m"/>
>>>      <pbcast.GMS print_local_addr="false" join_timeout="3000"
>> view_bundling="true"/>
>>>      <UFC max_credits="200k" min_threshold="0.20"/>
>>>      <MFC max_credits="200k" min_threshold="0.20"/>
>>>      <FRAG2 frag_size="60000"/>
>>>      <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />
>>> </config>
>>>
>>> There two DB, db1 and db2
>>
>>
>>
>> Why two databases? The point of JDBC_PING is to have a common DBMS
>> which
>> all members access...
>>
>>
>>
>>> and two processes p1 and p2. P1 uses db1 and p2 db2, they all are into the
>> same subnet and they both share the same code, for instance
>>> they both create a channel with the same name.
>>
>> Again, are those 2 DBs in sync, so when P1 inserts its data into DB1,
>> will P2 read the same data out of DB2?
>>
>>> P1 starts up and it becomes a single node cluster,  thereafter, for any
>> reason, the whole db1 is exported and imported into
>>> db2 thus causing the JDBC_PING table to be the same as p1 when p2 starts.
>>
>> OK. Any reason for this? What if P1 and P2 start at the same time?
>>
>> Or what happens if P2 updates DB2? Will the update be exported back to
>> DB1? What if P1 updates DB1 at *exactly the same time*? Exporting and
>> (probably) overwriting a DB's state seems very brittle IMO...
>>
>>
>>> The issue I am trying to get at the bottom of is that p2 keeps printing :
>>>
>>> 17:58:00.913 WARN OOB-2,LT1038-21441 org.jgroups.protocols.TCP xxxxxx:
>> no physical address for 0eece642-4813-bb4a-934f-46d886cad55b, dropping
>> message
>>
>>
>> This means that P2 probably wanted to send a message to P1 and didn't
>> find P1's IP address in DB2. DB2 should contain IP addresses for both P1
>> and P2. You should enable tracing on JDBC_PING to see what's going on,
>> or dump the contents of each DB when starting P2.
>>
>>
>>> ---------------- TRACE (received) ----------------------
>>> Event[type=MSG, arg=[dst: <null>, src: xxxxx (4 headers), size=1273 bytes]]
>> (headers=MuxRequestCorrelator: MuxHeader(29318), RequestCorrelator:
>> id=200, type=REQ, id=162, rsp_expected=false, NAKACK2: [MSG, seqno=161],
>> TCP: [channel_name=ControlChannel])
>>> --------------------------------------------------------
>>> ---------------- TRACE (received) ----------------------
>>> Event[type=MSG, arg=[dst: <null>, src: xxxxx (4 headers), size=1273 bytes]]
>> (headers=MuxRequestCorrelator: MuxHeader(29318), RequestCorrelator:
>> id=200, type=REQ, id=162, rsp_expected=false, NAKACK2: [MSG, seqno=161],
>> TCP: [channel_name=ControlChannel])
>>> --------------------------------------------------------
>>
>> Do you need muxing? I assume you use a shared transport (in TCP)? This
>> was deprecated and removed a long time ago, in favor of FORK.
>>
>>
>>> ---------------- TRACE (received) ----------------------
>>> Event[type=IS_MERGE_IN_PROGRESS, arg=null]
>>> --------------------------------------------------------
>>> ---------------- TRACE (received) ----------------------
>>> Event[type=IS_MERGE_IN_PROGRESS, arg=null]
>>> --------------------------------------------------------
>>>
>>> The two tables hold, no surprise, a very different state of the cluster and p2
>> is indeed receiving a message from somebody which is not part of its table.
>>
>> Why do the tables hold different state? They should hold exactly the
>> same state, or else you're export/import function didn't function correctly.
>>
>> I suggest try this with a single DB, and both P1 and P2 point to it.
>>
>>> I am not quite sure my reading of the situation is correct, it seems that p2
>> connects to p1 to notify its existence and thereafter p1, which is the
>> coordinator
>>> tries to perform a merge.
>>
>> Right: P2 starts up and becomes singleton because it cannot send a JOIN
>> request message to P1. So we have 2 clusters {P1} and {P2} and then they
>> try to merge. But this will probably fails, too, because P1 and P2
>> cannot communicate...
>>
>>> Do you have any suggestion on how I could further debug what happened ?
>>
>> 1: use a more recent version of JGroups
>> 2: use a single DB
>>
>>> I would like to trace down the JOINS() being sent to the p1 to make sure the
>> above statement is correct, and then I would like to work
>>> out if there is any problem with the stack's configuration or if the problem is
>> un-avoidable as the root of the rub is the DB being copied across.
>>>
>>> I am used to debug the view by setting a custom/dummy receiver which
>> would merely print the view without doing anything, I am a bit wary of doing
>> that in my application
>>> and I am wondering if it is possible to programmatically enable the logging
>> across all the stack so to have a whole view or what's going on without
>> impacting the stack's
>>> "functional" configuration.
>>
>> You could use probe.sh, e.g. probe.sh jmx=JDBC_PING.level="trace"
>>
>>> Thanks,
>>> P.
>>>
>>>
>>>
>>> Pietro Paolini
>>> Consultant
>>>
>>> Alfa
>>> ________________________________
>>> e: [email protected] | w:
>> alfasystems.com<https://www.alfasystems.com>
>>> t: +44 (0) 20 7920-2643 | Moor Place, 1 Fore Street Avenue, London, EC2Y
>> 9DT, GB
>>> ________________________________
>>>
>>> The contents of this communication are not intended to be binding or
>> constitute any form of offer or acceptance or give rise to any legal obligations
>> on behalf of the sender or Alfa. The views or opinions expressed represent
>> those of the author and not necessarily those of Alfa. This email and any
>> attachments are strictly confidential and are intended solely for use by the
>> individual or entity to whom it is addressed. If you are not the addressee (or
>> responsible for delivery of the message to the addressee) you may not copy,
>> forward, disclose or use any part of the message or its attachments. At
>> present the integrity of email across the internet cannot be guaranteed and
>> messages sent via this medium are potentially at risk. All liability is excluded
>> to the extent permitted by law for any claims arising as a result of the use of
>> this medium to transmit information by or to Alfa or its affiliates.
>>>
>>> Alfa Financial Software Ltd
>>> Reg. in England No: 0248 2325
>>>
>>> ------------------------------------------------------------------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> javagroups-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>>>
>>
>> --
>> Bela Ban | http://www.jgroups.org
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> javagroups-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> javagroups-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot