Bug + Patch (Intersync 0.9.5)

Hello,

The patch I posted does not solve all problems.
It does not work when the permit is revoked by
the server (after revoking from client B holding
permit) and the server is not yet informed of
KML waiting at client B. When this happens,
the server sends an outdated value of kml offset
(using JOB_SEND_KMLSIZE).

Notes:

1) Using (is_wait_kml()) at the server
does not work because it is not even aware that
it has to wait for something.

2) When client B sends notification to server
for KML, it does so asynchnously and queue the
job instead of running it immediately. This is
primarily why the server is often informed too
late of new kml. But, even with is_job_run()
there would be race conditions.

I have tried several ways to synchronize permit
revocation (which goes from client A ... to server
... to client B ... to server) with kml fetching
(which goes from client B ... to server ... to
client B), but none are easy and full-proof.

Any suggestions ?

PS.: I am wondering why you have not considered
using IBM's distributed lock package. I have found
so many places where (it seems) things could go
wrong ... is it because the stealing idea is not
compatible ?

Best regards,


------------------------------------------------
After debugging for a while (really hard with 6+
processes talking to each other), I finally found
the problem I had identified in December.

In a few words:
  - send_kmlsize was not called from the right place
  - rc handling in IZO_UPC_PERMIT was confusing
    but may not have caused real problems

See bug & comments in patch for more info.
Note that I am not yet familiar with SourceForge cvs,
so I would prefer to leave it to more experience people
for now.

Hope this helps,

Francois Gagnon.


BUG:
-----------------------------------------------
I seem to have found a bug in intersync-0.9.5.
I understand what happens, and it seems that
the code did not account for this specific case.

Scenario:

- 1 server, 2 clients
- Start server, start client A, start client B
- Client A: mkdir a
- Client B: mkdir b
- Client A: Abort, reintegration failed (directory exists)

What happens is:

- When client A does "mkdir a",
  it builds a kml locally that the server fetches.
  Since client A did not fetch anything
  from server, last received on client A = 0

- When client B does "mkdir b",
  server tells client A to fetch kml from server.
  However, client A starts reintegration based
  on last received which is 0.
  It then replays "mkdir a, mkdir b" instead of
  just "mkdir b" because rec->lr_remote_offset = 0

  CDEBUG(D_KML, "remote kmlsize: %Lu, local kmlsize:
  %Lu, already: %d\n", kmlsize, rec->lr_remote_offset, already);


SOLUTION:
----------------


--- ../source/intersync-0.9.5/intersync/upcall.c        2002-09-26
15:23:26.000000000 -0400
+++ intersync/upcall.c  2003-01-27 18:57:50.000000000 -0500
@@ -298,18 +298,31 @@
                         break;  /* this dramatic enough? */
                 }

-                if (fset->fset_server == NULL) {
-                        /* If we're the server, tell the permit holder
what the
-                         * new local KML size is so that it knows to skip
KML
-                         * that was generated as a result of records that
were
-                         * reintegrated while it held the permit. */
-                        rc = is_run_job(is, CONTEXT_JOB_SEND_KMLSIZE,
NULL, rep,
-                                        NULL);
-                }
                 if (rc == 0) {
                         is_wait_for_kml(fset);
                         fset->fset_permit_holder = NULL;
                 }
+
+                if (fset->fset_server == NULL) {
+                       /* This could happen if the server is initiating
permit revocation,
+                        * but in most cases (maybe all cases today ?),
clients will initiate request.
+                        * Note that IZO_UPC_PERMIT is only invoked by the
entity
+                        * asking permit revocation. See also
IZO_UPC_REVOKE_PERMIT below.
+
+                         * If we're the server, tell the permit holder
what the
+                         * new local KML size is so that it knows to skip
KML
+                         * that was generated as a result of records that
were
+                         * reintegrated while it held the permit. */
+                        rc = is_run_job(is, CONTEXT_JOB_SEND_KMLSIZE,
NULL, rep,
+                                        NULL);
+                        if (rc == -ENOTCONN) {
+                                CERROR("Disconnected from old permit
holder, this client will probably have reintegration conflicts.\n");
+                        } else  if (rc) {
+                                /* FIXME */
+                                CERROR("Failed, not sure what to do ?\n");
+                        }
+                }
+
                 upc_reply(pfd, &upc, rc);
                 break;
         }
@@ -360,6 +373,19 @@
                         } else {
                                 fset->fset_permit_holder = NULL;
                         }
+
+                       /* We are the server, tell the permit holder what
the
+                        * new local KML size is so that it knows to skip
KML
+                        * that was generated as a result of records that
were
+                        * reintegrated while it held the permit. */
+                       rc = is_run_job(is, CONTEXT_JOB_SEND_KMLSIZE, NULL,
rep,NULL);
+                        if (rc == -ENOTCONN) {
+                                CERROR("Disconnected from old permit
holder, this client will probably have reintegration conflicts.\n");
+                        } else  if (rc) {
+                                /* FIXME */
+                                CERROR("Failed, not sure what to do ?\n");
+                        }
+
                 }

                 /* Get the peer of the new permit holder. */



Francois Gagnon
Equant, CS&N, Strategic Technology Planning
Heraklion, 1041 route des Dolines, BP347
06906 Sophia Antipolis, Cedex, France
Phone: +33-(0)4-92-96-6325 (7-223-6325)




-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com