Re: synchronizing multiple services rotation

(* Reply to dev null'd *)

Joan Picanyol i Puig wrote:

> [FWIW I'm using runit, not daemontools]

I've just noticed with the mention of sv and svlogd.c (and prior to
reading this bit).  I don't know anything about runit so my comments
are likely to be way off.

> All "producer"s share a common processor script that reads it's
> "consumer"s from the environment (set up in each "producer"'s log
> configuration), signals them ('/command/sv once /service/consumer') and
> feeds them 'current' through stdin upon rotation. "consumer"s use $PWD
> to find out what they are being fed through stdin and go fetch the rest
> of the data they need at the other's (not being currently rotated)
> "producer"s log dir.

I'm not sure that it's relevant, but I'm confused at how you are tying
the producer's multilog processor to the consumer's stdin when consumer
is also a service?  Named pipe?

> My reading of svlogd.c (and multilog.c for that matter) tells me that
> rotation (i.e.: 'current' renaming) takes place before 'processor''s
> invocation. Thus invoking setlock within the processor does not close
> the race.

Hmmm.  My understanding is this - under daemontools, prior to the
processor script being run the current file is renamed to previous.
If your consumer script has obtained a lock that other producers
and consumers must abide by, then if any current file isn't named
current, it must be named previous.  If an open(2) of "current"
fails with ENOENT, then an open(2) of "previous" must succeed
provided all the other multilog processors are still holding
waiting for the lock (once multilog completes, the file should be
renamed to its tai64n format).  If the file is renamed after the
open() (from current to previous), then that's not an issue - you
already have the file open (of course, substitute open() for
whatever language you're using).

I'm unsure if this is the race you're thinking of.

> I hope the issue is clearer now. My only idea so far is to not feed
> current through stdin but rather make "consumer"s scan all "producer"s
> log directory for data.

I was presuming your were doing the latter - in fact, I'm not sure
how you would be doing the former (even after your explanation above)
without scanning for the other producer's files and opening them
yourself anyway?

Bear in mind that this lock acquiring mechanism will block the
producers if their multilog rolls and the intervening pipe fills
up.  This may be an issue if your consumer process is a long
running one.


-- 
Regards,
  Daryl Tester

"Scheme is an exotic sports car. Fast. Manual transmission. No radio.
 Common Lisp is Howl's Moving Castle."
  -- Steve Yegge, comparing Lisp families to cars.