Re: SEDA batchController and stage overloaded

All this conversation just put my brain to think :)
 
Hummm ... I just tried to "emulate" an asynchronous layer to perform jdbc access. I does "work" with only one thread now...  The stage is now balanced but I just moved the overload stuff to this "emulation" layer.
 
 
The system responds within 17s when it was expected to respond within 2 s.

So, my emulation layer:
- is composed of a timer thread that executes the task as soon as it is scheduled.
- has only one thread now
- has an internal queue that handles timings contraints concerned to the executions of its tasks
 
So, the second stage was being overloaded because of jdbc blocking access to db.
Now I'm getting this layer overloaded the same way.
 
Hummm... I've been thinking if SEDA had some kind of ACT (aka Asynchronous Completion Token) to process these blocking-tasks ...
Perhaps the Proactor pattern would fit pretty well ... but that would demand Java to have Asynchronous IO capabilities ... :(
 
Oh, well... just thinking...
 
Thanks, and best regards,
Jean Rodrigues
 

Eric Anderson <[email protected]> wrote:
Maybe it would make sense to write a blocking patch to SEDA? I know that might
sound like blasphemy, but there seem to be a number of situations in which it
makes sense. Case in point:

I'm doing research on Internet worms, and we've developed an event-driven
monitor that uses PCAP and reacts to new packets appearing on the network. In
this context, SEDA's normal behavior is just right: We want to do the best job
we can of keeping up with traffic, and drop events if we must. But, some of
the time, we want to use the program in off-line mode, giving it pre-recorded
network traces or simulation output. In that context, the "events" have no
pre-determined arrival rate, but can come at whatever rate we choose to dispatch
them. There's no reason to ever overflow a queue, but right now load
balancing doesn't guarantee that it won't happen, just makes it unlikely. We
could just write a separate, single-threaded, blocking version, but we'd rather
just maintain a single version.

Here's how we manage it now, I believe, which seems to work out OK: It happens
that our first processing stage is relatively computationally intensive. We
set this stage up with a finite queue and a maximum of one thread. We create a
"feeder" thread outside of SEDA which provides the events. It buffers the
event to be enqueued, and performs a lossy enqueue. If the queue is full, the
feeder thread sleeps in an exponential back-off. As long as the subsequent
stages aren't significantly heavier-weight than the first, this crude admission
control seems to work.

Leading you down the garden path,
Eric

Thus spake Quartz ([email protected]):

> Hi,
> 
> Think of it this way: if you are even driven, you must be all the way. And so, your stored proc
> must be asynchronous, yes.
> 
> But as soon as you cannot do async calls, then you must "emulate" async calls.
> 
> In those cases where async emulation cannot be done, you must forget event driven philosophy, and
> be glad that seda offers a good messaging framework as plan B. Stages are a good way to partition
> the architecture in roles. The constraint is to have a reliable transport of messages, hence the
> blocking enqueues.
> 
> I found out that having stages and threads are not usefull for finite mandatory processing, in
> which case you better process in one thread across plugged tasks (simple state machine pattern).
> But if you have optionnal tasks and morphing "data paths" which may have dynamically changing
> load, then sandstorm offer a great way to balance, and even limit processing abuses for tasks of
> lesser importances.
> 
> Good luck.
> 
> 
> --- Jean wrote:
> > Quartz,
> > Thanks for your reply.
> > 
> > I guess I wasn't paying attention to that stage (the one that is getting overloaded).
> > This stage needs to execute a stored procedure (oh, hell, that's the problem).
> > 
> > So, this is a blocking call that really messes up all control over this stage.
> > Now, what should be the best to do:
> > - Implement some kind of emulation layer, in order to turn this jdbc call into some kind of a
> > asynchronous one? (Hummm... still thinkin' how I would do that).
> > - Implement my own Thread Manager in order to ajust thread priorities to avoid this overload?
> > (as you suggested)
> > - Implement some kind of blocking-queue just for this overloaded stage... (hummm... this must
> > overload the first stage... shouldn't it?).
> > 
> > Overview of what is happening:
> > 
> > firstStage (1 thread) --> secondStage (1000 threads) ---> thirdStage (1 thread)
> > 
> > The second stage's queue length increases to 250 even with only on thread on the first stage!!
> > 
> > The other stage get no queue at all (sampled by 1000 ms).
> > 
> > Thanks in advance.
> > Jean Rodrigues
> 
> 
> 
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - 50x more storage than other providers!
> http://promotions.yahoo.com/new_mail
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
> digital self defense, top technical experts, no vendor pitches, 
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> SEDA-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/seda-users

-- 
Eric W. Anderson - [email protected]
University of Oregon Network Security Research Lab
PGP fingerprints:
D3C5 D6FF EDED 9F1F C36D 53A3 74B7 53A6 3C74 5F12
9544 C724 CAF3 DC63 8CAB 5F30 68AE 5C63 B282 2D79


> ATTACHMENT part 2 application/pgp-signature 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com