Re: SEDA batchController and stage overloaded

The only solution that deserve the title on being one, is to create/find/use a non-jdbc tcp/ip
driver for you DB, that would use async socket channels in jdk 1.4.
That is a lot of effort, I guess, no matter what db it is. Searched but never found any.

But as your emulation goes, you post a query to a query stage, giving it the return address stage
name where a result set would be posted.

Every query this stage will accept will lead to 1 more thread in background to block on the query.
If you got 30 requests/s and the query takes 2 seconds, you must prepare for 60 threads to block.
Your stage would need 60 threads. Note: your size threshold must be 1, so a single query can cause
 a thread to be added if none are available, up to a max of 60. I also suggest you set the min
threads high enough, maybe min=max=init=60. Otherwise, set controller update delay short enought
(100ms?), because that delay is directly impacting on the response time if a new thread must be
created. Also, hack sandstorm to raise the controller prio above regular, for good response, but
be sure to create regular thread with explicit prio set, because by default thread inherit their
creator priority.

If using mysql, and don't mind using 4.1 alpha, try the 3.1 connector/j, and use prepared
statements (you must add a property props.setProperty("useServerPrepStmts", "true"); )
I have seen 10 to 1 performance improvements.


--- Jean  <[email protected]> wrote:
> All this conversation just put my brain to think :)
>  
> Hummm ... I just tried to "emulate" an asynchronous layer to perform jdbc access. I does "work"
> with only one thread now...  The stage is now balanced but I just moved the overload stuff to
> this "emulation" layer.
>  
>  
> The system responds within 17s when it was expected to respond within 2 s.
> 
> So, my emulation layer:
> - is composed of a timer thread that executes the task as soon as it is scheduled.
> - has only one thread now
> - has an internal queue that handles timings contraints concerned to the executions of its tasks
>  
> So, the second stage was being overloaded because of jdbc blocking access to db.
> Now I'm getting this layer overloaded the same way.
>  
> Hummm... I've been thinking if SEDA had some kind of ACT (aka Asynchronous Completion Token) to
> process these blocking-tasks ...
> Perhaps the Proactor pattern would fit pretty well ... but that would demand Java to have
> Asynchronous IO capabilities ... :(



		
__________________________________
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com