Re: parallel processes and thread.sleep

Hello He Wei,

There you can increase the number of database connections in Shark configuration file. The actual problem is at the database side - for example, Oracle is configured by default with 300 connections. If we try to run sumultaneously 60 of our processes we will end with (probably) more than 500 database connections. Naturally, Oracle will refuse more than 300 connections and Shark will fail many operations on processes. There is always an option to increase the number of allowed connections, but what number of connections you will need to allow? What about the licenses for so many connections (depending on the license policy, of course)? And do you consider that having 500 database connections for only 60 (parallel) process instances is normal?

About the modified JavaClassToolAgent:
If you look at the original code, you will find that the first parameter is artificialy removed (it is commented-out in the code I sent to you) - these are some extended atributes (as I remember). This is the parameter we "replace" - actually, we just use its place in the parameters structure. No parameter values sent by the workflow application are replaced.

On the "other" side (the one of the tool) you have to know that the first parameter is a hash-map (as you "know" all other parameter types), and to threat it accordingly. Here you are a snippet:

  public static void execute(AppParameter toolAgentParameters, AppParameter userId, 
                             AppParameter documentUrl) {
    ...
    HashMap taParameters = (HashMap) toolAgentParameters.the_value;
    String processKey = (String) taParameters.get(JavaClassToolAgent.TA_PARAM_PROCESS_INSTANCE_ID);
    ...
  }

Here the "toolAgentParameters" is the "hidden" parameter. As you see, its name has nothing to do with the actual ("TAParams") or formal ("none") parameter name on the ToolAgent side.
"userId" and "documentUrl" are parameters, defined in the XPDL (again, the formal and actual parameter names has nothing in common):
        ...
        <Application Id="myToolAgent">
            <Description>Invokes the Java tool</Description>
            <FormalParameters>
                <FormalParameter Id="currentUserId" Index="1" Mode="IN">
                    <DataType>
                        <BasicType Type="INTEGER"/>
                    </DataType>
                </FormalParameter>
                <FormalParameter Id="result" Index="2" Mode="IN">
                    <DataType>
                        <BasicType Type="STRING"/>
                    </DataType>
                    <Description>Document URL</Description>
                </FormalParameter>
            </FormalParameters>
            <ExtendedAttributes>
                <ExtendedAttribute Name="ToolAgentClass" Value="ourPackage.JavaClassToolAgent"/>
                <ExtendedAttribute Name="AppName" Value="ourPackage.JavaTool"/>
            </ExtendedAttributes>
        </Application>
        ...

As you see, the workflow aplication is defined with two parameters, and the execute() method gets three parameters, the first of which is "hidden" - no one is aware of (knows about) it, except the ToolAgent and the tool itself.

Note that you will need to implement changes simultaneusly in both ToolAgent and tool(s), otherwise you will start to receive exceptions "... cannot find method execute(<corresponding number of parameters here>) ..."

Best Regards,
Monique

He Wei wrote:
> Hello Monique,
> 
> Thanks for your suggestions.
> I am using Shark without an application server and currently I didn't
> implement the deadline in the activities of my process. But I will have a
> lot of process instance running simultaneously, say more than 60. Could we
> increase the number database connection for Shark?
> Regarding the JavaClassAgentTool.java file you sent me, I have a question to
> consult you on how to retrieve the map variables. The first item in the tool
> parameters is replaced by the map variables holding the collected data. But
> what happened to the original first item having been replaced? How to
> retrieve the map variable parameter, by "TAParams"? any sample codes please?
> Thank you very much!
> 
> //VK: Replace the first item in the tool parameters with the data collected
> in this method
>          //
>          //The most important values in this constructor are:
>          // - mode ("IN")
>          // - parameter value
>          // - parameter type (class)
>          //The actual name and formal name are insignificant.
>          aps[0] = new AppParameter("TAParams", "none", "IN",
> toolAgentParams, HashMap.class);
>          m.invoke(null,aps);
> 
> Best Regards,
> He Wei
> 
> -----Original Message-----
> From: Monique Maker [mailto:[email protected]] 
> Sent: Friday, August 19, 2005 4:51 PM
> To: [email protected]
> Subject: Re: [shark] parallel processes and thread.sleep
> 
> Hello He Wei,
> 
> If your application is running in an application server, I think that the
> Scheduler ToolAgent is not the best choice.
> 
> I would like to warn you that, with or without an application server, you
> will need to limit the number of simultaneously running threads.
> 
> Note that Shark is not good suited to run in multi-threading environment -
> it aborts any batch executions (such as deadlines recalculation) on process
> lock timout (or any other error). As you say, your activity (probably
> "moving") is running significantly longer than the process lock timeout. Our
> experience is that if some activity runs for more than the lock timout
> (which is about one second by default), the deadlines recalculation (this is
> where we are using multi-threading in Shark) fails on the first process lock
> timeout and the mutli-threading becomes unusable. To be more specific,
> imagine a process that executes some activity, then waits a pre-defined
> period of time (a deadline condition) to continue and the next activity runs
> a long time. Now imagine several instances of this process. We call the
> deadlines recalculation on timer. On each timer tick Shark is contacted in a
> new thread. When one instance of the process proceeds after the deadline and
> the next tick of the
>  timer tries to proceed another process, the deadline recalculation fails
> because the first process instance is still locked. Therefore the next
> process instances cannot be proceed before the long-running activity of the
> first instance fiishes nad allows Shark to unlock the process.
> 
> Also note that Shark uses between five and ten database connections (for our
> relatively simple processes) despite of its own and application server's
> database connection pools. Also Shark does not release database connections
> on (some) exceptions. This means that you might need to restart periodically
> your application to release the database connections. These are two of the
> reasons you will need to limit the number of simultaneous Shark threads.
> 
> Best Regards,
> Monique
> 
> He Wei wrote:
> 
>>Hello Monique,
>>
>>Thank you for your code and suggestion.
>>My process is to handle several items simultaneously. Each of the items
> 
> will
> 
>>go through the same process, such as: unloading -> moving -> loading. If
> 
> we
> 
>>have, say, 10 items coming, 10 process instances at the same time will be
>>initiated to handle them. That is the reason why I need to run several
>>processes in parallel. 
>>Each process will take 30 seconds to complete. When initiating them in
>>different threads, the process 2 will wait 30s until process 1 completes.
> 
> Is
> 
>>the 30s considered too "fast" to shark?
>>I am testing on using Scheduler tool-agent in the process. It looks
>>promising. 
>>Thank you.
>>Best Regards,
>>He Wei
>>
>>-----Original Message-----
>>From: Monique Maker [mailto:[email protected]] 
>>Sent: Wednesday, August 17, 2005 6:01 PM
>>To: [email protected]
>>Subject: Re: [shark] parallel processes and thread.sleep
>>
>>Hello He Wei,
>>
>>Probably your processes are "too fast" (as they should be according to
> 
> Shark
> 
>>documentation) and each process manages to run in a single thread
> 
> "switch",
> 
>>i.e. before the JVM manages to give execution time to another thread.
>>
>>Please explain why do you need to run several processes in parralel.
>>
>>Please note that (almost) any solution will be similar to Java execution
> 
> of
> 
>>one and the same code in non-synchronized threads - you never know what
> 
> part
> 
>>of the code is executed in each thread.
>>
>>Best Regards,
>>Monique
>>
>>He Wei wrote:
>>
>>
>>>Hello Vlada,
>>>
>>>I have tested according to what you suggested to start a new thread for
>>
>>each
>>
>>
>>>process instance. However the results show they are still running in
>>>sequence based on the sequence to run them, but not in parallel. Could you
>>>please kindly advice further? Thank you very much.
>>>(pl. my process contains only automatic activities.)
>>>Best Regards,
>>>He Wei
>>>
>>>-----Original Message-----
>>>From: Vladimir Puskas [mailto:[email protected]] 
>>>Sent: Wednesday, August 17, 2005 3:54 PM
>>>To: [email protected]
>>>Subject: Re: [shark] parallel processes and thread.sleep
>>>
>>>Hello He Wei
>>>
>>>you wrote:
>>>
>>>
>>>
>>>>Dear Vlada,
>>>>
>>>>Regarding to the question in the email below, could you please kindly
>>>>explain more how to "let your application start a new Thread for those 5
>>>>instances" to make it *really* parallel?
>>>
>>>
>>>Since Shark does not start a new thread, your application must, for 
>>>scenario you described.
>>>
>>>      //final SharkConnection sc;
>>>      //final Map workflowRelevantData;
>>>      for (int i = 0; i < 5; i++) {
>>>         new Thread() {
>>>            public void run() {
>>>               try {
>>>                  WfProcess theProcess = sc.createProcess("pkg","pdef");
>>>                  theProcess.set_process_context(workflowRelevantData);
>>>                  theProcess.start();
>>>               } catch (RootException e) {
>>>                  e.printStackTrace();
>>>               }
>>>            }
>>>         }.start();
>>>      }
>>>
>>>
>>>
>>>
>>>
>>>>Thank you very much.
>>>>Best Regards,
>>>>He Wei
>>>>
>>>>-----Original Message-----
>>>>From: Vladimir Puskas [mailto:[email protected]] 
>>>>Sent: Wednesday, July 20, 2005 4:55 PM
>>>>To: [email protected]
>>>>Subject: Re: [shark] parallel processes and thread.sleep
>>>>
>>>>Hello Sebastian
>>>>
>>>>you wrote:
>>>>
>>>>
>>>>
>>>>
>>>>>hi list,
>>>>>
>>>>>i try to realise some parallel processes (instances) and each of them
>>>>>uses the javascript-agent with java.lang.thread.sleep(20000). after that
>>>>>break, the process sends an email. now, i have the following problem:
>>>>>i start 5 processes at the same time.
>>>>>the first process starts, waits 20 seconds and sends an email.
>>>>>then, after that 20 seconds, the second process waits 20 seconds and
>>>>>sends an email.
>>>>>then, after 40 seconds, the thrid process waits 20 seconds and sends an
>>>>>email.
>>>>>...
>>>>>is it possible, that all the processes start at the same time, wait 20
>>>>>seconds and send then an email? i tried the beer pause example too -
>>>>>same result.
>>>>
>>>>
>>>>POJO Shark is library, so it won't open a new thread to execute process.
>>>>If your tool-agent says sleep 20sec Shark will obey, and your
>>>>application will too, that's why these process instances were executed 
>>>>in sequence :-)
>>>>
>>>>To make it *really* parallel let your application start a new Thread for
>>>>those 5 instances, or use Scheduler tool-agent in your xpdl (like
>>>>test-Scheduler.xpdl).
>>>
>>>
>>>hope this helps
>>
>>
>>
>>
>>
> 
> 
> 
>

--
You receive this message as a subscriber of the [email protected] mailing list.
To unsubscribe: mailto:[email protected]
For general help: mailto:[email protected]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws