Re: Threads and findUsingPrototype

If you have non-trivial queries for MSSQL try 

UPDATE STATISTICS

on the main tables.  I have found that sometimes it just looses track and starts running slowly for no apparent reason.

Anthony

At 01:18 PM 7/11/2009, you wrote:
>  
>
>Thanks for the feedback!  Right now the database is responsible for basically 100% of the run time because the main thread (which is actually querying the dataset, not the database) is always faster than the database query thread.  So there's no point speeding up findUsingPrototype (used in the main thread) anymore until we get the database query thread working faster.
>
>But that could just be because something odd is happening over in SQL Server.  For now, we're able to blame Microsoft for our problems.  
>
>Maybe we'll try HSQLDB.  I wasn't aware that it did hybrid disk/memory operations (our 65 million records will NOT fit into memory all at once), and I wasn't aware that it could process .CSV files directly.  Those two features together with a potential speed advantage make it look very promising.  Our users like .CSV files for the smaller tables.
>
>--
>John 
>
>
>On 6-Nov-09, at 7:19 PM, <mailto:[email protected]>[email protected] wrote:
>
>>Hello John,
>>
>>THREADING
>>
>>Your threading approach is OK. 
>>
>>In normal operation, you would have one session per thread, one thread per session. This can be overridden in special cases, but normally there are sanity checks to ensure that it is the case. Sharing a JDBC connection between threads is certainly asking for deep trouble.
>>
>>Queing the SDataSets is a reasonable approach. There are examples in the LongTransactionTest as to how to detach and reattach SDataSets.
>>
>>An alternative would be to simply have two threads, each of which does the reading and the processing, and then synchronize to ensure that only one ever does critical processing at a time. Avoids need for a queue. 
>>
>>But generally, I would not introduce complex architectures for maximum gain of only 30%.
>>
>>More importantly, can the processing itself be multi threaded? You have not indicated the application, so hard for me to know. But a quad core machine might increase throughput by 400%, much better than 30% improvement.
>>
>>One way to go much faster is to use a Java embeded database. HSQL, for example can be well over 1000% faster than Oracle. Daffodil, H2 and JavaDB are probably more reliable, have not tested for speed. But again, if the database is only consuming 30% of your time, 
>>
>
>Dr Anthony Berglas, [email protected]       Mobile: +61 4 4838 8874
>Just because it is possible to push twigs along the ground with ones nose
>does not necessarily mean that is the best way to collect firewood.


------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

<*> To change settings via email:
    [email protected] 
    [email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/