Re: [ITM] Large JDBC result sets with Agent Builder

Yep...We increased the max rows due to the large result set.  I know we can alter the query and change a few things around, but I was surprised that such a small result set (in my opinion) caused such performance problems, and was wondering if others had the same experiences.

-JKS

On Dec 10, 2012, at 9:39, Julius Loman <[email protected]> wrote:

> On Mon, Dec 10, 2012 at 09:16:49AM -0500, Jay Slay <[email protected]> wrote:
>> Hey guys.  Was wondering what sort of performance you have seen with agent
>> builder agents pulling more than a few thousand rows back from a query.  It
>> appears that over 10k rows is taking one of our agents more than a minute to
>> process.   While the agent server itself isn't the beefiest box out there,
>> over a minute seems a bit slow to me.  There is only a dozen or so columns
>> and the data in each is fairly small.  From the logging, we can see it is not
>> the query taking time, but the processing of the result set.
> Internally JDBC processing is done in separate JVM and communicating to the
> agent using local TCP connection. Data is transferred in XML and must this XML
> must be constructed and and parsed back. This could be the bottleneck you are
> facing.
> 
> I recall some settings that are limiting the result to 1000 rows - for
> performance resons. CDP_JDBC_MAX_ROWS. Did you increase this value?
> 
> I think you will to refactor your agent and return less rows anyway. Probably
> split the data into more attribute groups and using some filters in the SQL
> query.
> 
> 
> -- 
> 
> [ Julius Loman ][ [email protected] ][ http://lomo.kyberia.net ][ icq:35732873 ]
> 
> _______________________________________________
> TME10 mailing list
> [email protected]
> Unsubscribe:[email protected]

_______________________________________________
TME10 mailing list
[email protected]
Unsubscribe:[email protected]