Re: [ITM] Large JDBC result sets with Agent Builder

On Mon, Dec 10, 2012 at 09:49:50AM -0500, Jay Slay <[email protected]> wrote:
> Yep...We increased the max rows due to the large result set.  I know we can alter the query and change a few things around, but I was surprised that such a small result set (in my opinion) caused such performance problems, and was wondering if others had the same experiences.
Use the latest 6.2.3.1 Agent Buider and test the query directly in the agent
builder.  How long does the query taket to process? Which processes are CPU
intensive at that time?
> 
> -JKS
> 
> On Dec 10, 2012, at 9:39, Julius Loman <[email protected]> wrote:
> 
> > On Mon, Dec 10, 2012 at 09:16:49AM -0500, Jay Slay <[email protected]> wrote:
> >> Hey guys.  Was wondering what sort of performance you have seen with agent
> >> builder agents pulling more than a few thousand rows back from a query.  It
> >> appears that over 10k rows is taking one of our agents more than a minute to
> >> process.   While the agent server itself isn't the beefiest box out there,
> >> over a minute seems a bit slow to me.  There is only a dozen or so columns
> >> and the data in each is fairly small.  From the logging, we can see it is not
> >> the query taking time, but the processing of the result set.
> > Internally JDBC processing is done in separate JVM and communicating to the
> > agent using local TCP connection. Data is transferred in XML and must this XML
> > must be constructed and and parsed back. This could be the bottleneck you are
> > facing.
> > 
> > I recall some settings that are limiting the result to 1000 rows - for
> > performance resons. CDP_JDBC_MAX_ROWS. Did you increase this value?
> > 
> > I think you will to refactor your agent and return less rows anyway. Probably
> > split the data into more attribute groups and using some filters in the SQL
> > query.
> > 
> > 
> > -- 
> > 
> > [ Julius Loman ][ [email protected] ][ http://lomo.kyberia.net ][ icq:35732873 ]
> > 
> > _______________________________________________
> > TME10 mailing list
> > [email protected]
> > Unsubscribe:[email protected]
> 
> 

> _______________________________________________
> TME10 mailing list
> [email protected]
> Unsubscribe:[email protected]


-- 

[ Julius Loman ][ [email protected] ][ http://lomo.kyberia.net ][ icq:35732873 ]

_______________________________________________
TME10 mailing list
[email protected]
Unsubscribe:[email protected]