Re: **SOLVED** objectsWithFetchSpecification freezes during a slow call

Well, 

By examining the state of the hung instance, we saw that the mysql connector class ReadAheadInputStream was just sitting with a constant socketRead.

So, it turned out that it is a problem in the mysql-connector-java-3.1.8.jar
See:
	- http://bugs.mysql.com/bug.php?id=9515
	- http://bugs.mysql.com/bug.php?id=31353

Those links describe the bug i.e. Hangs in ReadAheadInputStream.fill() 

We upgraded from:
mysql-connector-java-3.1.8.jar 
	to 
mysql-connector-java-5.1.26.jar
	
and the problem went away.

This may also have been the cause of an occasional freezing of our instances (which was happening infrequently and randomly).  We had to force-quit them to get them going again.


Thanks for you help in this.

Mark Gowdy


On 9 Aug 2013, at 06:50, Chuck Hill wrote:

> 
> On 2013-08-08, at 12:21 PM, Mark Gowdy wrote:
> 
>> On 8 Aug 2013, at 19:21, Chuck Hill <[email protected]> wrote:
>> 
>>> Hi Mark,
>>> 
>>> It freezes for a while (10+ seconds) or it freezes forever?  
>> 
>> Forever.  No errors. It just doesn't progress past the fetch command. 
>> 
>>> Have you tried a "sudo jstack -F <process id> to see where it is frozen?  
>> 
>> Will do, but not right now.
>> (Waiting for my main course to arrive.)
>> 
>>> It does sound like a timeout thing, perhaps at the JDBC level.  
>> 
>> I think so too, but not sure how to fix/increase it. 
> 
> The jstack may suggest where it is happening.
> 
>> 
>>> Does CREATED_DATE have an index?  
>> 
>> No, and we will add one.
>> But the point is, it should not freeze if MySql takes a few second too long too respond. 
> 
> No, it should not.  I have several apps that have much longer running queries and it is not a problem for EOF.  I am pretty sure you will find the problem lies in MySQL.
> 
> 
> Chuck
> 
> 
>> 
>>> CREATED_DATE, QUEUE_ITEM_PK would be the fastest.
>>> 
>>> Chuck
>>> 
>>> 
>>> On 2013-08-08, at 10:22 AM, Mark Gowdy wrote:
>>> 
>>>> Hi, 
>>>> 
>>>> Have been trying to track down the cause of a very strange problem we have been having.
>>>> 
>>>> Using the MySQLPlugIn in Wonder, if we do a fetch (or anything SQL related), and it takes a while for MySQL to respond, the instance freezes (it does not get past the objectsWithFetchSpecification line).
>>>> We originally thought it was something to do with the memory & tables becoming large, but now we are thinking it is a response time-out thing.
>>>> 
>>>> We can reproduce the problem quite easily now by specifying a sort ordering, we will give MySQL a bit of work to do (and slow down the response).
>>>> 
>>>> So, if we run the generated SQL directly in Sequel Pro:
>>>> 
>>>> (1) Including the slow field (CREATED_DATE)- Takes about 10 seconds for MySQL to return some values.
>>>> SELECT RTRIM(t0.CONFIG), t0.CREATED_DATE, t0.IDENTIFIER_1, t0.IDENTIFIER_2, t0.IDENTIFIER_3, t0.IDENTIFIER_4, t0.ITEM_DATE, t0.MODIFIED_DATE, t0.QUEUE_DETAILS_FK, t0.QUEUE_ITEM_PK, t0.QUEUE_TAG, t0.STATUS FROM queue_item t0 WHERE (t0.QUEUE_TAG = 'mb:nui:RENDER_MEDIA' AND (t0.ITEM_DATE < '2013-08-08 16:46:17 Etc/GMT' OR t0.ITEM_DATE is NULL)) ORDER BY t0.CREATED_DATE ASC, t0.QUEUE_ITEM_PK ASC LIMIT 50
>>>> 
>>>> (2) Remove the slow sort field - Takes about 72ms for MySQL to return some values.
>>>> SELECT RTRIM(t0.CONFIG), t0.CREATED_DATE, t0.IDENTIFIER_1, t0.IDENTIFIER_2, t0.IDENTIFIER_3, t0.IDENTIFIER_4, t0.ITEM_DATE, t0.MODIFIED_DATE, t0.QUEUE_DETAILS_FK, t0.QUEUE_ITEM_PK, t0.QUEUE_TAG, t0.STATUS FROM queue_item t0 WHERE (t0.QUEUE_TAG = 'mb:nui:RENDER_MEDIA' AND (t0.ITEM_DATE < '2013-08-08 16:46:17 Etc/GMT' OR t0.ITEM_DATE is NULL)) ORDER BY t0.QUEUE_ITEM_PK ASC LIMIT 50
>>>> 
>>>> Now, in our app, during the objectsWithFetchSpecification(..)
>>>> (1) freezes the app totally.. no error messages, just frozen.
>>>> (2) works fine.
>>>> 
>>>> Any thoughts?
>>>> 
>>>> Thanks, 
>>>> 
>>>> Mark
>>>> 
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>>>> It's a free troubleshooting tool designed for production.
>>>> Get down to code-level detail for bottlenecks, with <2% overhead. 
>>>> Download for free and get started troubleshooting in minutes. 
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
>>>> Wonder-disc mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/wonder-disc
>>> 
>>> -- 
>>> Chuck Hill             
>>> Executive Managing Partner, VP Development and Technical Services
>>> 
>>> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.    
>>> http://www.global-village.net/gvc/practical_webobjects
>>> 
>>> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest Growing Companies in B.C! 
>>> 
>>> Global Village Consulting ranks 44th in 25th annual PROFIT 500 ranking of Canada’s Fastest-Growing Companies by PROFIT Magazine!
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
> 
> -- 
> Chuck Hill             
> Executive Managing Partner, VP Development and Technical Services
> 
> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.    
> http://www.global-village.net/gvc/practical_webobjects
> 
> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest Growing Companies in B.C! 
> 
> Global Village Consulting ranks 44th in 25th annual PROFIT 500 ranking of Canada’s Fastest-Growing Companies by PROFIT Magazine!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk