RE: Serious OutOfMemory issue while reading with mysql-connector-j-5.0.7 ...
"A.J. Blanchard \(ajblanch\)" <[email protected]> Mon, 27 Aug 2007 10:07:43 -0700
| Newsgroups | gmane.comp.db.mysql.java |
|---|---|
| Message-ID | <8C127CB1A6D97448A5CBB1A90395827E048CEFD4@xmb-sjc-22a.amer.cisco.com> |
Hum, I had this problem long ago on a much older version of drivers/engine. It amounted to the client side trying to read in the entire ResultsSet into the local JVM. Turned out we had to limit the ResultSet size to 30,000 some odd size to prevent the OOM Exception (LIMIT). Recently, we have been able to at least get the row count and seems the data was streaming. We did nothing special (in options) to achieve this. But that was pre-5.x. I have not run this specific test lately. Just to see if you can avoid the OOM, try LIMIT and heap up with the popular -Xmx512M (if I recall the syntax correctly). A. J. -- -----Original Message----- From: Robert DiFalco [mailto:[email protected]] Sent: Monday, August 27, 2007 11:40 AM To: Paul Palaszewski; Ronald Klop; [email protected] Subject: RE: Serious OutOfMemory issue while reading with mysql-connector-j-5.0.7 ... Sorry, Paul, I'm out of ideas. I have not seen this issue when I have server side prepared statements and cursors enabled -- even when iterating through tables with millions of rows. -----Original Message----- From: Paul Palaszewski [mailto:[email protected]] Sent: Monday, August 27, 2007 4:13 AM To: Ronald Klop; [email protected] Subject: Re: Serious OutOfMemory issue while reading with mysql-connector-j-5.0.7 ... Yes. Tried with and without fetch size. Did not change anything. Regards Paul Ronald Klop schrieb: > > Did you set Statement.setFetchSize()? > > Ronald. > > On Fri Aug 24 18:56:06 CEST 2007 Paul Palaszewski > <[email protected]> wrote: > > Hi Robert! > > I've tried useCursorFetch=true . According to the documentation that > also sets useServerPrepStmts=true - which it did. I confirmed that > with > the debugger. Did not help. > > You can see the options I've tried in the test class that I > attached to > the original message. > > Best regards, > Paul > > Robert DiFalco schrieb: > > Have you tried it with: > > > > useServerPrepStmts=true > > useCursorFetch=true > > > > ? > > > > -----Original Message----- > > From: Paul Palaszewski [mailto:[email protected]] > > Sent: Thursday, August 23, 2007 2:32 AM > > To: [email protected] > > Subject: Serious OutOfMemory issue while reading with > mysql-connector-j-5.0.7 ... > > > > Hi! > > > > Yesterday I was supprised to get an out of memory exception when > reading a big table rec by rec without any caching. The program > died in stmt.executeQuery(), long before I can do anything with > the records. > > > > I've tried different connection options and debugged connector/j > ... > > every time code runs into MysqlIO.readAllResults which can not > work, if the table has more records than available in the jvm. > More interesting - also server side prepared statements run into > that readAll-routine? > > > > Tested with > > connector-j-5.0.3 + 5.0.7, > > mysql db 5.0.41 + some older 5.0 on windows. > > > > C:\Test\src>javac -cp > ..\..\lib\jdbc\mysql-connector-java-5.0.7-bin.jar > > MySQLRead.java > > C:\Test\src>java -cp > ..\..\lib\jdbc\mysql-connector-java-5.0.7-bin.jar;. > > MySQLRead > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap > space > > at com.mysql.jdbc.Buffer.getBytes(Buffer.java:198) > > at com.mysql.jdbc.Buffer.readLenByteArray(Buffer.java:318) > > at > > com.mysql.jdbc.MysqlIO.extractNativeEncodedColumn(MysqlIO.java:3567) > > at > > com.mysql.jdbc.MysqlIO.unpackBinaryResultSetRow(MysqlIO.java:3483) > > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1391) > > at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2369) > > at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:451) > > at > > > com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2076) > > at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1451) > > at > > > com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1314) > > at > > > com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:740) > > at > > > com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1448) > > at MySQLRead.main(MySQLRead.java:23) > > > > Regards > > Paul > > > > -- > > -- > > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ > > _/ > > _/ BDM Business Data Management GmbH > > _/ > > _/ Firmenbuch: FN 242059w > > _/ Gerichtsstand: Landesgericht Wiener Neustadt > > _/ UID: ATU57685758 > > _/ > > _/ Grenzgasse 111, Objekt 9/4 > > _/ 2340 M?dling > > _/ Tel,Fax: +43 2236 46 393 > > _/ > > _/ Ing. Paul Palaszewski, (Hons) B.Sc. > > _/ Gesch?ftsf?hrer > > _/ Mobil: +43(650) 37 36 030 > > _/ E-Mail: [email protected] > > _/ > > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ > > > > > > > > > > > > > -- > -- > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ > _/ > _/ BDM Business Data Management GmbH > _/ > _/ Firmenbuch: FN 242059w > _/ Gerichtsstand: Landesgericht Wiener Neustadt > _/ UID: ATU57685758 > _/ > _/ Grenzgasse 111, Objekt 9/4 > _/ 2340 M?dling > _/ Tel,Fax: +43 2236 46 393 > _/ > _/ Ing. Paul Palaszewski, (Hons) B.Sc. > _/ Gesch?ftsf?hrer > _/ Mobil: +43(650) 37 36 030 > _/ E-Mail: [email protected] > _/ > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ > > > > -- > MySQL Java Mailing List > For list archives: http://lists.mysql.com/java > To unsubscribe: > http://lists.mysql.com/[email protected] > -- -- _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ _/ _/ BDM Business Data Management GmbH _/ _/ Firmenbuch: FN 242059w _/ Gerichtsstand: Landesgericht Wiener Neustadt _/ UID: ATU57685758 _/ _/ Grenzgasse 111, Objekt 9/4 _/ 2340 Mödling _/ Tel,Fax: +43 2236 46 393 _/ _/ Ing. Paul Palaszewski, (Hons) B.Sc. _/ Geschäftsführer _/ Mobil: +43(650) 37 36 030 _/ E-Mail: [email protected] _/ _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ -- MySQL Java Mailing List For list archives: http://lists.mysql.com/java To unsubscribe: http://lists.mysql.com/[email protected] -- MySQL Java Mailing List For list archives: http://lists.mysql.com/java To unsubscribe: http://lists.mysql.com/[email protected] -- MySQL Java Mailing List For list archives: http://lists.mysql.com/java To unsubscribe: http://lists.mysql.com/[email protected]