FetchSpecificationBatchIterator
Max Muller <[email protected]>
| Newsgroups | gmane.comp.web.webobjects.wonder-disc,gmane.comp.web.webobjects.eof |
|---|---|
| Message-ID | <[email protected]> |
Hi, So I finally got around to writing the batch iterator that was discussed a while back on the lists. I opted for the simpler solution of first fetching all of the primary keys for the given qualifier of the fetch specification and then iterating through chunks of those primary keys fetching the actual eos with an IN qualifier (Thanks Anjo). This has the effect of on the request for the first batch causing two fetches to the database. All subsequent batch requests just require one fetch. I also allow the editing context used to be swapped out between batch requests. So a typical use case for iterating through 1,000,000 eos would be to create a new editing context and set that on the iterator before calling the nextBatch method. This way all of the eos created within a batch could be garbage collected. The batch iterator works great except for one draw back. Currently if the batch size is set over around 300 then the JDBC query actually runs out of memory (something like more than 64 KB has been exceeded). I remember this wasn't the case in the 4.5 world, Oracle actually chokes for IN queries greater than 1,000 if I recall correctly. For my purposes batching groups of 250 eos is fine, but for other applications a larger batch size would be nice. Any suggestions? Regards, Max ps - Source for ERXFetchSpecificationBatchIterator is in ERExtensions also here: (http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/wonder/Wonder/Common/ Frameworks/ERExtensions/Sources/er/extensions/ ERXFetchSpecificationBatchIterator.java?rev=HEAD&content-type=text/ vnd.viewcvs-markup). ------------------------------------------------------- This sf.net email is sponsored by: Influence the future of Java(TM) technology. Join the Java Community Process(SM) (JCP(SM)) program now. http://ads.sourceforge.net/cgi-bin/redirect.pl?sunm0004en