Re: Large queries

The feature in Hibernate is called "ScrollableResult". I think this is what
you need?

On Wed, Jul 23, 2008 at 1:20 PM, Tuna Toksöz <[email protected]> wrote:

> I think there was something similar to this in progress/in plans, but not
> sure. My idea would be to get all the ids for the database so that you'll
> have the required when you access them. Then user may specift chunk size,
> the data is retrieved depending on their ids when required. Data is stored
> as weakreference, so they can be collected by garbage collector when it runs
> out of memory. I think this is what db4o currently uses.
>
> Regards,
>
> On Wed, Jul 23, 2008 at 12:38 PM, Jan Oravec <[email protected]> wrote:
>
>> Hi,
>>
>>
>> It would be interesting to decrease memory consumption with very large
>> queries (i.e. millions to billions of returned objects), especially when
>> returned data are processed one-by-one and thrown away (and
>> CacheMode.Ignore is used).
>>
>> Currently, ICriteria.List() first loads results into List, which is then
>> returned. First result is not usable by application before all results
>> are loaded, increasing both latency and memory consumption.
>>
>> I propose conversion of all internal code paths using Lists to
>> IEnumerable (either explicitly defining enumerator, or using 'yield
>> return ...').
>>
>> External APIs should be backward compatible (i.e. ICriteria.Enumerable()
>> would return IEnumerable and ICriteria.List would use .Enumerable() to
>> create List.
>>
>> I expect this change would greatly decrease memory consumption and
>> latency (even for smaller queries), while very slightly increasing
>> required CPU time. Overall single-thread throughput should be also
>> improved, as application could process data while they are still on the
>> way from database server.
>>
>> If this change is acceptable, I would be interested in working on this.
>>
>>
>> Jan
>>
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win great
>> prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the
>> world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Nhibernate-development mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/nhibernate-development
>>
>>
>
>
> --
> Tuna Toksöz
>
> Typos included to enhance the reader's attention ...
>



-- 
Tuna Toksöz

Typos included to enhance the reader's attention ...

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

_______________________________________________
Nhibernate-development mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nhibernate-development