Re: Large queries

Hmm, not sure about that. Depends on current progress in NHCore. I don't
think it is available in trunk and not sure when it becomes available. But I
think this can be made optional and can be done with a modification to query
options& translator without so much pain.


On Wed, Jul 23, 2008 at 2:17 PM, Jan Oravec <[email protected]> wrote:

> Yep, something like ScrollableResult. I think LINQ provider should use
> it by default (there is no point in keeping any result lists for LINQ),
> like:
>
> foreach (Foo x in session.Linq<Foo> ().Where(...))
> {
>  ...
> }
>
> While thinking more about it, there could be a problem when code inside
> foreach perform other database operations and database driver doesn't
> support multiple queries. Your solution with keeping all IDs would solve
> that, but with huge number of records the amount of memory required can
> be still large. Also, processing must wait for initial query, so latency
> is not decreased.
>
>
> Jan
>
>
> On Wed, 2008-07-23 at 13:37 +0300, Tuna Toksöz wrote:
> > The feature in Hibernate is called "ScrollableResult". I think this is
> > what you need?
> >
> > On Wed, Jul 23, 2008 at 1:20 PM, Tuna Toksöz <[email protected]>
> > wrote:
> >         I think there was something similar to this in progress/in
> >         plans, but not sure. My idea would be to get all the ids for
> >         the database so that you'll have the required when you access
> >         them. Then user may specift chunk size, the data is retrieved
> >         depending on their ids when required. Data is stored as
> >         weakreference, so they can be collected by garbage collector
> >         when it runs out of memory. I think this is what db4o
> >         currently uses.
> >
> >         Regards,
> >
> >
> >         On Wed, Jul 23, 2008 at 12:38 PM, Jan Oravec
> >         <[email protected]> wrote:
> >
> >
> >                 Hi,
> >
> >
> >                 It would be interesting to decrease memory consumption
> >                 with very large
> >                 queries (i.e. millions to billions of returned
> >                 objects), especially when
> >                 returned data are processed one-by-one and thrown away
> >                 (and
> >                 CacheMode.Ignore is used).
> >
> >                 Currently, ICriteria.List() first loads results into
> >                 List, which is then
> >                 returned. First result is not usable by application
> >                 before all results
> >                 are loaded, increasing both latency and memory
> >                 consumption.
> >
> >                 I propose conversion of all internal code paths using
> >                 Lists to
> >                 IEnumerable (either explicitly defining enumerator, or
> >                 using 'yield
> >                 return ...').
> >
> >                 External APIs should be backward compatible (i.e.
> >                 ICriteria.Enumerable()
> >                 would return IEnumerable and ICriteria.List would
> >                 use .Enumerable() to
> >                 create List.
> >
> >                 I expect this change would greatly decrease memory
> >                 consumption and
> >                 latency (even for smaller queries), while very
> >                 slightly increasing
> >                 required CPU time. Overall single-thread throughput
> >                 should be also
> >                 improved, as application could process data while they
> >                 are still on the
> >                 way from database server.
> >
> >                 If this change is acceptable, I would be interested in
> >                 working on this.
> >
> >
> >                 Jan
> >
> >
> >
> >
> -------------------------------------------------------------------------
> >                 This SF.Net email is sponsored by the Moblin Your Move
> >                 Developer's challenge
> >                 Build the coolest Linux based applications with Moblin
> >                 SDK & win great prizes
> >                 Grand prize is a trip for two to an Open Source event
> >                 anywhere in the world
> >
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >                 _______________________________________________
> >                 Nhibernate-development mailing list
> >                 [email protected]
> >
> https://lists.sourceforge.net/lists/listinfo/nhibernate-development
> >
> >
> >
> >
> >         --
> >         Tuna Toksöz
> >
> >         Typos included to enhance the reader's attention ...
> >
> >
> >
> > --
> > Tuna Toksöz
> >
> > Typos included to enhance the reader's attention ...
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> > Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the
> world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________ Nhibernate-development
> mailing list [email protected]
> https://lists.sourceforge.net/lists/listinfo/nhibernate-development
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Nhibernate-development mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nhibernate-development
>
>


-- 
Tuna Toksöz

Typos included to enhance the reader's attention ...

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

_______________________________________________
Nhibernate-development mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nhibernate-development