Re: Fetch-on-demand iterators (was Re: [CDBI] Make CDBI go fast)

Perrin Harkins wrote:
> On 2/15/07, Michael G Schwern <[email protected]> wrote:
>> Whatever the backend implementation, it sure sped things up.  The
>> performance of search()->first when the search returns a million rows
>> went from about 2 seconds to almost negligable.  Remember, even if the
>> database is doing the same amount of work it still has to transfer it
>> to your process and then run it through DBI and then bind it all into
>> a list of hashes.
> 
> With DBD::mysql (and others), it will still transfer the rows to your
> process, even if you never fetch them.  Using "mysql_use_result"
> prevents this.

Looks handy and dangerous.  Thanks.


>> Do not be fooled!  There really is no "give me everything" bulk row
>> loading optimization in DBI.  selectall/fetchall is just doing
>> $sth->fetch in a loop!
> 
> It does it in a fast internal C loop though, which does help.

You're right, I keep getting fooled by those pure Perl implementations.