Re: large resultset

On 2010-06-15, vinny <[email protected]> wrote:
>
> On Tue, 15 Jun 2010 10:33:05 +0100, Thom Brown <[email protected]>
> wrote:
>> On 15 June 2010 09:31, AI Rumman  wrote:
>>  How to return large resutlset (almost 2 millions record) in php?
>> 
>> Presumably this is so people can download results rather than display
>> them in a browser?
>> 
>> Here's a basic a raw version of what you can do:
>> 
>> $db = pg_connect($connection_string);
>> 
>> $results = pg_query($db, $query);
>> 
>> while ($result = pg_fetch_array($results)
>> {
>>     echo implode("|", $result) . "n";
>> }
>> 
>> pg_close($db);
>> 
>> Obviously that wouldn't do by itself, but it's quite simple to loop over
>> a result set.
>> 
>> Regards
>> 
>> Thom
>> 
>
> If that's the case surely you'd use some SQL to merge the data into one
> long string and return it in a single record.
> Looping over two million results is going to take a while,

yeah, so is transmitting the bytes.

> not to mention quite a bit of memory.

yeah, libpq is going to want to load the result-set into memory.
you'll probably hit PHPs memory limit and it'll be game over.

The only pure php solution is to use a cursor. (seing as pg_get_line
is not avaulable)

OTOH you can do copy like this...

  $c=escapeshellarg($connstr);
  $q=escapeshellarg($query);
  system ("psql $c -c 'copy ( '$q' ) to stdout with csv header'");
  
needs *nix based server and pgversion >= 8.3 (I think).

You don't get to choose the format beyond what pg supports, but I
think you can tweak encoding on the command-line if needed.
OTOH you could use ICONV for that. 

-- 
Sent via pgsql-php mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-php