Re: fetchmany weirdness?

So, in my db.is_good_user method, instead of using the self.curs() created
at the class level, I created a new cursor local to the method. That solves
the issue, though I'd still like to know more about this problem if anyone
has some wisdom to share.

Thanks.

On Mon, Nov 30, 2009 at 2:53 PM, Brian Jones <[email protected]> wrote:

> Hi all,
>
> I'm using the latest psycopg2 download against a pgsql 8.4 server, and I'm
> getting a really odd behavior. It's not clear whether it's related to
> psycopg2 directly, but I figured someone here could help me out (and if it's
> a bug, all the better that I report the issue here).
>
> So I have some code that replicates the issue here:
>
>     for id in db.get_users_by_joindate([joindate]):
>         idcount += 1
>         print idcount
>         param = [id]
>         if db.is_good_user(param):
>             print "User good"
>             skip_count += 1
>             continue
>         else:
>             print "Not good"
>             continue
>
> Here's the deal:
> *** db.get_users_by_joindate is a generator function. It calls a
> server-side function, uses fetchmany() to get back results, and then yield
> to return the id's.
>
> *** If I insert a "continue" after the "print idcount" line, I'll see that
> about 3000 records are returned for my test case.
>
> *** If I *don't* insert a continue, then db.is_good_user is only called 100
> times.
>
> The place where I'm stuck debugging is this:  db.get_users_by_joindate sets
> self.curs.arraysize to 100, so fetchmany() will grab 100 results at a time.
> This is the same number of results that get processed by db.is_good_user if
> I left it as-is above. I've changed the self.curs.arraysize to 10 as well,
> and then only 10 results are processed by db.is_good_user.
>
> Since the two database calls are methods living in the same object, using
> the same connection and cursor, it seems like there's some kind of weird
> interplay here that seems off to me. In the above code, I'm calling
> db.is_good_user once for each iteration of the for loop, passing one
> parameter each time. So why is it only executing 100 times instead of 3000?
>
> Only thing I can think of is that there's some odd reuse thing going on
> where, since the cursor is still processing db.get_users_by_joindate, it'll
> only pass 100 results to db.is_good_user or something? Can someone help me
> figure out how to avoid this issue, and help me understand it better?
>
> Thanks a million.
>
> brian
>
> --
> Brian K. Jones
> Python Magazine  http://www.pythonmagazine.com
> My Blog          http://www.protocolostomy.com
>



-- 
Brian K. Jones
Python Magazine  http://www.pythonmagazine.com
My Blog          http://www.protocolostomy.com

_______________________________________________
Psycopg mailing list
Psycopg-IAPFreCvJWPBWskQ1e/[email protected]
http://lists.initd.org/mailman/listinfo/psycopg