fetchmany weirdness?

Hi all,

I'm using the latest psycopg2 download against a pgsql 8.4 server, and I'm
getting a really odd behavior. It's not clear whether it's related to
psycopg2 directly, but I figured someone here could help me out (and if it's
a bug, all the better that I report the issue here).

So I have some code that replicates the issue here:

    for id in db.get_users_by_joindate([joindate]):
        idcount += 1
        print idcount
        param = [id]
        if db.is_good_user(param):
            print "User good"
            skip_count += 1
            continue
        else:
            print "Not good"
            continue

Here's the deal:
*** db.get_users_by_joindate is a generator function. It calls a server-side
function, uses fetchmany() to get back results, and then yield to return the
id's.

*** If I insert a "continue" after the "print idcount" line, I'll see that
about 3000 records are returned for my test case.

*** If I *don't* insert a continue, then db.is_good_user is only called 100
times.

The place where I'm stuck debugging is this:  db.get_users_by_joindate sets
self.curs.arraysize to 100, so fetchmany() will grab 100 results at a time.
This is the same number of results that get processed by db.is_good_user if
I left it as-is above. I've changed the self.curs.arraysize to 10 as well,
and then only 10 results are processed by db.is_good_user.

Since the two database calls are methods living in the same object, using
the same connection and cursor, it seems like there's some kind of weird
interplay here that seems off to me. In the above code, I'm calling
db.is_good_user once for each iteration of the for loop, passing one
parameter each time. So why is it only executing 100 times instead of 3000?

Only thing I can think of is that there's some odd reuse thing going on
where, since the cursor is still processing db.get_users_by_joindate, it'll
only pass 100 results to db.is_good_user or something? Can someone help me
figure out how to avoid this issue, and help me understand it better?

Thanks a million.

brian

-- 
Brian K. Jones
Python Magazine  http://www.pythonmagazine.com
My Blog          http://www.protocolostomy.com

_______________________________________________
Psycopg mailing list
Psycopg-IAPFreCvJWPBWskQ1e/[email protected]
http://lists.initd.org/mailman/listinfo/psycopg