Copy_from gets stuck with bigger input file

Dear all,

I encounter exactly the same problem and symptoms with a file of about
7500 lines, using psycopg 2.0.6 and 2.0.7 along postgresql 8.3.1.

Any fresh news about this issue ? Meanwhile, what is the best way to
deal with that ?

Best regards,
Julien

>
> I encountered this problem some time ago. I don't think I found a
> solution. See:
> http://lists.initd.org/pipermail/psycopg/2007-June/005552.html
> http://lists.initd.org/pipermail/psycopg/2007-June/005553.html
>
> On Feb 12, 2008, at 15:07 , Christian J Dietrich wrote:
>
> > Dear list,
> >
> > I am experiencing some problems with copy_from() when feeding it with
> > input files that are quite large (spoken in number of lines, not file
> > size). If a file exceeds a certain number of lines, copy_from simply
> > hangs and does not return. I am using psycopg2 2.0.6. PostgreSQL is
> > version 8.2.4.
> >
> > In detail, I tried to feed the table as defined below:
> > create table test_2 (my_num int);
> >
> > a) - created a file of 118k lines, each line is just "30000\n" (could
> >       be any string)
> >     - fed it into the DB via psycopg2
> >
> > b) - added 1k lines, each line is just "30000\n" to the file before
> >     - fed it to the DB via psycopg2
> >
> > Although a) works fine for me, b) just stucks. To me it's not clear
> > what happens here, since neither CPU nor memory is wasted during this
> > time. SIGINT does not interrupt the copy_from() call, first SIGTERM
> > actually stops the computation. Looks like a memory story to me, or
> > some
> > constraints of stdin. Apparently the number of lines is not the only
> > criteria, since copying tables with multiple columns leads to the same
> > problem with less rows.
> >
> > Would be great if someone could help me out. Just for completeness,
> > both
> > scenarios worked well using the cmd line client psql combined with
> > COPY.
> > Here are the file sizes:
> > 692K  /tmp/input118k
> > 698K  /tmp/input119k
> >
> > Thanks in advance!
> >
> > Cheers,
> > Christian
> >
> > _______________________________________________
> > Psycopg mailing list
> > Psycopg at lists.initd.org
> > http://lists.initd.org/mailman/listinfo/psycopg
> >