Re: performance is slow

Al Niessner <[email protected]>
Newsgroups gmane.comp.python.db.psycopg.devel
Message-ID <1244668238.4273.98.camel@charon>
I have some results. I create the cursor and keep it active until I have
inserted all of the information that I have and done all of my updates
at which point I do a commit and then close the cursor. It has improved
the 44 name-value pair entries by an order of magnitude but only halved
the blob insertion. All in all it is a 5x improvement for the whole
process. It is at least becoming tolerable.

Any more suggestions on how to speed things up? The one thing that I do
not like is that I see the network usage sky rocket during this time. Is
there a way that I can have a pipe connection instead of network?

On Wed, 2009-06-10 at 11:17 -0700, Tim Roberts wrote:
> Al Niessner wrote:
> > The main thing that I think I am doing wrong that I want to change is
> > that I am doing a commit after after insert. I am looking at changing
> > that to a bigger more "transactional" item. Not sure if this will
> > improve the overall time, but it seems like a more DB way of doing
> > things.
> >   
> 
> That is the key right there.  If you change your code so that you do not
> commit until your whole series of inserts and updates is complete,
> you'll find that the performance is quite acceptable.
> 
-- 
Al Niessner

I have never found the companion that was so companionable as solitude.
- From Walden by Henry David Thoreau

The universe is indifferent, and life is brutal; however, it is man's
choice of behavior that makes them malevolent rather than benevolent.

Some will fall in love with life and drink it from a fountain
That is pouring like an avalanche coming down the mountain.
- From the song Pepper by the Butthole Surfers
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.