performance Insert/update

Hi,

I have binary files containing numerical simulation data, written
from several cpu's, e.g. 4 processes.
No I want to put all in a table, but if an entry has already added
for cpu 1, and now I want to add a similar entry but from cpu 2,
an update has to be done (except for columns defined as indices)


Now I have a file with 1 400 000 rows in binary format. I read them
and putting all in my table needs around 250 seconds, which is
to slow. Has anybody an Idea of how to speed it up?
Here is the method, I'm using predefined statements with placeholders "?":

    def addRows(self, rowList):
        """ Add a list of rows to the database table"""
        
        # check data set
        if (len(rowList[0]) != self.nCols):
            raise InsertRowError('Wrong number of columns in inserted data 
set!')
        
        # begin transaction
#        self.dbCurs.execute("BEGIN TRANSACTION")
        
        # loop over all entries in list
        for row in rowList:
            try:
                self.dbCurs.execute(self.isrtStmt, row)
            except IntegrityError:
                # data set already exists, construct key in a tuple
                slctParams = row[self.indizes[0]],
                for i in self.indizes[1:]:
                    slctParams += row[i],
                
                # do select using the conflicting key to get the data
                self.dbCurs.execute(self.selStmt, slctParams)
                res = self.dbCurs.fetchall()
                
                # there has to be exactly one match
                if (len(res) != 1):
                    raise IntegrityError("Conflict with more than one line!")
                
                # get result
                conflRow = res[0]
                
                # get values from rowData which need to be added to the 
mutable columns of the conflicting row
                # and values of the conflicting row
                upParams = (row[self.mutableCols[0]] + 
conflRow[self.mutableCols[0]]),
                for i in self.mutableCols[1:]:
                    upParams += (row[i] + conflRow[i]),
    #            print 'parameters for update: ', upParams
                    
                # update conflicting row
                self.dbCurs.execute(self.updStmt, (upParams + slctParams))
                
                # debug
                self.dbCurs.execute(self.selStmt, slctParams)
                res = self.dbCurs.fetchall()
            
        # finish transaction
        self.dbCon.commit()

Actually, my idea was to bundle everything in one transaction...

Thanks in advance,

Chris
-- 

============================
M.Sc. Christoph Scheit
Institute of Fluid Mechanics
FAU Erlangen-Nuremberg
Cauerstrasse 4
D-91058 Erlangen
Phone: +49 9131 85 29508
============================