[f2py] MemoryLeak

Trond Kristiansen <[email protected]>
Newsgroups gmane.comp.python.f2py.user
Message-ID <[email protected]>
Hi all.

I have a problem with releasing memory from my Python+F2PY  
application. From Python, I loop over a list of large netCDF4 files,  
and for each loop I extract a chunk of data (~200MB) which is stored  
in numpy arrays. These data are then sent to Fortran functions wrapped  
using F2PY. This works great for a while, before the application  
crashes with the error: MemoryError (error code=12). I assume this is  
because for each time I send an array to my Fortran function, a copy  
of the data is made which is not released but builds up in memory. I  
can loop over 16 files before my application crashes.

Since I assumed this was because of the extra copy of my array (a)  
done in F2PY to convert from 'C' order to 'Fortran' order, I did a  
"numpy.asarray(a,order='Fortran'). When I check my Fortran application  
using -DF2PY_REPORT_ON_ARRAY_COPY I made sure that no copies were made.

However, my application still constantly increases in memory, and  
leaves me with few clues to figure out what is going on. I call my  
function in Fortran using expressions like the one shown below.

 From Python:
  outdata =  
interp.interpolation.dovertinter(np.asarray(outdata,order='Fortran'),
                                                         
np.asarray(data_ST,order='Fortran'),
                                                         
np.asarray(grdROMS.depth,order='Fortran'),
                                                         
np.asarray(grdROMS.z_r,order='Fortran'),
                                                         
np.asarray(grdSODA.z_r,order='Fortran'),
                                                         
int(grdROMS.Nlevels),
                                                         
int(grdSODA.Nlevels),
                                                         
int(grdROMS.xi_rho),
                                                         
int(grdROMS.eta_rho),
                                                         
int(grdROMS.xi_rho),
                                                         
int(grdROMS.eta_rho))

My Fortran routine looks like this:

double precision rz2, rz1
             integer eta_rho, xi_rho, II, JJ, ic, jc, kc, kT, Nsoda,  
Nroms
             double precision, dimension(Nsoda,JJ,II) :: dat
             double precision, dimension(eta_rho,xi_rho) :: bathymetry
             double precision, dimension(Nroms,JJ,II) :: outdat
             double precision, dimension(Nsoda) ::  zs
             double precision, dimension(Nroms,eta_rho,xi_rho) :: zr

!f2py intent(in) dat, bathymetry, zr, zs
!f2py intent(in) Nroms, Nsoda, JJ, II, xi_rho, eta_rho
!f2py intent(in,out) outdat
!f2py intent(hide) ic,jc,kc,kT,rz1,rz2


I hope that some of you with better knowledge of this can take a look  
and give me a hint where it goes wrong.

Thank you for your help.

Cheers, Trond
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.