[f2py] MemoryLeak
Trond Kristiansen <[email protected]>
| Newsgroups | gmane.comp.python.f2py.user |
|---|---|
| Message-ID | <[email protected]> |
Hi all.
I have a problem with releasing memory from my Python+F2PY
application. From Python, I loop over a list of large netCDF4 files,
and for each loop I extract a chunk of data (~200MB) which is stored
in numpy arrays. These data are then sent to Fortran functions wrapped
using F2PY. This works great for a while, before the application
crashes with the error: MemoryError (error code=12). I assume this is
because for each time I send an array to my Fortran function, a copy
of the data is made which is not released but builds up in memory. I
can loop over 16 files before my application crashes.
Since I assumed this was because of the extra copy of my array (a)
done in F2PY to convert from 'C' order to 'Fortran' order, I did a
"numpy.asarray(a,order='Fortran'). When I check my Fortran application
using -DF2PY_REPORT_ON_ARRAY_COPY I made sure that no copies were made.
However, my application still constantly increases in memory, and
leaves me with few clues to figure out what is going on. I call my
function in Fortran using expressions like the one shown below.
From Python:
outdata =
interp.interpolation.dovertinter(np.asarray(outdata,order='Fortran'),
np.asarray(data_ST,order='Fortran'),
np.asarray(grdROMS.depth,order='Fortran'),
np.asarray(grdROMS.z_r,order='Fortran'),
np.asarray(grdSODA.z_r,order='Fortran'),
int(grdROMS.Nlevels),
int(grdSODA.Nlevels),
int(grdROMS.xi_rho),
int(grdROMS.eta_rho),
int(grdROMS.xi_rho),
int(grdROMS.eta_rho))
My Fortran routine looks like this:
double precision rz2, rz1
integer eta_rho, xi_rho, II, JJ, ic, jc, kc, kT, Nsoda,
Nroms
double precision, dimension(Nsoda,JJ,II) :: dat
double precision, dimension(eta_rho,xi_rho) :: bathymetry
double precision, dimension(Nroms,JJ,II) :: outdat
double precision, dimension(Nsoda) :: zs
double precision, dimension(Nroms,eta_rho,xi_rho) :: zr
!f2py intent(in) dat, bathymetry, zr, zs
!f2py intent(in) Nroms, Nsoda, JJ, II, xi_rho, eta_rho
!f2py intent(in,out) outdat
!f2py intent(hide) ic,jc,kc,kT,rz1,rz2
I hope that some of you with better knowledge of this can take a look
and give me a hint where it goes wrong.
Thank you for your help.
Cheers, Trond