[f2py] f2py with openMPI
Gaetan Kenway <kenway-kBRAJjP0/PtSpjfjxSPG1fd9D2ou9A/[email protected]> Fri, 9 Nov 2012 08:30:23 -0500
| Newsgroups | gmane.comp.python.f2py.user |
|---|---|
| Message-ID | <CAA3=JrN-z8=gTh5WCpuN+uFsyqMS7GPVg36Lj+Z7JXUcHkEF-g@mail.gmail.com> |
--===============3195602987682986302==
Content-Type: multipart/alternative; boundary=e89a8f502fc03902ca04ce0ff423
--e89a8f502fc03902ca04ce0ff423
Content-Type: text/plain; charset=ISO-8859-1
Hello everyone
We have wrapped a number of mpi codes with f2py and have found a slightly
different way of compiling things was much easier for us.
The basic idea is to only use f2py to generate the wrapper .c file and if
you have modules or common blocks a .f and a .f90 file. You then proceed to
compile them yourself manually, and with your desired (mpi) compiler. In
the example I have below, I have a pre-computed and static .pyf file with a
module called "warp" defined in warp.pyf
PYTHON_OBJECTS = fortranobject.o \
warpmodule.o \
warp-f2pywrappers2.o \
# Generate Python include directory
$(eval PYTHON_INCLUDES = $(shell python-config --includes))
# Generate Numpy include directory
$(eval NUMPY_INCLUDES = $(shell python -c 'import numpy; print
numpy.get_include()'))
# Generate f2py root directory
$(eval F2PY_ROOT = $(shell python get_f2py.py))
# Run f2py to generate required files and compile
f2py warp.pyf
# Start compiling the required files:
$(CC) $(CC_FLAGS) $(PYTHON_INCLUDES) -I$(NUMPY_INCLUDES) \
-I$(F2PY_ROOT)/src -c warpmodule.c
$(CC) $(CC_FLAGS) $(PYTHON_INCLUDES) -I$(NUMPY_INCLUDES) -c \
$(F2PY_ROOT)/src/fortranobject.c -o fortranobject.o
# Compile the module wrappers if you have any
$(FF90) $(FF90_ALL_FLAGS) -I$(WARP_MODDIR) -c warp-f2pywrappers2.f90
# Do final linking with your MPI fortran compiler
$(FF90) -shared $(PYTHON_OBJECTS) $(LINKER_ALL_FLAGS) -o warp.so
The contents of get_f2py.py are:
#This is somewhat roundabout since if a .f2py_f2cmap exits, output is
printed to stdout
# before anything else happens and it is difficult to just pluck out the
directory we want.
# All we are doing redirecting stdout while we import numpy so we don't see
the extra
# output. There is probably a cleaner way to doing this, but I found this
works.
import os,sys
tmp = sys.stdout
sys.stdout = sys.stderr
import numpy.f2py
sys.stdout = tmp
print os.path.dirname(os.path.abspath(numpy.f2py.__file__))
This basically mimics what f2py does automatically. It is most likely not
as robust as using f2py directly, but you have more control on what
happens. That being said, this procedure has been tested on Linux, OSX and
several computing clusters and has proven sufficient for our (academic)
needs. Also, it may be important to do the final linking with your Fortran
compiler, not the C compiler as I believe is done with f2py. I believe this
has given us some issues in the past.
Pearu, feel free to comment on this procedure.
Hope that helps,
Gaetan
On Fri, Nov 9, 2012 at 5:00 AM, <f2py-users-request-Y4l6ocDipWCuvFJfX82//[email protected]> wrote:
> Send f2py-users mailing list submissions to
> f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://cens.ioc.ee/mailman/listinfo/f2py-users
> or, via email, send a message with subject or body 'help' to
> f2py-users-request-Y4l6ocDipWCuvFJfX82//[email protected]
>
> You can reach the person managing the list at
> f2py-users-owner-Y4l6ocDipWCuvFJfX82//[email protected]
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of f2py-users digest..."
>
>
> Today's Topics:
>
> 1. Re: Has ANYBODY gotten f2py working with Python 2.6 on Mac OS
> 10.6? (QuantCoder)
> 2. (no subject) (Razibul Islam)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 8 Nov 2012 19:28:03 +0000 (UTC)
> From: QuantCoder <[email protected]>
> Subject: Re: [f2py] Has ANYBODY gotten f2py working with Python 2.6 on
> Mac OS 10.6?
> To: f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]
> Message-ID: <[email protected]>
> Content-Type: text/plain; charset=us-ascii
>
> Raymond P. <rtp1@...> writes:
>
> >
> > Dear Trond,
> > Thanks for the detailed, if somewhat intimidating,
>
>
> This is an old post, but in case of any interest, I found the following
> information very helpful:
> http://cens.ioc.ee/pipermail/f2py-users/2009-August/001873.html
>
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 8 Nov 2012 12:05:38 -0800 (PST)
> From: Razibul Islam <tamal105-/[email protected]>
> Subject: [f2py] (no subject)
> To: "f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]" <f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]>
> Message-ID:
> <1352405138.94810.YahooMailNeo-+7zY+oFt7UweBhY5O9xny5EhsgyP+Z75VpNB7YpNyf8@public.gmane.org>
> Content-Type: text/plain; charset="us-ascii"
>
>
>
> Hello
>
> Can anybody use the f2py with OpenMPI fortran code?It would be very
> helpful if someone share their knowledge in this regard.
>
> Best Regards
>
> Tamal
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://cens.ioc.ee/pipermail/f2py-users/attachments/20121108/a6ff7411/attachment.html
>
> ------------------------------
>
> _______________________________________________
> f2py-users mailing list
> f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]
> http://cens.ioc.ee/mailman/listinfo/f2py-users
>
>
> End of f2py-users Digest, Vol 88, Issue 4
> *****************************************
>
>
--e89a8f502fc03902ca04ce0ff423
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Hello everyone<div><br></div><div>We have wrapped a number of mpi codes wit=
h f2py and have found a slightly different way of compiling things =A0was m=
uch easier for us.=A0</div><div><br></div><div>The basic idea is to only us=
e f2py to generate the wrapper .c file and if you have modules or common bl=
ocks a .f and a .f90 file. You then proceed to compile them yourself manual=
ly, and with your desired (mpi) compiler. =A0In the example I have below, I=
have a pre-computed and static .pyf file with a module called "warp&q=
uot; defined in warp.pyf</div>
<div><div><br></div><div><div>PYTHON_OBJECTS =3D fortranobject.o \</div><di=
v>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0warpmodule.o \</div><div>=A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0warp-f2pywrappers2.o \</div></div><div><br></div><di=
v># Generate Python include directory =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0</div>
<div>=A0 =A0 =A0 =A0 =A0$(eval PYTHON_INCLUDES =3D $(shell python-config --=
includes))</div><div><br></div><div># Generate Numpy include directory =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0</div>
<div>=A0 =A0 =A0 =A0 $(eval NUMPY_INCLUDES =3D $(shell python -c 'impor=
t numpy; print numpy.get_include()'))</div><div><br></div><div># Genera=
te f2py root directory =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0<=
/div>
<div>=A0 =A0 =A0 =A0 $(eval F2PY_ROOT =3D $(shell python get_f2py.py))</div=
><div><br></div><div># Run f2py to generate required files and compile</div=
><div>=A0 =A0 =A0 =A0 f2py warp.pyf</div><div><br></div><div># Start compil=
ing the required files:</div>
<div>=A0=A0=A0 =A0 =A0 $(CC) $(CC_FLAGS) $(PYTHON_INCLUDES) -I$(NUMPY_INCLU=
DES) \</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre"> =
</span>-I$(F2PY_ROOT)/src -c warpmodule.c</div><div>=A0 =A0 =A0 =A0 $(CC) $=
(CC_FLAGS) $(PYTHON_INCLUDES) -I$(NUMPY_INCLUDES) -c \</div>
<div>=A0 =A0 =A0 =A0 $(F2PY_ROOT)/src/fortranobject.c -o fortranobject.o</d=
iv></div><div><br></div><div># Compile the module wrappers if you have any<=
/div><div><div>=A0 =A0 =A0 =A0 $(FF90) $(FF90_ALL_FLAGS) -I$(WARP_MODDIR) -=
c warp-f2pywrappers2.f90</div>
<div><br></div><div># Do final linking with your MPI fortran compiler</div>=
<div>=A0 =A0 =A0 =A0 $(FF90) -shared $(PYTHON_OBJECTS) $(LINKER_ALL_FLAGS) =
-o warp.so</div><div><br></div><div>The contents of get_f2py.py are:</div><=
div>
<div><br></div><div>#This is somewhat roundabout since if a .f2py_f2cmap ex=
its, output is printed to stdout=A0</div><div># before anything else happen=
s and it is difficult to just pluck out the directory we want.=A0</div><div=
>
# All we are doing redirecting stdout while we import numpy so we don't=
see the extra</div><div># output. There is probably a cleaner way to doing=
this, but I found this works.</div><div>import os,sys</div><div>tmp =3D sy=
s.stdout</div>
<div>sys.stdout =3D sys.stderr</div><div>import numpy.f2py</div><div>sys.st=
dout =3D tmp</div><div>print os.path.dirname(os.path.abspath(numpy.f2py.__f=
ile__))</div><div><br></div><div>This basically mimics what f2py does autom=
atically. It is most likely not as robust as using f2py directly, but you h=
ave more control on what happens. That being said, this procedure has been =
tested on Linux, OSX and several computing clusters and has proven sufficie=
nt for our (academic) needs. Also, it may be important to do the final link=
ing with your Fortran compiler, not the C compiler as I believe is done wit=
h f2py. I believe this has given us some issues in the past.=A0</div>
<div><br></div><div>Pearu, feel free to comment on this procedure.</div><di=
v><br></div><div>Hope that helps,</div><div>Gaetan</div><div><br></div><div=
class=3D"gmail_quote">On Fri, Nov 9, 2012 at 5:00 AM, <span dir=3D"ltr">&=
lt;<a href=3D"mailto:f2py-users-request-Y4l6ocDipWCuvFJfX82//[email protected]" target=3D"_blank">f2py=
-users-request-Y4l6ocDipWCuvFJfX82//[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Send f2py-users mailing list submissions to<=
br>
=A0 =A0 =A0 =A0 <a href=3D"mailto:f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]">[email protected]=
oc.ee</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
=A0 =A0 =A0 =A0 <a href=3D"http://cens.ioc.ee/mailman/listinfo/f2py-users" =
target=3D"_blank">http://cens.ioc.ee/mailman/listinfo/f2py-users</a><br>
or, via email, send a message with subject or body 'help' to<br>
=A0 =A0 =A0 =A0 <a href=3D"mailto:f2py-users-request-Y4l6ocDipWCuvFJfX82//[email protected]">f2py-user=
s-request-Y4l6ocDipWCuvFJfX82//[email protected]</a><br>
<br>
You can reach the person managing the list at<br>
=A0 =A0 =A0 =A0 <a href=3D"mailto:f2py-users-owner-Y4l6ocDipWCuvFJfX82//[email protected]">f2py-users-=
owner-Y4l6ocDipWCuvFJfX82//[email protected]</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of f2py-users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
=A0 =A01. Re: Has ANYBODY gotten f2py working with Python 2.6 on Mac =A0 =
=A0 =A0 =A0OS<br>
=A0 =A0 =A0 10.6? (QuantCoder)<br>
=A0 =A02. (no subject) (Razibul Islam)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Thu, 8 Nov 2012 19:28:03 +0000 (UTC)<br>
From: QuantCoder <<a href=3D"mailto:[email protected]">liguokong@gmail=
.com</a>><br>
Subject: Re: [f2py] Has ANYBODY gotten f2py working with Python 2.6 on<br>
=A0 =A0 =A0 =A0 Mac =A0 =A0 OS 10.6?<br>
To: <a href=3D"mailto:f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]">f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]</a><br=
>
Message-ID: <<a href=3D"mailto:[email protected]">=
[email protected]</a>><br>
Content-Type: text/plain; charset=3Dus-ascii<br>
<br>
Raymond P. <rtp1@...> writes:<br>
<br>
><br>
> Dear Trond,<br>
> Thanks for the detailed, if somewhat intimidating,<br>
<br>
<br>
This is an old post, but in case of any interest, I found the following<br>
=A0information very helpful:<br>
<a href=3D"http://cens.ioc.ee/pipermail/f2py-users/2009-August/001873.html"=
target=3D"_blank">http://cens.ioc.ee/pipermail/f2py-users/2009-August/0018=
73.html</a><br>
<br>
<br>
<br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Thu, 8 Nov 2012 12:05:38 -0800 (PST)<br>
From: Razibul Islam <<a href=3D"mailto:tamal105-/[email protected]">tamal105@yaho=
o.com</a>><br>
Subject: [f2py] (no subject)<br>
To: "<a href=3D"mailto:f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]">f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]<=
/a>" <<a href=3D"mailto:f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]">[email protected]=
.ee</a>><br>
Message-ID:<br>
=A0 =A0 =A0 =A0 <<a href=3D"mailto:1352405138.94810.YahooMailNeo@web1411=
03.mail.bf1.yahoo.com">1352405138.94810.YahooMailNeo-+7zY+oFt7UweBhY5O9xny26OjHEfMyxF@public.gmane.org=
oo.com</a>><br>
Content-Type: text/plain; charset=3D"us-ascii"<br>
<br>
<br>
<br>
Hello<br>
<br>
Can anybody use the f2py with OpenMPI fortran code?It would be very helpful=
if someone share their knowledge in this regard.<br>
<br>
Best Regards<br>
<br>
Tamal<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href=3D"http://cens.ioc.ee/pipermail/f2py-users/attachments/2012110=
8/a6ff7411/attachment.html" target=3D"_blank">http://cens.ioc.ee/pipermail/=
f2py-users/attachments/20121108/a6ff7411/attachment.html</a><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
f2py-users mailing list<br>
<a href=3D"mailto:f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]">f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]</a><br>
<a href=3D"http://cens.ioc.ee/mailman/listinfo/f2py-users" target=3D"_blank=
">http://cens.ioc.ee/mailman/listinfo/f2py-users</a><br>
<br>
<br>
End of f2py-users Digest, Vol 88, Issue 4<br>
*****************************************<br>
<br>
</blockquote></div><br></div></div>
--e89a8f502fc03902ca04ce0ff423--
--===============3195602987682986302==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
f2py-users mailing list
f2py-users-Y4l6ocDipWCuvFJfX82//[email protected]
http://cens.ioc.ee/mailman/listinfo/f2py-users
--===============3195602987682986302==--