Re: cytpes and python C API combinations

Hi Thomas,

>
>From: Thomas Stover [mailto:[email protected]] 
>Sent: Wednesday, January 13, 2010 0:08 AM
>
>> From: Mark McMahon <[email protected]>
>
>> For example...
>> - Application asks the plugin to do something (passing various bits of
>> information, one or more of which are pointers to C stuff)
>> - The part of my plugin receives this and calls the appropriate Python
>> with same parameters (Pyrex code for me - but could be done with Python
>C
>> API, most pointers are treated as longs)
>> - The python code either:
>>     * knows how to deal with the pointer (e.g. a structure it knows how
>to
>>     deal with) and does a "from_address" (ctypes)
>>     * or doesn't need to know the internals of the pointer (usually a
>>     pointer to a C++ interface) and just stores it for later
>> - When the python code needs to request something from that interface it
>> calls (ctypes) a method in the plugin DLL which accepts the pointer +
>other
>> parameters (the function it calls is usually one line, which casts the
>> pointer back to the right C++ type and calls a method of that object
>with
>> the other parameters)
>> 
>
>That is actually very similar to what I'm trying to do. Instead of a
>plugin to a C++ application, in my case it is plugin / runtime dynamically
>loaded module to a C library. I'm creating a shim between the C plugin API
>and python so that one will be able to write a module for this library in
>python. These modules support being handed arbitrary pointers to stuff from
>the C side of things, and in some cases they may be relevant to a module
>written in python. Just as in your case, sometimes they might need to get
>passed to something else in the C world, or utilized directly in the python
>module. The front side of the library these modules are for, itself has a
>ctypes based wrapper. 
>
>After some more reading, I discovered that the python C api has a facility
>known as the PyCObject
>(http://docs.python.org/extending/extending.html#providing-a-c-api-for-an-extension-module).
>This is a regular python object for the intended use of passing opaq
>arbitrary pointers between python modules from python code. It can only be
>created and inspected on the C side. On the python side only passed around.
>

I never had to worry about that directly - though it is possible that pyrex 
did that for me :) Though I am not really sure you need it, it sounds 
like that is mainly for Python objects, and from your description
it sounds more like it is normal C objects you are passing around.

>
>This is similar to, and almost the mirror opposite of, the CTypes object
>c_void_p. A c_void_p is a python object holding an opaq arbitrary pointer
>that (apparently) can only be created and inspected on the python side. On
>the C side it can only be passed around (all python objects can at least be
>passed around by the python API; afaik there is no CTypes C API for further
>manipulation of ctypes objects).

If you want there is - The C language itself :). It was made for dealing
With C Types ;)

>
>If there was a sane way of converting between the those two object types
>in both C and in Python, then all combinations would be possible at all
>times. It sounds like you solved part of that puzzle by using c_longs to
>hold pointers, but in a 64 bit land c_longlong would have to be used
>creating another portability issue. It may turn out that this the only way.
>

size_t might be the type you want.
http://docs.python.org/library/ctypes.html#ctypes.c_size_t)
from somewhere on the web...
size_t type is a base unsigned integer type of C/C++ language. It is the result of sizeof operator's execution. The type's size is chosen so that it could store the maximum size of a theoretically possible array. On a 32-bit system size_t will take 32 bits, on a 64-bit one 64 bits. In other words, a variable of size_t type can safely store a pointer. The exception is pointers to class functions but this is a special case. size_t type is usually used for loop counters, array indexing and address arithmetic.


>I'd like to work out a cheat sheet for all the various possibilities. That
>might get started today after some more tests. As python and C move towards
>world domination, this will keep coming up.
>
>(off topic, I was under the impression that plugins/dynamically loaded
>modules in C++ was somewhat infeasible - at least portably - because of the
>whole name mangling issue. Was this with COM or something? Or maybe this
>was a "plugged in" at build time?)
>

That was why I wrapped the C++ library in some Automatically generated C
Code first (to convert the C++'ness of it to C).

>
>-- 
>www.thomasstover.com
>

Good luck,
  Mark



------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev