Re: C++ to Python and returning members of a C++ object

Hi William,

Thanks for the information. In the case of the class library I'm 
wrapping that would end up with an awful amount of code in SWIG to 
handle every case where this occurs. Most, if not all of the 
classes/structs I am encapsulating use other classes/structs as members 
down multiple levels and changes would be needed to this code for every 
library update. It might end up it would be easier not to use SWIG :)

Yes, memory management, pass by value, reference/pointer etc is tough, 
but I am surprised that SWIG doesn't handle this, at least for Python, 
with a user configurable "feature" or all of the time. From what I can 
see for C++ -> Python at least it would be possible to add the code, as 
given in the references you give, by default whenever a reference to a 
C++ object members data is being returned as a pointer. I think it would 
be a generally the case that if you return a reference to a C++ objects 
member you would want the overall C++ object to hang around while that 
pointer is in existence.

Yes, this would reduce efficiency, but if instead of the methods 
suggested a simpler and more efficient Py_INCREF() and Py_DECREF() 
mechanism on the PyObject* to the parent object could be  used the 
overhead should be pretty small. I have tried a simple implementation of 
this by modifying SWIG to do the Py_INCREF() and it appears to work well 
(although it is a hack as I don't understand the SWIG internals well 
enough) and I don't know how to call Py_DECREF() efficiently so I am not 
doing this. I am assuming it would be possible to modify the SWIG C++ 
wrapper object to have the extra container objects pointer so its 
reference count can be decremented but I need to try and understand the 
SWIG code better.

If you think something like this might be feasible and useful, I will 
have a look at the SWIG code a bit more in my limited spare time.

Terry
On 28/06/2020 13:32, William S Fulton wrote:
> Hi Terry
>
> This is one of the most horrible problem areas mixing languages, that 
> is, memory management. Here is some background reading:
>
> http://www.swig.org/Doc4.0/Python.html#Python_memory_management_member_variables
> https://github.com/swig/swig/issues/945
> https://github.com/swig/swig/pull/1234
>
> Below is one way you can fix this (it's a variation of returning 
> members by reference documented in the 1st link above). Note that SWIG 
> uses pointers for accessing non-primitive member variables, see 
> http://www.swig.org/Doc4.0/SWIG.html#SWIG_structure_data_members.
>
> %fragment("member_parent_attribute_init", "init") {
>   // thread safe initialization
>   member_parent_attribute();
> }
>
> %fragment("member_parent_attribute", "header", 
> fragment="member_parent_attribute_init") {
>   static PyObject* member_parent_attribute() {
>     static PyObject* attr = SWIG_Python_str_FromChar("__member_parent");
>     return attr;
>   }
> }
> %typemap(ret, fragment="member_parent_attribute", noblock=1) ListA 
> *TypeB::list1 %{
>   PyObject_SetAttr($result, member_parent_attribute(), $self);
> %}
>
> William
>
> On Thu, 25 Jun 2020 at 14:58, Terry Barnaby <[email protected] 
> <mailto:[email protected]>> wrote:
>
>     I have had a quick look at the SWIG code.
>
>     It seems that it would be possible, and desirable to do a
>     Py_INCREF() on
>     the PyObject* object that is wrapping the C++ object which you are
>     returning a pointer to the member from. This would hold the whole
>     'C++'
>     object in memory so that returning a pointer to one of this objects
>     members would be ok. The PyObject* returned for the member would
>     have to
>     have a pointer to its parents PyObject* and do a Py_DECREF() on the
>     parent object when it is finally deleted so that the original C++
>     object
>     gets cleaned up eventually.
>
>     I don't know the SWIG code to know if this is really possible and if
>     people think this is a reasonable SWIG feature that could be added ?
>     At the moment a user of your SWIG generated Python API gets a
>     pointer to
>     a destroyed object which is not good.
>
>     Terry
>     On 24/06/2020 10:04, Terry Barnaby wrote:
>     > I have a class library that has List objects as members of the
>     class.
>     > If I use this class in a Python function and try to return one of
>     > these more complex members all hell breaks loose. It appears the
>     SWIG
>     > generated wrapper returns a pointer to the objects member rather
>     than
>     > making a copy of it when returning the member from the function.
>     >
>     > So I have something like:
>     >
>     > class TypeB {
>     >
>     > public:
>     >
>     >     List<TypeA> list1;
>     >
>     > };
>     >
>     > If in Python I have a function like the following, the returned
>     item
>     > is effectively (SWIG wrapped) a pointer the b objects member and b
>     > disappears when the function returns.
>     >
>     > def func1():
>     >
>     >     b = TypeB();
>     >
>     >     b.list1.append(TypeA());
>     >
>     >     return b.list1;
>     >
>     > Now I can use "return ListTypeA(b.list1)", where ListTypeA has been
>     > declared using a %template declaration such that the copy
>     constructor
>     > is called to explicitly make a copy of the list, but is there a
>     better
>     > way of handing this ?
>     >
>     > I know this is difficult to handle, but perhaps something like a
>     > %template(return) to add code when returning a type in a Python
>     > function somehow or maybe the overall objects reference counter
>     can be
>     > incremented ?
>     >
>     >
>     >
>     > _______________________________________________
>     > Swig-user mailing list
>     > [email protected]
>     <mailto:[email protected]>
>     > https://lists.sourceforge.net/lists/listinfo/swig-user
>
>
>
>
>     _______________________________________________
>     Swig-user mailing list
>     [email protected]
>     <mailto:[email protected]>
>     https://lists.sourceforge.net/lists/listinfo/swig-user
>

_______________________________________________
Swig-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/swig-user