Re: [Implementation] Calling wrapped functions, converters, policies

[Aleksey, please see below w.r.t. lambda exprs]

Daniel Wallin <[email protected]> writes:

> David Abrahams wrote:
>> Daniel Wallin <[email protected]> writes:
>> 
>>>>>IIRC this is close to what you have in boost python today.
>>>
>>>>Yep, close.  It's missing the "hidden this downcasting" feature,
>>>>which we'll need.
>>>
>>>Right,
>>>
>>>   template<class R, class T, ..., class WrappedClass>
>>>   signature<
>>>       mpl::vector3<WrappedClass&, R, ...>
>> What does the first element of the vector mean?  Oh, looking back at
>> your previous message that's a *very* strange ordering!  Why would you
>> put the return type *second*?!
>
> The ordering was arbitrary. Well, sort of at least. The reason was to
> keep the indexing consistent. A member function signature is the same as
> a function signature, except it has the self-type at index 0. So the
> index to some type in the signature is just offset one step. I don't
> know if it matters though. :)

Errf.  I think of these two as equivalent:

       A f(B*, C, D, E)
       A B::f(C, D, E)

And they *are* equivalent as far as Boost.Python is concerned, except
at the last second when the C++ function is invoked and we have to do
some syntactic adjustment to account for the difference between
member and non-member functions.  I don't see any reason to drag the
B* to the beginning of the signature.

>>>       ...
>>>   >
>>>   signature_of(R(T::*)(...), type<WrappedClass>)
>>>
>>>?
>> I'm not sure whether what you're suggesting above actually does the
>> right thing.  In fact, I'm almost certain it doesn't.  You need the
>> most_derived hook used in signature.hpp.  The idea is, "downcast if
>> you can, otherwise leave it alone".
>
> I had a look in signature.hpp. I'm not sure I understand this though;
> Target is not more derived than T => Target == T, no? Or is Target
> something else than the current WrappedClass type?

Well, this should probably be done for non-member functions as well,
but...

suppose someone does this:

        struct Base  // not wrapped
        {
            void f();
        };

        struct Baz // wrapped
        {
            void g();
        };

        struct Derived : Base // wrapped
        {};


   class_<Derived>("Derived")
       .def("f", &Base::f)
       .def("g", &Baz::g)
       .staticmethod("g")
       ;

   >>> d = Derived()
   >>> d.f()
   >>> d.g(Baz())

That should show why the most_derived thing is there.

>> I have no problem in principle (I think) with bidirectional
>> converter *generators*, but bidirectional converters won't work.  They
>> need completely different data members, and from-XXX converters need
>> to be 2-phase for overloading, while to-XXX converters are one
>> phase.  In other words, I have no problem with:
>> [snip]
>
> OK agreed. So, should the nested member be a metafunction class?
>
>    some_generator
>    {
>        typedef /* to converter */ to;
>        typedef /* from converter */ from;
>    };

Well, it can be more efficient than using any generalized lambda expr,
but it's also more work for the user.  AFAIK MPL is going to change so
that apply<...> uses lambda<...> internally, but I guess there would
always be a lower-level apply_metafunction_class<...> for efficiency.
Even though we've written about that in our book, it's not implemented
yet.  Aleksey, care to comment?

> And how does this fit in with how the converter_tuple is generated?
> Shouldn't all converters be in the tuple, regardless of in what
> direction they should convert? 

Why not?

> How could the converter_tuple_generator know which types should be
> "to" and which should be "from"? 

If you use a consistent signature ordering as I'm suggesting, it's
easy.  To for the first and from for the rest.

> My version just iterates over all types and generates a converter
> for every one, not knowing anything about directions.
>
>>>To clarify, this models what we have in luabind.
>>>
>>>   policy1(_2) + policy2(_1)
>>>
>>>Would create something like:
>>>
>>>   composite_call_policies<
>>>       mpl::vector<
>>>           node<mpl::int_<2>, policy1_type>
>>>         , node<mpl::int_<1>, policy1_type>
>> --------------------------------------^
>> 
>>>       >
>>>    >
>> Really??  
>
> No, of course I mistyped. Should have choose better names. :)
>
>> Just so as not to confuse things, I suggest policyA and
>> policyB.  So:
>>      policyA(_2) + policyB(_1)
>>   Would create something like:
>>      composite_call_policies<
>>          mpl::vector<
>>              node<mpl::int_<2>, policyA_type>
>>            , node<mpl::int_<1>, policyB_type>
>>          >
>>       >
>> ??
>
> Exactly.
>
>>>And it's nested apply<> would do something like:
>>>
>>>   apply(Type, Index)
>>>      if we have a child, C,  that matches node<Index, _>
>>>          C::apply(Type, Index)
>>>       else
>>>          default_call_policies::apply(Type, Index)
>> It looks to me like the Children model a different concept from the
>> composite, in that they're index-agnostic (or should be).  The
>> children are converter generators, aren't they?  
>
> No I think the children are CallPolicies as well - they should have
> precall/postcall actions.
>
> I'm not sure what you mean by index-agnostic.

I think I'm just forgetting stuff you already convinced me of, but
what I meant was that the others are only really concerned with
converting a single argument and so don't need an index, while the
CallPolicies are concerned with the whole argument tuple.  But I
think we agreed they all need the index.

>> Didn't we make a bunch of these decisions about naming already?
>
> Yeah we did. I think we said that there are basically two concepts,
> ConverterGenerator and CallPolicies. CallPolicies is a refined
> ConverterGenerator which has precall/postcall actions. I'm going to
> browse through the archives after I post this, if I was wrong I'll
> repost. :)
>
>>>The indices here are generally non-contiguous, since the user rarely
>>>would specify an explicit policy for each argument index.
>> Why not use an mpl::map or mpl::multimap?
>
> That's fine by me.
>
>> Yes, you or Aleksey will have to implement it.  But I'm sure he'd
>> be happy to oblige.   ;->
>
> Didn't you already implement an associative mpl container? IIRC I have
> seen you talk about it in some posts.

I implemented the prototype, and Aleksey has already checked in
set/multiset based on my prototype.

See enclosed prototype


> I did it once for fun, except it wasn't really a map but sort of a
> container adaptor.
>
>     // BaseContainer := mpl sequence of mpl::pair<Key, Value>
>     template<class BaseContainer>
>     struct map_adaptor;
>
> That was pretty far from mpl standard though, so lets wait for Aleksey. :)

OK.

>>>>>    It also combines it's childrens argument_package types and
>>>>>    state types by inheritance.
>>>>
>>>>Not sure what you mean by "by inheritance".
>>>
>>>It means it's nested state_type inherits from all it's "childrens"
>>>state_types.
>> I guess states have to be classes then?  I think there's a way to
>> lift
>> this restriction using an explicit get function and inheritance from
>> wrappers:
>>      template <class Head, class Tail>
>>      struct cons : Tail
>>      {
>>          Head h;
>>      };
>>      template <class Head, class Tail>
>>      Head& get(cons<Head,Tail>& x) { return x.h; }
>> Now each state type (the "head" of a cons sub-list) can be whatever
>> you like.
>
> And extract the state type by explicitly supplying Head?
>
>    get<my_state_type>(state);
>
> It isn't as nice as implicit conversion, but I guess it'll work. 

I think it's more principled.

> I don't know though; do states really need to be non-class types?
> Sure, I can see why someone could find the need for it, but if you
> don't wrap your type in a class, there's always a risk for
> collisions with other policies in a composite. Two policies can use
> "int" for completely different things, and we can't discriminate
> them.

I guess.  OK, no need to generalize prematurely..

>>>Again,
>>>
>>>   composite_call_policies<
>>>       mpl::vector<
>>>           node<mpl::int_<2>, policy1_type>
>>>         , node<mpl::int_<1>, policy1_type>
>>>       >
>>>    >
>>>
>>>Would make something like:
>>>
>>>    template<class Base, class CallPolicies>
>>>    struct inherit_state : Base, CallPolicies
>>>    {
>>>        template<class U>
>>>        inherit_state(const U& init)
>>>            : Base(init), CallPolicies(init)
>>>        {}
>>>    };
>>>
>>>    typedef typename inherit_linearly<
>>>        PoliciesSequence
>>>      , empty
>>>      , inherit_state<_1, _2>
>>>    >::type
>>>
>>>Except that it's a bit more complicated; every state type is only
>>>inherited once.
>> mpl::set, anyone?
>
> Sure. Now I'm doing something like:
>
>    template<bool B, class Base, class CallPolicies>
>    struct inherit_state_aux : Base, CallPolicies::state_type
>    {};
>
>    template<class Base, class CallPolicies>
>    struct inherit_state_aux<true, Base, CallPolicies>
>      : Base
>    {};
>
>    template<class Base, class CallPolicies>
>    struct inherit_state
>       : inherit_state_aux<
>             is_convertible<
>                 Base*, typename CallPolicies::state_type>::value
>             >::value
>           , Base
>           , CallPolicies
>         >
>    {
>       ...
>    };
>
>    typedef typename inherit_linearly<
>          PoliciesSequence
>        , empty
>        , inherit_state<_1, _2>
>    >::type
>
> Except I'm not using partial specialization..

Cute, and reasonably efficient.  Only the generated type is too
complicated in the case where the same state type is re-used.  Should
be:

   typedef typename mpl::fold
       PoliciesSequence
     , empty
     , mpl::if_<
           is_convertible<add_pointer<_2>,add_pointer<_1> >
         , _1                     // here
         , mpl::if_<
               is_same<_1,empty>
             , _2                 // eliminate empty (?)
             , mpl::inherit<_1,_2>
           >
       >
   >::type


> Whether or not we are using mpl::set or this doesn't matter to
> me. My guess is that my version would compile faster, but obviously
> is less clear. I could be wrong though.

You're probably right.

>>>The same goes for argument_package. Here we also have get() functions:
>>>
>>>    template<class Base, int N>
>>>    PyObject* get(PyObject* args, mpl::int_<N>)
>>>    { return PySomethingSomething(.., N); }
>>>
>>>    template<int N>
>>>    struct offset_arg // IIRC you have something like this in BPL
>>>    {
>>>        offset_arg(PyObject*) {}
>>>    };
>> Yep, in boost/python/make_constructor.hpp
>> 
>>>With a special overload that handles the composite case where every
>>>argument_package has a base type which must be queried:
>>>
>>>    template<class Base, int M, int N>
>>>    PyObject* get(offset_arg<M>, const Base& base, mpl::int_<N>)
>>>    { return get(base, mpl::int_<N + M>()); }
>> Wha??
>> Why isn't the base object stored in the offset_arg<>, as in BPL?  Do
>> you need to explain what Base is?
>
> Because the different CallPolicies in the composite_call_policies type
> have no knowledge of each other. Meaning we would need to make the
> nested argument_package a metafunction:
>
>    struct my_policy
>    {
>        template<class Base>
>        struct argument_package
>        {
>            typedef offset_arg<1, Base> type;
>        };
>    };
>
> I think it's nicer to create a composite hierarchy:
>
>    composite_argument_package<
>        offset_arg<1>
>      , composite_argument_package<
>            offset_arg<9>
>          , composite_argument_package_native<
>                PyObject*
>            >
>        >
>    >
>
>    template<class Pkg, class Base>
>    struct composite_argument_package : Base
>    {
>        // some forwarding constructors goes here
>
>        Pkg pkg_;
>    };
>
>    template<class Pkg, class Base, int N>
>    PyObject* get(
>        const composite_argument_package<Pkg, Base>& x, mpl::int_<N>)
>    {
>        // forward to correct get() function
>        return get(x.pkg_, static_cast<const Base&>(x), mpl::int_<N>());
>    }
>
> See what I'm driving at?

Not immediately.  I"ll have to give it some thought.

> BTW, if you implement the composite CallPolicies like you have in BPL
> there is no reason why offset_arg couldn't store it's Base. That's of
> course a perfectly fine model of CallPolicies as well.
> composite_call_policies is just an example of how it can be done without
> explicit inheritance from your "Base policy".

I think you can also get there by having the names that the user
handles

    some_policyA(_1)
    ^^^^^^^^^^^^

refer to instances of some concept other than CallPolicies which can
generate CallPolicies.  Not sure what's best yet.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com