Re: [C++-sig] [Implementation] Calling wrapped functions, converters, policies

I'm responding to this in the context of boost.langbinding.

At 21:15 2003-09-16, David Abrahams wrote:

>I was recently trying to implement
>http://boost-consulting.com/boost/libs/python/todo.html#injected-constructors
>and ran into some issues with the way Boost.Python calls wrapped
>functions.  As I began to explore boost/python/detail/caller.hpp and
>boost/python/detail/invoke.hpp, I realized that the things I wanted to
>do were very closely tied to some work Lijun Qin has been doing
>(http://aspn.activestate.com/ASPN/Mail/Message/C++-sig/1771145), and
>with the desired ability to allow converters to execute post-call
>processing actions.
>
>Since the impact of these change has the potential to be quite
>sweeping, I thought we should discuss the broad requirements here
>before I start implementing anything.
>
>Here are some things I think we need:
>
>1. Per-call state.
>
><snip>
>    What's needed is a way to get the per-call state onto the program
>    stack.  I can think of two main approaches:
>
>    a. A copy of the Policies object is made on the stack and used
>       during the function call; the copy can maintain its own state.
>       An advantage of this approach is simplicity.  A disadvantage is
>       that you may for things you don't use: storage for per-call
>       state in the wrapped function itself, and cycles for copying
>       per-call state to the stack.
>
>    b. The policies object has a init_state() function which
>       produces a new state object of a possibly-different type.  This
>       doesn't have the disadvantages above, but costs complexity in
>       several ways:
>
>       i. The requirements on the Policies class become more
>          complicated.  It probably needs to have a nested ::state
>          type.  It's possible to get around the need for this typedef
>          by passing the results of init_state() directly to a function
>          template parameter (e.g. into invoke(...)), but that may
>          force us to have a function call boundary at an undesirable
>          place.
>
>       ii. Policies composition may become much more complicated.  How
>           do you come up with the state object corresponding to
>           several composed policies?  You could use tuples... hmm,
>           maybe this is a job for mpl::inherit_linearly.
>
>    I'm leaning towards a. but I'm really not sure which one is best
>    and would appreciate comments.

I think b seems far better. Actually, we were going to implement something
like this in luabind as well, but never got around to it. The idea there was
that the policy would instantiate the converter object, so that the converter
could get data stored in the policy into it's state.

on (ii), one state per policy?

>2. The ability to select a specialized from-Python conversion method
>    for each argument.  Right now, we can only select the to-Python
>    conversion method for the result, but Luabind has shown the wisdom
>    of being able to do the converse, and in fact Mike Rovner was just
>    asking how we could implement Luabind's "adopt" policy for
>    stealing ownership of C++ objects held by auto_ptr.
>
>    Right now, an MPL argument signature sequence gets passed to
>    caller_arity<N>::impl, and the elements of that (except the first,
>    which is the return type) fully determine the _static_ type (see
>    below) of the from-Python argument converters.  We need a way to
>    customize the type of the argument converters which are actually
>    used for each argument.  I think Luabind has a solution for this.
>    I can imagine several similar approaches, so I don't think this is
>    hard.

Right, no problem.

>3. Dynamic converter per-call state and postcall actions.
>
><snip>
>    I propose that whichever state model is chosen in item 1 above, the
>    state contains a chain of dynamically-allocated polymorphic
>    postcall objects, and that dynamic converter implementations are
>    passed a reference to that chain so that they can register new
>    postcall actions.  The postcall actions are invoked when the call
>    completes successfully, and are unconditionally deleted at the end
>    of the call.  One thing I'm still not clear about is whether a
>    converter's convertible() function need access to that chain.

I agree. If we see the policies just as "generators", which generates
both converters and converter-context-states (or whatever they would
be called) this fits in nicely without needing to make a special case
of it:

struct ContextConcept
{
    ...
    something postcall(something_else);
};

struct ConverterConcept
{
    ...
    T convert(PolicyConcept::context_type& context);
};

struct PolicyConcept
{
    typedef /* ... */ context_type;

    template<class T>
    struct apply
    {
       typedef /* ... */ type;
    };

    template<class T>
    typename apply<T>::type make_converter(...);

    context_type make_context(...);
};

The default policy could just generate a state type which holds the
post-call chain, and call that chain in it's own postcall() function.

Is this making sense? I haven't thought this through enough yet, but
posting it anyway..

>4. The ability to release the Python interpreter lock during the
>    wrapped call based on the choice of Policies.  If releasing the
>    lock becomes the default behavior, it's important that it be
>    automatically disabled when the wrapped function is handling
>    python::object or any of its derived classes.  The release must
>    happen inside the invoke(...)  function, or at least, if the
>    implementation changes, after all converters have completed their
>    work, since the lock must be held while any reference counts are
>    changed.
>
>    This suggests that there is an inner layer of action needed, and it
>    should probably be generalized.  So the flow looks something like:
>
><snip>
>     I'm not sure how this interacts with Daniel's recursive "best
>     overload" resolution.

Whatever the flow is, it's easy enough to adapt to fit into the recursive 
invoke
code, I think.

>Comments?

I'll think more and comment more tomorrow. :)

---
Daniel Wallin



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf