Re: Conversion system

"Marty Poulin" <[email protected]> writes:

> Hello David,
>
> Sorry for any confusion I may have inflicted.  I am a newby to both
> python and Lua and my vocabulary tends to get a bit muddled at
> times.

That's OK.  It gets especially difficult to be clear whenever there
are two or more languages involved.  Between Python, Lua, and C++,
you've got a real challenge in front of you.

That said, you don't seem to directly answer any of the questions I
asked in my previous posting; instead you "try to explain".  Those
questions were designed to elicit answers which would help me
understand, but your explanations are just leaving me feeling vague.

>> > Speaking as someone who will probably end up using most of the
>> > features you implement I would like to reinforce the need for
>> > namespaces
>>
>> C++ or Python namespaces?  And, in either case, specifically what did
>> you have in mind?
>
> In one part of our project we have a communication layer that has been
> designed in its own namespace 

C++ namespace?  Python has its own notion of "namespace", you know.

> and to require a nested namespace per module.

Python module?  Do you mean that for every Python module you want a
submodule?  Tiny code examples help.

> This way each module limits its interaction with other modules and allows
> programmers working on each module to work independently.  

Sure.

> While it is certainly possible to enforce this sort of development
> without namespaces, and it is also possible to circumvent the extra
> namespaces with #using directives

That's not C++.  Do you mean "using directives", or something else?

> it does act a nice reminder and buffer for any programmer.

OK, but I still have almost no idea what capability you're asking for.
Tiny code examples can be a big help here.

FWIW, Boost.Python supports the development of separate, interacting
extension modules with wrapped C++ objects from one extension module
passed to functions in another extendion module, and that capability
will not go away.

>> > independent interpreters per thread
>>
>> Independent interpreters can't share data, can they? Why not just run
>> multiple processes in that case?
>
> I understand that Python does not support this

What, multiple interpreters?  It certainly does.

> but if I am not mistaken from an earlier post Lua does. " The
> LuaState is a complete interpreter state." "you create the state for
> each context you are calling in."

Once again, I ask:

     1. Independent interpreters can't share data, can they? 

     2. If not, why not just run multiple Python or Lua processes in
        that case?

> We do all our low level data sharing across threads in C++.  We are
> not looking for Python or Lua to be a glue between threads as much
> as a sandbox that each thread can utilize to allow production level
> programmers to interact with the system without worrying about the
> concepts of threads or mutex locking. Any object that we share
> between threads would not be exposed to the production level
> programmers.

Why do you think you want multiple interpreters, then?  I don't know
about Lua, but you can certainly do threading with Python without ever
using multiple interpreters.

>> > support for DLL's
>>
>> What kind of support, specifically, did you have in mind?
>
> I have been programming DLL's for years in C and took them for
> granted as a simple way to compartmentalize functionality that was
> shared across applications. With the addition of C++ into the code
> we are using for our shared libraries I found a host of problems
> that were not there with C.
>
> One that was always there in C was the allocation of memory within
> the DLL's heap and requiring that this memory be freed there as
> well. 

Use boost::shared_ptr<>.  It solves that problem handily.

> With the limited API what we have been dealing with up this point it
> was not a problem but as our needs expanded and the addition of C++
> the ability to ensure that objects were destroyed in the proper DLL
> became a problem.  The solution was fortunately provided quite
> simply by redefining new and delete for objects that are potentially
> allocated within the DLL's heap to always be allocated and freed
> there.

Ouch, that sounds unreliable and it means that you can't have the
same type of object allocated on either side of a DLL boundary.

Regardless, I don't see what any of this has to do with Boost.Python
or Luabind.  Could you explain the relationship?

> A new one that I had no experience with was fragile base
> classes. The ugly solution for this being to add a number of unused
> virtual functions and data as padding.

What do you mean by "fragile base classes"?

> The next is the problem is that RTTI is broken on some platforms
> (some versions of gcc don't support RTTI across shared objects) 

All the ones I've got experience with (gcc 2.95.x->3.3) do, provided
that the shared objects are linked together.

> and believe it not unavailable on a few others.  Our solution to
> this is to base our own RTTI on the exported address of a specific
> static instance located within a global registry in only one DLL.
>
> While I do not expect the langbinding project to be at a state where
> you are able to create a static API that can be exported from a DLL
> and not change with each release, I do have hope from what I have
> seen of your knowledge of DLL's that you will be taking into account
> most of what I have outlined and a few I don't know about yet as you
> go forward.

I can only account for what I can understand, and so far I don't
understand most of what you're describing.

>> > and fast object interactions.
>>
>> What specifically do you mean by "object interactions"?
>
> Also bad working on my part. boost python does a good job of
> wrapping C++ objects in a way that is extremely efficient while
> providing flexibility and features. I expect from what I have seen
> so far that as this new project progresses that the combined
> technology from both projects will provide a project that is at
> least as efficient and flexible as the current non-merged projects
> are now.

I certainly hope so.

>> ...and how fast is "fast"?  Part of the point of binding C++ to any of
>> these other languages is that you gain speed on the C++ side, so
>> worrying about the cost of crossing the language boundary too much
>> tends to be counterproductive.
>
> Agreed, to a point. The more efficient crossing that language
> barrier is the more applications we can come up with for the
> technology.  

Sure.

> We currently have many areas where embedded scripting could add a
> lot of flexibility, but some of those may be limited by how
> efficient the language and its interface to the underlying C++.  Am
> I worrying about that a lot now?  Not really, but if I am buying a
> car I do pay attention to what is under the hood, I tend to buy a V6
> or V8 before I buy a 4 cylinder, I may want to pass someone someday
> ;-)

Fair enough.  We'll "try to be good" <wink>.

>> > We have many developers working on the same project so the ability
>> > to limit interactions with namespaces is a definite help.
>>
>> What specific kinds of "interactions with namespaces" did you have in
>> mind?
>
> Hopefully I have finally explained that well enough above ;-)

Not yet, sorry.

>> Maybe you should be using a Java->C compiler instead; that sort of
>> thing is portable to all compilers, innit?
>
> Actually I don't really consider Java a good solution for our problems.  It
> just isn't as easy as Python ;-)

There's always Jython.

>> You probably have good reasons for using XSLT, but personally the idea
>> of writing programs *in XML* horrifies me a bit.  Consider this Python
>> metaprogram for doing XML transformations:
>>
>>     f = body[
>>         table(id="outer", width="100%", height="100%", border="0")[
>>             tr(valign="bottom")[
>>                 td(id="output", width="75%", valign="top", model="latestOutput")[
>>                     div(pattern="listItem", view="html")[
>>                         "Foo"
>>                     ]
>>                 ],
>>                 td(id="room-contents", valign="bottom")[
>>                     strong[
>>                         "Stuff you have"
>>                     ],
>>                     div(model="playerInventory", view="List")[
>>                         if_(not _1)[
>>                             div(_class="item")["Nothing"]
>>                         ].else_[
>>                             for_each(_1)[
>>                                 div(
>>                                     style=["color: red", "color:blue", None]
>>                                   , view="item"
>>                                   , controller="look")[_1]
>>                                 )
>>                             ]
>>                         ]
>>                     ]
>>                 ]
>>             ]
>>         ]
>>     ]
>>
>>     print f(['x', 'y', 'z', 'foo'])
>
>
> Hehe, I actually like that.  I know odd, but one part of our project
> involves a dynamic user interface that must be able to be updated from the
> host on the fly.  

That's not so odd.

> There are several ways to do that but the most flexible is some form
> of XML derivative.  Why?  Simple we will have quite a few
> non-programmers that have to manipulate these interfaces in multiple
> languages.  Everything in XML can be localized including the
> tags. Well at least that is the concept, we will have to see how it
> works in practice.

Hmm.  You can localize that Python structure just as well, but I'd
strongly advise against trying to localize tag names.  Having a
common programming language is important.

>> > Having to put together different wrappers for each of these
>> > instances would quickly become a maintenance nightmare.  The path
>> > you are taking to combine all the wrappers into one interface syntax
>> > is critical to supporting multiple languages.
>>
>> I don't think we've given much thought to allowing users to write one
>> generic program which wraps their class for both lua and Python.
>
> Ouch, sorry about that.  I guess I once again read more into this
> project than I should have.  I read the conversations about the
> different wrapper syntax's and assumed that since there seamed to be
> some sort of meeting of minds that perhaps a common syntax was in
> the works.

There is, so it might not be too difficult to achieve what you want.
It's just that there's a big difference between "using a very similar
syntax" and "the same wrapping code can be compiled for both a Python
and a Lua (and JavaScript, and...) environment".  That may require
sacrificing some language-specific features.

> While I am happy about the shared technology will help both Lua Bind and
> Boost.Python I was really excited about being able to write one interface
> and being able to compile it to produce both Lua and Python wrappers.  The
> other advantage is that we might have been able to piggyback some of our own
> definitions from your syntax.  I really hate repetitive interface
> definitions, forgetting to update one when you do another is just one
> maintenance headache I do not want to deal with.

Agreed, understood.  You might want to look at Alexander Nasonov's
recent introspection proposal on the main Boost list.

  http://aspn.activestate.com/ASPN/Mail/Message/boost/1697406

>> > When you guys finish this I have no doubt that it will become the
>> > defacto interface between any language and C++.
>>
>> Hmm, that's a bold statement.  It would sure be interesting.
>
> Yes it would, and it really depends on the goals outlined for the
> project as much as the people behind it.  From what I can see you
> guys are top notch, now its just a matter of setting your sites at
> the right target because I believe you can hit what you aim at.

If you have strong ideas about which is the right target the best
thing you can do is to describe it clearly and precisely.
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com