Re: pyrex like compiler

Mark Hahn Wrote:

After Greg Ewing shot down my garbage collector by pointing out that it
cannot work concurrently with the interpreterer, it is only fitting that we
use his Pyrex (or a ?PyrexLikeCompiler) to implement reference-counting
transparently, an idea being proposed by PaulH.

This rather radical idea could be very beneficial to Prothon by making
writing extensions much easier. You would not only avoid the burden of
reference counting code, but you could use the many other features Pyrex has
to offer.

In order to make this work we would have to make sure that Pyrex is totally
capable of being used to wrap C and C++ libraries. This would be an all or
nothing proposition.

Greg: What is the Pyrex licensing situation? I assume the current Pyrex code
is wrapped around Python objects and stack, so we would need to rewrite it
for Prothon, correct? Is Pyrex capable of wrapping C and C++ libraries?

Paul Prescod replied:
I think it would be counter-productive to try to adapt Pyrex-as-code as
opposed to the Pyrex-as-idea.
Here's how I would implement the idea if I were you
write a single parser that generates abstract syntax trees. This is the
right thing for a variety of reasons and I've been meaning to suggest it for
a while.
expose the syntax trees to Prothon
in Prothon, write two compilers that both work from the same syntax trees.
. AST -> C
. AST -> Bytecode
reuse the underlying runtime as much as possible.

Mark Hahn replied:
1) If we don't do Pyrex-as-code now as the standard extension, then I would
have to support a standard API with standard ref-counts. If I do it now, I
can use human-unfriendly techniques that are superior. I would like to
investigate the advantages of having only the ?PyrexLikeCompiler extension
facility.
2) I'm not sure why you want Prothon -> C. What would this be used for?

Mark Hahn replies to ?PyrexLikeCompiler
I've looked at the Pyrex website and it appears that the goals of Pyrex
differ from that of a "foreign function interface mechanism".

Paul replies:
What is Pyrex if not a "foreign function interface mechanism?" According to
the Pyrex overview: "Pyrex is a language specially designed for writing
Python extension modules. It's designed to bridge the gap between the nice,
high-level, easy-to-use world of Python and the messy, low-level world of
C."

Mark Hahn:
There is no support for linking with external C library headers and no
support at all for C++.

Paul:
cdef extern from "spam.h":

    int spam_counter

    void order_spam(int tons)
Pyrex supports C++ to the same extent that the Python and Prothon extension
APIs do. You need to treat C++ types as basically C pointers.

Mark Hahn:
It appears Pyrex is best supported for writing new code, not interfacing
with foreign code.

Paul:
You're incorrect. Pyrex is almost always used for interfacing with foreign
code.

Mark Hahn:
Having said this, Pyrex appears to be the closest thing to what we need for
a FFI. I think we need something more C/C++ oriented with concepts stolen
from Pyrex.
Right now I am thinking of a macro language for C that knows about the
Prothon API and ref-counting. It would be sort of a C mix-in that adds Pyrex
like features to C instead of a real Pyrex that straddles the fence as well
as Pyrex does. So instead of a Prothon language with C types and Prothon
types it would be an enhanced C language with Prothon types and C types.
To me the big advantage of Pyrex is the mixture of Python and C types in one
language. The Python syntax is not a big deal.

Paul Prescod:
By definition every C/Python programmer has access to a mixture of Python
and C types in one language. PyString_AsString converts from one to the
other. PyString_CheckString checks that a Python object is really a string.
The Python syntax (and semantics) of Pyrex is the whole point. After all,
Python programmers like programming in Python and often dislike C. Prothon
programmers will like programming in Prothon and will often dislike C.

Mark H:
There must be some advantage to writing the C api and the Pyrex compiler for
it at the same time. Knowing that programmers will never have to code it
directly in C might make it a simpler API to develop and maintain, and write
a compiler for. It would be a sort of "machine language" C api.