Re: pyrex as Prothon standard extension language

I wanted to edit but the Wiki is starting to ask for edit permission. 
Here's what I wanted to say:

====

The standard foreign function interface mechanism for most interpreters 
is a C API. This is easy for the implementors because they are already 
coding in C and they just need to decide which functions to make 
publically available as part of the API and which to keep internal. The 
downside is that they now need to maintain compatibility with two 
different interfaces: the language itself and the API. Changes that are 
easy in the language may be difficult in the API and vice versa. If the 
API is very high level, it tends to be inefficient. If it is low-level 
then it constrains your future implementation. For instance if the API 
uses mark and sweep garbage collection then it is hard to switch to 
reference counting later and perhaps vice versa.

Instead of defining the FFI in C, it could be possible to define it in 
the language itself. (e.g. Python's Ctypes and struct modules). This 
model is almost guaranteed to be inefficient and it has portability 
problems because there are times when C code uses different types 
depending on the platform.

An alternative is to define another language that is the intersection of 
C types and Prothon types (as Pyrex does for python). A compiler for 
this language compiles to C code that binds Prothon types to C types and 
Prothon code to C code. Like any compiler, it can do quite a bit of 
optimization to bridge the gap between the high level abstractions that 
the programmer works with and the low-level code they might have written 
if they were writing by hand. As the underlying language implementation 
changes (arbitrarily severely) the compiler can change its behaviour 
without any code being affected.

Completely independent of the benefits of having a Pyrex-like compiler 
as the standard FFI (for the interpreter implementor), it is great for 
such a thing to be available even if only as an option. For instance, 
Python's Pyrex is gaining increasing popularity over direct C or SWIG 
interfaces.

The primary issue with implementing a compiler as a foreign function 
interface is that it is a lot of effort. It isn't quite as much as 
writing both a full compiler and a full interpreter (because you share 
the parser and the runtime) but it isn't trivial.

====

  Paul Prescod