FW: PHP on DLR

Just forwarding our earlier mail conversation to the mailing list...
I'll send my comments in a few minuttes.

Thanks,
Tomas Petricek

-----Original Message-----
From: Tomas Matousek [mailto:[email protected]] 
Sent: Tuesday, November 06, 2007 9:27 PM
To: 'Wez Furlong'
Cc: Tomas Petricek
Subject: RE: PHP on DLR

Thanks Wez!

Sorry for delayed response, I was busy at RubyConf.

I feel a need to clarify some facts about Phalanger and my ideas how it
could possible evolve in the future to avoid possible misunderstanding.
Please, feel free to give me your feedback. Tomas will probably provide more
comments as well and we can also start answering any questions in the new
mailing list.

First, it would be great if our cooperation could target .NET/Mono platform
in general - not only Silverlight but also desktop CLR, i.e. client as well
as server scenarios. I know it would kind of compete with your PHP
implementation and that the compatibility is an issue as well. Anyway, I
think the two implementations could live side by side adding value to each
other. I also think that if the community gets involved, the CLR
implementation (be it Phalanger or a new one) could eventually become highly
compatible with the current PHP implementation.

Let me explain what the current state of Phalanger is, especially how it
relates to Silverlight and DLR.

Tomas's current attempt to get Phalanger to Silverlight inheres in
restructuring/refactoring the code base so that all types and methods that
are not needed in browser scenario on Silverlight could be simply commented
out using #ifdefs. For example, all the ASP.NET integration infrastructure
could be moved to a separate namespace and conditionally compiled only for
server version of Phalanger. The same holds for infrastructure for native
PHP extensions support, file-system libraries, Visual Studio integration
support, etc. The goal is to make Phalanger as small as possible w/o
restricting the language itself (some library functions are indeed cut).

The current Phalanger code doesn't use DLR at all. The plan here is to
incrementally replace those components of Phalanger where it does make sense
by a new code that uses DLR. DLR itself uses CLR features only. There is no
black magic behind the scenes, no native code execution, no special hooks
into the CLR runtime - it's like any other C# library on .NET platform.
Therefore, DLR runs on Mono w/o modifications. DLR can be viewed as a
library of components that makes a compiler writer's life a little bit
easier. Any language implementation can use as much or as little as
convenient. Some examples of DLR features I can now think of that might
possibly be used in Phalanger or in a PHP on DLR implementation in general
include:

1) DLR AST. This is an intermediate representation that the language
compiler could transform its language specific AST to. DLR transforms DLR
AST to IL for you so the language compiler doesn't need to directly generate
IL at all or only a little. DLR will provide some AST analyses that results
in better generated IL. Finally, DLR AST supports interpretation so that the
language doesn't need to implement it for its AST. Interpretation is
generally useful for code that runs just once (e.g. eval'ed code that
doesn't contain loops). Although Phalanger's IL generation is done, it
doesn't support interpretation and also some pieces of it could be done
better. So here I see the value of using DLR ASTs: the replacement of IL
generation code to AST transformation code could be done incrementally and
could lead to much nicer and simpler source code as well as enable eval
interpretation. Thought the current version of DLR ASTs is not enough for
Phalanger, I hope the final version will provide all necessary features.

2) DLR Dynamic Sites. Call site caching could be used mainly for CLR method
calls (especially for speeding up method overload resolution), for
implementation of PHP method calls and property access operations, for
conversion operations that switch over types and maybe some other minor
features. Note however, that Phalanger highly optimizes some method/function
calls by performing global code analysis resulting even into direct CLR
method calls. For that cases, DLR dynamic sites provide no added value. DLR
will probably also not help a lot for by-name variable/method/function calls
(e.g. $$x, $x(), $x->{$y} etc.), but that might change in future.

3) DLR Hosting API. Hosting API provides the host application (be it an IDE,
scripted application/game, etc.) a uniform API that it can leverage to
execute scripts and manipulate objects in a language agnostic or specific
way. DLR e.g. provides a REPL loop based on the hosting API. To get
advantage of these APIs a language is needed to provide an implementation of
certain DLR abstract types/interfaces.

4) Interop with other dynamic languages provided by DLR. If Phalanger types
that represent PHP classes and primitive types (PhpObject, PhpArray,
PhpResource, etc.) implement certain DLR interfaces/operators, it will be
possible to seamlessly use their instances from other dynamic languages. It
would be then possible to work with PHP libraries from Ruby or Python and
vice versa.


Phalanger is not quite "done"; there are still some big TODOs. Apart from
integration with DLR, some features could be refactored and improved, some
PHP language features are missing, namespaces implementation need to be
synced with PHP 5.3, native extensions interop layer (Extension Manager)
needs to be upgraded to support PHP 5 native extensions (the current support
ends with PHP 4.2 as far as I remember). On the other hand many components
are complete and IMO in great state - we refactored the compiler's type
system when moving from version 1.0 to 2.0, improved scanner, parser, source
file inclusion subsystem, configuration infrastructure, ASP.NET server
integration components, PHP array implementation, and many other parts of
Phalanger. These pieces as well as implemented library functions and managed
extensions are IMO of great value and they absolutely needn't to be altered
when porting other parts to DLR. There is no need and no point in changing
them at all. DLR doesn't provide any replacement for them.

I see your point in initial community exclusion from Phalanger development.
So I'm not surprised that you would like to try to implement PHP on CLR from
scratch. It has been a great adventure for us and it would be definitely
interesting for anyone else to do so. However I also know how many details
needed to be polished and how many not-so-interesting hours of work it was
to convert the school project PHP.NET implementation that wasn't able to run
major PHP apps, wasn't scalable to huge PHP apps and wasn't able to interop
with CLR well to a decent quality software: Phalanger v2.0. Nowadays,
Phalanger is able to run at least dozen of wide spread PHP apps/frameworks
(maybe with some details to be polished), scales well (we tested a real
world application with 5000+ script files) and provides easy to use and high
fidelity CLR interop experience.

It's definitely up to you and community which way to go: whether to
participate on Phalanger or start a new project from scratch. It's your time
and abilities you'll maybe contribute to the PHP on CLR implementation. I
would be not offended at all if you decided to implement PHP for CLR/DLR
from scratch and I would wish you all good. On the other hand Phalanger is
here, it works quite well, you can contribute to make it even better, you
might maybe learn from it, add new features, make it true to PHP language
and add value to the community this way. As it is usual in open source
community. Either way Phalanger will be there, I guess that Tomas will still
work on it because he already dig into its internals and he enjoys it; it's
of course up to him how much time he devotes to it. And it wasn't easy for
him to comprehend it at the beginning. Sure. It wouldn't be easy for anyone
as it's quite large piece of software. I was at similar position when I came
to Microsoft to work on DLR - at that time DLR was entangled with IronPython
runtime and was already a large project whose internals I had never seen
before. It was tough to understand the code base, but I'm glad that I took
this challenge as I learned a lot. I also used my PHP experience to make DLR
more universal (and less Pythonish) and I'm doing it again with IronRuby.

Let me know what you think.
Thanks for your PHP/DLR effort within the community!

Tomas