Hi all,

>> First, it would be great if our cooperation could target .NET/Mono
platform 
>> in general - not only Silverlight but also desktop CLR, i.e. client as
well as server 
>> scenarios. 

Currently I think that the priority of the .NET implementation should be
Silverlight (as a platform) because this is, as far as I understand, what
the PHP community is interested in (correct me if I'm wrong!)  and this is
also where the official PHP cannot be used, so we could start nicely the
cooperation between .NET based and the official PHP version.

I personally think that there are very interesting possibilities of
supporting.NET/Mono too. First of all, this makes it possible to develop
(i.e.) WinForms or Gtk# applications in PHP (and distribute them in a
user-friendly way) or other project types that can be done on .NET/Mono -
this doesn't require 100% compatibility (especially in terms of libraries)
with the official PHP implementation, so it is very well manageable even for
a small development team. 

Finally, having a PHP running on .NET on a server-side has some appealing
properties as well, especially for a large companies that need to integrate
their .NET and PHP solutions (and I received a few mails asking about that),
but as already mentioned - there is a compatibility problem (especially when
it comes to a large number of PHP libraries). I would not like to "drop"
this possibility in the beginning, because I believe that once the project
evolves we can let the PHP community decide - if there will be enough people
interested in this topic and willing to help with the implementation, than
we could support this as well.

>> For example, all the ASP.NET integration infrastructure could be moved to
a 
>> separate namespace and conditionally compiled only for server version of
Phalanger. 

Yes, this is in general how it currently works. There is only one "branch"
in the CodePlex project and it can be compiled with a "SILVERLIGHT" option
to get the Silverlight version. There are two solutions ("Phalanger.sln" and
"Phalanger (Silverlight).sln") which contain different files (where a
functionality is not needed in one of the implementations) and there are a
few #ifs in the code, but I plan to polish the code a little bit to remove
some of them and keep the different functionality in a separate files, which
makes the project IMHO more understandable than a lot of #ifs.

If you would be interested in managing the Silverlight (and in the future
DLR) implementation as part of the official PHP project, I don't see any
problems with moving the part of the project (or even a whole project) to
your SVN, but I'd still like to replicate all the relevant changes and fixes
to the Phalanger on CLR "branch".

>> The current Phalanger code doesn't use DLR at all. The plan here is to
incrementally 
>> replace those components of Phalanger where it does make sense by a new
code that 
>> uses DLR. DLR itself uses CLR features only. There is no black magic
behind the scenes,
>> no native code execution, no special hooks into the CLR runtime - it's
like any other C# 
>> library on .NET platform. Therefore, DLR runs on Mono w/o modifications. 

Yes, currently Phalanger just implements its own "DLR", which has a few
disadvantages i.e. no interoperability with other dynamic languages (though
I'm not sure how big issue this really is..) and the fact that we're not
leveraging an existing component developed by a skilled developers from MS
(which would give us some things for free), but also has some advantages (at
least in the current version), especially that our runtime is highly
optimized for PHP.

Regarding the Phalanger project, we'd like to start taking advantage of DLR
and eventually port it fully to DLR (when DLR will be able to run the PHP
language fully and when it will support producing "standalone" assemblies,
so we can write WinForms etc. apps, or even running server-side web
applications).


To comment the specific features suggested by Tomas (from the Phalanger
point of view):

>> 1) DLR AST.  (...)  Thought the current version of DLR ASTs is not enough
>> for Phalanger, I hope the final version will provide all necessary
features.
I think it would be possible to start using these in simple cases where they
are sufficient - for example in simple uses of the "eval" function. This
would be useful for both Phalanger folks and the PHP community, because we
all need to learn what the DLR AST can and cannot do and how we can use it
(and of course, tell Microsoft what we need to get PHP fully running on
DLR!)

>> 2) DLR Dynamic Sites. 
I had a chance to see a presentation by Martin Maly who is working on DLR
and where he demonstrated these and I quite like this feature. I think these
could be used very nicely (even though the Phalanger implementation of .NET
method calling is quite good now). 

>> 3) DLR Hosting API.
We can indeed look at this as well - I guess that it would allow big .NET
applications to use PHP as another possible scripting language. On the other
side Phalanger currently provides very good interop with .NET (I would say
that even better than DLR), which allows people to use PHP objects from
C#/... with minimal effort.

>> 4) Interop with other dynamic languages provided by DLR
I think supporting this in a current Phalanger implementation shouldn't be
difficult.


>> Phalanger is not quite "done"; there are still some big TODOs. (...) On
the other 
>> hand many components are complete and IMO in great state (...) I see
>> your point in initial community exclusion from Phalanger development. 
>> So I'm not surprised that you would like to try to implement PHP on CLR
from 
>> scratch. 

I can just confirm that Phalanger has a lot of components that will be
needed for the PHP on DLR implementation (when the DLR will be flexible
enough to handle PHP language) and I think that if we wanted to get PHP on
Silverlight before the required adjustments will be made in DLR, we would
have to re-implement most of the Phalanger (without any significant added
value). I of course understand that coming to a partially completed project
which was started without the community involvement is difficult. 

I personally would be happier if I could work on a project fully supported
by the PHP community (with PHP developers involved in the development of the
project) instead of working on a project that sort of competes with the
official PHP implementation and has very small community support. I think
that Phalanger currently has a lot to offer to the community (and I mean
especially the fact that it already works reasonably with Silverlight, which
is what PHP needs to match Python and Ruby), so I believe that the best
thing to do would be to use what Phalanger currently has and see how we can
work together to evolve it and replace the parts that have corresponding
part in the DLR library by the DLR implementation.

I of course understand how difficult it is to get involved in a project of
this size - I think that the best option would be if we could all meet in
person and spend a few days looking into Phalanger internals and start
working on integration of reasonable DLR parts. This wouldn't be indeed easy
to arrange, but I think that if you were interested in that we could arrange
it (with some possible help of DLR team in MS).

I tried to focus more on technical issues that we would have to face, but I
clearly understand that currently it is still emotionally sensitive topic
for all of us, so please don't feel offended by any part of my reply. I hope
we can find a reasonable solution based primarily on the technical reasons
after a few more iterations!

Thanks,
Tomas Petricek

-----Original Message-----
From: Tomas Matousek [mailto:Tomas.Matousek@microsoft.com] 
Sent: Tuesday, November 06, 2007 9:27 PM
To: 'Wez Furlong'
Cc: Tomas Petricek
Subject: RE: PHP on DLR

Thanks Wez!

Sorry for delayed response, I was busy at RubyConf.

I feel a need to clarify some facts about Phalanger and my ideas how it
could possible evolve in the future to avoid possible misunderstanding.
Please, feel free to give me your feedback. Tomas will probably provide more
comments as well and we can also start answering any questions in the new
mailing list.

First, it would be great if our cooperation could target .NET/Mono platform
in general - not only Silverlight but also desktop CLR, i.e. client as well
as server scenarios. I know it would kind of compete with your PHP
implementation and that the compatibility is an issue as well. Anyway, I
think the two implementations could live side by side adding value to each
other. I also think that if the community gets involved, the CLR
implementation (be it Phalanger or a new one) could eventually become highly
compatible with the current PHP implementation.

Let me explain what the current state of Phalanger is, especially how it
relates to Silverlight and DLR.

Tomas's current attempt to get Phalanger to Silverlight inheres in
restructuring/refactoring the code base so that all types and methods that
are not needed in browser scenario on Silverlight could be simply commented
out using #ifdefs. For example, all the ASP.NET integration infrastructure
could be moved to a separate namespace and conditionally compiled only for
server version of Phalanger. The same holds for infrastructure for native
PHP extensions support, file-system libraries, Visual Studio integration
support, etc. The goal is to make Phalanger as small as possible w/o
restricting the language itself (some library functions are indeed cut).

The current Phalanger code doesn't use DLR at all. The plan here is to
incrementally replace those components of Phalanger where it does make sense
by a new code that uses DLR. DLR itself uses CLR features only. There is no
black magic behind the scenes, no native code execution, no special hooks
into the CLR runtime - it's like any other C# library on .NET platform.
Therefore, DLR runs on Mono w/o modifications. DLR can be viewed as a
library of components that makes a compiler writer's life a little bit
easier. Any language implementation can use as much or as little as
convenient. Some examples of DLR features I can now think of that might
possibly be used in Phalanger or in a PHP on DLR implementation in general
include:

1) DLR AST. This is an intermediate representation that the language
compiler could transform its language specific AST to. DLR transforms DLR
AST to IL for you so the language compiler doesn't need to directly generate
IL at all or only a little. DLR will provide some AST analyses that results
in better generated IL. Finally, DLR AST supports interpretation so that the
language doesn't need to implement it for its AST. Interpretation is
generally useful for code that runs just once (e.g. eval'ed code that
doesn't contain loops). Although Phalanger's IL generation is done, it
doesn't support interpretation and also some pieces of it could be done
better. So here I see the value of using DLR ASTs: the replacement of IL
generation code to AST transformation code could be done incrementally and
could lead to much nicer and simpler source code as well as enable eval
interpretation. Thought the current version of DLR ASTs is not enough for
Phalanger, I hope the final version will provide all necessary features.

2) DLR Dynamic Sites. Call site caching could be used mainly for CLR method
calls (especially for speeding up method overload resolution), for
implementation of PHP method calls and property access operations, for
conversion operations that switch over types and maybe some other minor
features. Note however, that Phalanger highly optimizes some method/function
calls by performing global code analysis resulting even into direct CLR
method calls. For that cases, DLR dynamic sites provide no added value. DLR
will probably also not help a lot for by-name variable/method/function calls
(e.g. $$x, $x(), $x->{$y} etc.), but that might change in future.

3) DLR Hosting API. Hosting API provides the host application (be it an IDE,
scripted application/game, etc.) a uniform API that it can leverage to
execute scripts and manipulate objects in a language agnostic or specific
way. DLR e.g. provides a REPL loop based on the hosting API. To get
advantage of these APIs a language is needed to provide an implementation of
certain DLR abstract types/interfaces.

4) Interop with other dynamic languages provided by DLR. If Phalanger types
that represent PHP classes and primitive types (PhpObject, PhpArray,
PhpResource, etc.) implement certain DLR interfaces/operators, it will be
possible to seamlessly use their instances from other dynamic languages. It
would be then possible to work with PHP libraries from Ruby or Python and
vice versa.


Phalanger is not quite "done"; there are still some big TODOs. Apart from
integration with DLR, some features could be refactored and improved, some
PHP language features are missing, namespaces implementation need to be
synced with PHP 5.3, native extensions interop layer (Extension Manager)
needs to be upgraded to support PHP 5 native extensions (the current support
ends with PHP 4.2 as far as I remember). On the other hand many components
are complete and IMO in great state - we refactored the compiler's type
system when moving from version 1.0 to 2.0, improved scanner, parser, source
file inclusion subsystem, configuration infrastructure, ASP.NET server
integration components, PHP array implementation, and many other parts of
Phalanger. These pieces as well as implemented library functions and managed
extensions are IMO of great value and they absolutely needn't to be altered
when porting other parts to DLR. There is no need and no point in changing
them at all. DLR doesn't provide any replacement for them.

I see your point in initial community exclusion from Phalanger development.
So I'm not surprised that you would like to try to implement PHP on CLR from
scratch. It has been a great adventure for us and it would be definitely
interesting for anyone else to do so. However I also know how many details
needed to be polished and how many not-so-interesting hours of work it was
to convert the school project PHP.NET implementation that wasn't able to run
major PHP apps, wasn't scalable to huge PHP apps and wasn't able to interop
with CLR well to a decent quality software: Phalanger v2.0. Nowadays,
Phalanger is able to run at least dozen of wide spread PHP apps/frameworks
(maybe with some details to be polished), scales well (we tested a real
world application with 5000+ script files) and provides easy to use and high
fidelity CLR interop experience.

It's definitely up to you and community which way to go: whether to
participate on Phalanger or start a new project from scratch. It's your time
and abilities you'll maybe contribute to the PHP on CLR implementation. I
would be not offended at all if you decided to implement PHP for CLR/DLR
from scratch and I would wish you all good. On the other hand Phalanger is
here, it works quite well, you can contribute to make it even better, you
might maybe learn from it, add new features, make it true to PHP language
and add value to the community this way. As it is usual in open source
community. Either way Phalanger will be there, I guess that Tomas will still
work on it because he already dig into its internals and he enjoys it; it's
of course up to him how much time he devotes to it. And it wasn't easy for
him to comprehend it at the beginning. Sure. It wouldn't be easy for anyone
as it's quite large piece of software. I was at similar position when I came
to Microsoft to work on DLR - at that time DLR was entangled with IronPython
runtime and was already a large project whose internals I had never seen
before. It was tough to understand the code base, but I'm glad that I took
this challenge as I learned a lot. I also used my PHP experience to make DLR
more universal (and less Pythonish) and I'm doing it again with IronRuby.

Let me know what you think.
Thanks for your PHP/DLR effort within the community!

Tomas