meta-compiler

I was discussing my meta-compiler ideas with Howerd, I thought I might as 
well publish them here, to see what others think. Just to note, some of 
the code below (building dictionary entries) is untested.

If you don't know what a meta-compiler is, the best I can describe it is 
as an application which allows you to compile a new Forth kernel itself 
defined in Forth.

If the Forth kernel is targeted to a different processor, there may be 
additional complications beyond what is described here. At the moment, I 
am interested in compiling a pentium colorForth kernel using the present 
pentium colorForth kernel. What this means is that compiling the 
colorForth kernel will be little different than compiling any other 
colorForth application. The differences however are important to note:

* The kernel must be self contained, it cannot refer to words defined 
outside of itself, except for macros which compile instructions inline.

* Variables must refer to locations within the kernel, not locations in 
source.

* All absolute addresses which reference locations within the kernel must 
be adjusted for the kernel's target location.

In addition to these differences, the kernel requires that some of the 
words defined as part of the kernel also be defined in a new dictionary 
which is part of the kernel. This second dictionary will be used by the 
new kernel to interpret and compile and execute, just as the active 
dictionary is used by the active kernel to interpret and compile and 
execute.

Given the above requirements, we can design and implement a meta-compiler 
suitable for compiling a new pentium colorForth. One choice I have made is 
to have two seperate spaces to compile to, this mimics other meta and 
target compilers I have seen (cmForth, X18), and I think it has some 
advantages.

I have a variable 'h' and a word 'target' which will switch between 
compiling in the two spaces. This can be acomplished because all 
compiling words reference the variable 'h' in the kernel. I have chosen 
the location of the kernel to be at block $800 (2 MB). This value is 
itself not signifigant and can be changed to another location if need be, 
the same is true for some of the other constants used below. A word 
'delta' is defined to encode the difference between where the kernel is 
compiled and where it targeted to be used. To this point here are the 
words defined as part of the meta-compiler:

( Metacompiler ) empty variable h forth
: memory [ $800 block ] ; memory 4 * h !
: delta [ memory negate 0 block + ] ;
: target [ h ] @ [ last -1 + ] dup @ [ h ] ! ! ;

colorForth gives a meaning to each word by its color, the color acts 
simply as an index into a table of function pointers, each executed upon 
interpreting a word of that color. This table is accessible by the word 
'sp', therefore the meaning of each color can changed by writing to 
locations offset from 'sp'.

: f! [ sp ] + ! ;
: f@ [ sp ] + @ ;

This is suitable to change the meaning of variable (magenta), but for 
building the new dictionary, instead of changing the meaning of red words, 
which will be changed by 'macro' and 'forth', I use the kernel variable 
'class', the value of which is executed after a red word is defined only 
if the value is non-zero.

: class [ last 1 + ] ! ;

The format of the dictionary built in the new kernel is the same as the 
active kernel, two parallel arrays 'macro' and 'forth' are interleaved, 
each with a count of the number of definitions at the head. I define the 
following words to build entries in the new dictionary:

?macro - return the address of the count and the address of the name
          array of either the forth or macro dictionary depending on which
          is being defined.
entry - return the address and name of the last defined word.
define - define a word in the new dictionary.

: defer pop ;
: ?macro 3 f@ [ 3 f@ negate ] + drop if [ 5 block delta + dup -2 + ] dup 2 
+ ; then [ -1 + ] dup 129 + ;
: entry [ last ] @ dup @ [ delta -4 * ] + swap [ gap negate ] + @ ;
: define defer 1 ?macro over @ + dup push entry pop ! [ 128 1024 + ] + ! 
+! ;

I change the meaning of variable, so that a variable when referenced 
in yellow gives an address of a cell in the kernel:

: execute push ;
: cell here 3 ? if 1, cell ; then 2/ 2/ [ delta ] + ;
: variable defer 0 class [ 12 f@ ] execute define class cell [ last ] 
@ [ gap negate 1 + ] + ! 0 , ;

Finally, I need a word 'meta' to enable 'define', and give variable 
(magenta) its new meaning. A word '/meta' will return things back to 
normal:

: meta variable 12 f! define class ;
: empty empt
: /meta [ 12 f@ ] 12 f! 0 class ;

The full one block meta compiler:

( Metacompiler ) empty variable h forth
: memory [ $800 block ] ; memory 4 * h !
: delta [ memory negate 0 block + ] ;
: target [ h ] @ [ last -1 + ] dup @ [ h ] ! ! ;
: f! [ sp ] + ! ;
: f@ [ sp ] + @ ;
: class [ last 1 + ] ! ;
: defer pop ;
: ?macro 3 f@ [ 3 f@ negate ] + drop if [ 5 block delta + dup -2 + ] dup 2
+ ; then [ -1 + ] dup 129 + ;
: entry [ last ] @ dup @ [ delta -4 * ] + swap [ gap negate ] + @ ;
: define defer 1 ?macro over @ + dup push entry pop ! [ 128 1024 + ] + !
+! ;
: execute push ;
: cell here 3 ? if 1, cell ; then 2/ 2/ [ delta ] + ;
: variable defer 0 class [ 12 f@ ] execute define class cell [ last ]
@ [ gap negate 1 + ] + ! 0 , ;
: meta variable 12 f! define class ;
: empty empt
: /meta [ 12 f@ ] 12 f! 0 class ;