[stack] Purrr

Dear All,

Allow me to introduce myself. My name is Tom Schouten. I live in
Leuven, Belgium and I'm 32 now, if that helps paint a picture. I've
been interested in concatenative programming for a while and lurking
here and there.. To educate myself, I wrote quite a lot of code in the
last couple of years, and I'd like to share some of the results, but
maybe even more the resulting questions. (warning: long post, story of
my life :)

My background is in electrical engineering. My heart lies in music
DSP.  I've been working up the ladder from electronics, to machine
language and C/C++, through Pure Data (a data flow language) to Perl &
Python to finally end up at Scheme and functional programming. I'm
flirting a bit with Haskell, but really just read because most recent
interesting functional programming texts use that language.

http://en.wikipedia.org/wiki/Pure_Data

The problem I'm trying to solve to guide me a bit is "Build tools to
write DSP code, mostly for sound and video processing, in a high level
language." I ran into limits of expressiveness writing video
extensions in C for Pure Data, about 4-5 years ago. Apparently there
are no freely available tools that solve this problem, so I take that
to be my mission.

About 3-4 years ago I started writing Packet Forth (PF) as an
attempt to grow out of my C shoes. It was at the time I discovered
colorForth, and I was wondering if I could create some kind of cross
breed between Pure Data and Forth. PF now looks a bit like Factor on
the outside, though is less powerful. PF uses linear memory management
(data is a tree), with some unmanaged pointers for data and code
variables. PF's point is a to be a scripting language which tosses
around some DSP operations written in C. It doesn't aim to be a
general purpose language. The darcs archive is here:

http://zwizwa.goto10.org/darcs/packetforth/

Some more more highlevel docs aimed at media artists here:

http://packets.goto10.org

I got a bit frustrated with the internals of PF, mostly because there
is still too much verbose C code, and a lot of C preprocessor macro
tricks that could best be done with a _real_ C code generator.

So I dived a bit deeper into Forth, and early 2005 I started at the
bottom again: I wrote an indirect threaded forth for Pure Data (mole),
and started BADNOP (now dubbed BROOD 1), an interactive cross compiler
for the Forth dialect Purrr, an 8-bit stack machine model for Flash
based PIC Microcontrollers.

http://zwizwa.goto10.org/darcs/mole
http://zwizwa.goto10.org/darcs/brood-1

Mole made me 'get' Forth finally: the first versions of PF were mostly
blind hackery to get to know the problems before the solution. For
mole, I actually followed tradition a bit more (Brad Rodriguez'
"Moving Forth"). This lead to a more decent PF interpreter.

The forth I wrote to write the cross-compiler for the PIC Forth was a
mess. I was experimenting with some quotation syntax but realized that
what I was really looking for was lisp, or a lisp--like concatenative
language. At that time, early 2006, I discovered Joy, so I ditched the
compiler and rewrote it in CAT (not Christopher's Cat) which was
written in scheme (BROOD-2). After some refactoring and rewriting due
to beginner mistakes I am now at BROOD-4, with the CAT core written as
a set of MzScheme macros. This CAT is a dynamicly typed concatenative
language with Scheme semantics. I consider it an intermediate
language. Currently it is only used to implement the Purrr compiler (a
set of macros) and the interaction system.

http://zwizwa.goto10.org/darcs/brood

Purrr is as far as I know somewhat novel. All metaprogramming tricks
one would perform in Forth using the [ and ] words are done using
partial evaluation only. I've tested this in practice and it seems to
work surprisingly well.

I am still struggling a bit with the highl level Purrr semantics
though. Concretely, it is a fairly standard macro assembler with
peephole optimization. Nothing special there. On the other hand, its
macro language is a purely functional concatenative language which is
'projected' onto a real machine architecture after being partially
evaluated. I tried to explain these concepts in the following papers:

http://zwizwa.goto10.org/darcs/brood/tex/purrr.pdf
http://zwizwa.goto10.org/darcs/brood/tex/brood.pdf

(for the latest versions it's always better to use the .tex from the
darcs archive)

The latter needs some terminology cleanup, but it contains an
explanation of the the basic ideas, and an attempt to clarify the
macro semantics in a more formal way. I'm interested to learn what i
need to read in order to frame these concepts in proper CS speak... It
looks like I'm either terribly ignorant of something that already
exists (I went through a couple of stages of that already), or I found
a clean way of giving cross-compiled Forth a proper semantics.

On a lighter note, I'm using Purrr to build The Sheep, a retro synth
inspired by 1980's approach to sound generation. It runs on CATkit,
and has been used successfully many times in beginner "physical
computing" workshops, as electronics is called in non-engineering
circles :)

http://zwizwa.goto10.org/darcs/brood/tex/synth.pdf
http://packets.goto10.org/packets/wiki/CATkit

(the scary guy in the picture is not me :)