VNC server based on kdrive using damage extension?

We would like to build a lightweight X server which will operate "headless".
The only display device will be a remote VNC device. In other words, the
result would be similar to the xf4vnc project at sourceforge, but based on
freedesktop.org X server technology rather than XFree86 4.  We would be
willing to undertake all of the development, but are looking for some
architectural guidance. We would of course do this as an open-source
project.
 
What should we start with as a base? Is there a "new"  X server similar to
the old Xvfb server? Or should we use something like Xvfb, but create a
"dummy" frame buffer device?
 
Or, should we just run a standard X server (e.g. kdrive) , and handle VNC
via a compositor-like application which tracks damage regions (via the
damage extension) and sends them via the VNC protocol? In this case, we
wouldn't need to modify the server at all; a special compositor application
would handle VNC.
 
Note: are running an embedded GTK application, so we need enough X support
to handle at least that. We'd also like to be able to rotate the X display,
which kdrive can do but our current solution (xf4vnc) cannot. Transparency,
anti-aliased text, and other eye-candy is also desirable.
 
Thanks for any guidance you can provide.
 
Bruce Bauman
WorldGate Communications, Inc.