Re: About our effort at NoMachine

> Sender: [email protected]
> From: Gian Filippo Pinzari <[email protected]>
> Date: Wed, 26 Mar 2003 02:48:01 +0100
> To: [email protected]
> Subject: Re: [forum] About our effort at NoMachine
> -----
> On Tuesday 25 March 2003 07:38 pm, Juliusz Chroboczek wrote:
> > P> - NX compresses X protocol (and foreign protocols)
> > P>   to a degree that makes possible to run complete
> > P>   desktop sessions on a remote server across the
> > P>   Internet, even through a modem link.
> >
> > Your website seems to imply that you do magic to deal with latency.
> > Any chance for any more information on that ?  (Pointers to papers are
> > more than welcome, and I'd also understand if you wouldn't tell.)
> >
> >                                         Juliusz
>
> Sorry, I read my reply and it looks picky, that was not my intention :-)
> As I told in my first mail we didn't expect much interest about -how-
> this stuff was working, not yet, at least. Documents are currently
> aimed at users, not developers :-).
>
> I tried to explain how NX is working, but I realize that's far from
> enough. I'll add some sidenotes:
>
> - NX doesn't add any special request or extension to the X protocol.
>   It is a set of tools on top of X aimed to make X ubiquitous.
>
> - The performances of remote X sessions are due to nxagent's
>   ability to optimize the X traffic and the compression performances
>   of Xcomp/nxproxy.
>
> - Xlib modifications are needed to simplify the way you run
>   agents and to perform some other functions like cleaning padding
>   bytes in requests and (something LBX was not doing) in images.
>
> - Xcompext library is needed to let agents compress images in
>   various formats and transform replies in events.

Sounds right.  Hadn't thought about cleaning padding bytes, but you're
probably on to something there.  It is almost certainly worth clearing
the bytes entirely, as it is likely the memory cycle will happen in any
case, and then general stream compression works better.

Keith Packard has been recently gathering network data
to see where performance issues are (from the wire, up through
the X protocol stream on top, with time information everywhere),
with real (modern) applications (e.g. GTK, Qt, Mozilla) for a
Usenix paper we're working on together.

We're in the middle of grovelling through this data right now, but here
are a few highlights from this work in progress:

1) Client side fonts are making a significant reduction in round trips,
as expected, with real applications, and application startup time is
significantly reduced, relative to using core fonts, in all cases.
(local machine, local net, or simulated broadband network).

2) bandwidth usage for client side fonts is about the same as
with server side fonts, as expected, even with no code to compress the glyphs.
Compressing glyphs (or the entire protocol stream) would be a serious win.

3) LBX is no better than using straight SSH, and appears to be actually worse.
LBX looks to have been a bad idea, at least as implemented.

4) Some of the new toolkits are doing stupid things: we're talking
with the appropriate people to get them fixed.  I won't name names right
now, but you know who you are :-).

5) as expected, intern atom is a hotspot, along with fetching properties.
There needs to be a new callback based Xlib interface to allow streaming of
get property replies, to get rid of alot of round trips.  This is
causing havoc not only with application startup, but with things
like drag and drop feel.  Havoc Pennington coded some of this up (the
interface not ready for prime time yet) and it is used in his metacity
window manager, used in Gnome.

6) enough extensions get loaded, with typically at least 2 round trips
at least to get each going, and maybe 4-5 extensions used
by the applications (toolkits). We need an extension extension to avoid
these round trips, (one could just kludge around with a batching call;
but there are other shortcomings with the extension system,
like the lack of extension versioning in the framework) that I think
it is wise to think about a better framework).

7) image transport is a real issue, whether for glyphs or other images.
This is in part due to image sizes having grown due to greater depth
(most people have 32 bits/pixel these days). So besides compressing
glyphs, having the window system able to better
take images in pretty native formats would be a serious win for alot of
applications.

The trap to avoid here is the XIE slippery slope, and focus on image transport,
rather than image processing.

If you do all of the above, there is for some interesting applications
(e.g. Mozilla), maybe a factor of 5-6(!) speed is available, over simulated
broadband links.  We haven't looked at dialup, though that would accentuate
things further in various directions.

All these things, would significantly speeds things up in the local
case as well (e.g. elimination of unneeded round trips, avoiding gratuitous decompression/compression stages, which cause CPU usage along with
context switches and cache flushes).

More to come when the paper is finished and sent in to Usenix.  We're
in the middle of grovelling through the data right now.
                             - Jim


---
Jim Gettys
Cambridge Research Laboratory
HP Labs, Hewlett-Packard Company
[email protected]