Re: Phil meet y-devel; y-devel meet Phil

On Thu, 22 Jul 2004, Phil wrote:

> > On Wednesday 21 July 2004 11:35, Ulrik Mikaelsson wrote:
> >>Hmm, is it possible to do useful GC:ing on a network-protocol with
> >>different client-implementations? For instance, how would the server know
> >>if a client is REALLY still using it's reference to an object? Is it
> >>possible to garbage-collect memory, or even detect memory leaks from a
> >>misbehaving client?

We can't GC and we can't detect memory leaks (incidentally, garbage
collectors won't stop you from leaking memory, I have seen plenty of leaky
java programs.)

> It can never be safe to destroy a valid object owned by a connected
> client (an object should become invalid when the client notifies the
> server that it is finished with it),

Currently, the server knows of all the objects that "belong" to a client
(they're stored in a struct Index called "objects" in the struct Client
[src/Y/message/client_p.h:12]).  It uses these to automatically destroy
all the objects when a client exits.  This will need to be enhanced when
we start allowing cilents to pass object references between each other,
but I think simple reference counting will suffice.

ISTR there isn't a way for clients to manually destroy an object they
created (unless Andy wrote a "destroy" method when he fixed up the object
system).  An exercise for someone wanting something to do to poke around
the tree?

> since there is no way to guarantee
> it has really forgotten it.  We might need an assassin function to kill
> those clients which have too many apparently unused server objects, but
> this should only be used when there are insufficient resources on the
> server (like the Linux OOM killer).  Allowing for the server to
> unilaterally delete objects which may still be valid is too risky.

I think this is our only choice.  It's currently what X does.  There was a
bug in mozilla firebird (back when it was called firebird) that meant they
had a pixmap leak.  Every few days my browser (which I never close) would
implode.  After a bit of poking around, I noticed that before the
implosion, X was coming up to using nearly all my RAM, afterwards, it was
back to its usual self.

Since we have widgets in the server, we can display a nice message:

  "This program is using too many system resources and will be terminated.
   Please bitch and moan at the developers to fix their code. Have a nice
   day."

(Wording subject to negotiation)

> Side note: Since the client and server are separated by the protocol, a
> memory leak in the client can never be detected by the server.  It's
> also pretty much impossible to detect a memory leak in a running program
> when this is not the case.

Actually, I wouldn't be surprised if detecting true memory leaks was
halting-problem equivalent.


> > One simple way to tell is to keep a client reference count for each object,
> > which would only keep track of how many clients hold references to an object.
> > When that count reaches zero (and you might have a server flag which would
> > keep this from being garbage collected), the resource could be freed.  This
> > way, the GC would only need to be run each time a client disconnects,
> > reducing overhead.
>
> I am reasonably sure that for those objects which can be destroyed it
> will only be possible for one client to have a reference to that object
> anyway.

Not entirely.  We need a way to implement what Gtk calls Plug and Socket
widgets.  The easiest way is to allow a client to make a whole bunch o'
widgets and pass the oid of the topmost widget safely to another client,
which can then pack it inside its own widgets wherever.  This opens a
whole can of worms regarding what to do when each client exits (at the
very least, they need to be told about it).

> Of course, it may be that that client has multiple references
> to the same thing, but I would imagine it will be cleaner to have the
> client end of things handle mapping multiple refernces onto a single
> server object.

Yes, each client can only have one reference to an object since that is
its oid, an integer which is server-unique.

For now at least, we don't need to worry about wayward clients allocating
too much.  Long term, the using-too-much-resources-killer is probably our
only defence from being targeted by the linux OOM killer, or whatever the
equivalent is on whichever platform.

Regards,

  Mark.
-- 
|| Mark Thomas
|| efaref.net
||
|| The trouble with life is the lack of cool background music.