Re: Nevow flat improvement

On Tue, 7 Mar 2006 13:52:27 -0800, John Benediktsson <[email protected]> wrote:
>
>I found an improvement that can be made to nevow/flat/twist.py.
>
>It doesn't look like divmod.org accepts anonymous tickets, so I am
>posting this here.

Registration is open to all users.  You should see a "register" link near the top-right of the screen.  Registration is required so that we can follow up on all bug reports.

>
>The definition of deferflatten() causes cyclic references that force a
>user to use the garbage collector to clean references to contexts
>(PageContext, JavascriptContext, etc.) as well as other data referenced
>by those contexts.
>
>Any recursive inner function is self referencing and cannot be collected
>by reference counting.

True.  But it should be collectable by the cyclic garbage collector, so long as no object involved in the cycle defines __del__.  So this is a bug in CPython.  Still, I suppose Nevow should work around it until someone fixes it for real. ;)

>
>Basically the problem can be reduced to:
>
>  def f():
>      def g():
>          ...
>          g()
>      g()
>
>To fix this, you need to add a global statement:
>
>  def f():
>     global g
>     def g()
>         ...
>         g()
>      g()
>
>A patch is attached to show where the change could be made.

Unfortunately, this change is potentially quite broken, since it will cause concurrent flattening to randomly mix up results between clients.  We'll have to figure out some other solution.  Likely the simplest approach is to make the flattener a stateful object of some sort, rather than a closure.

I'm curious how you noticed this.  At various points I've seen Nevow apps grow in memory size, but I've never noticed function or cell objects in gc.garbage when this happens, which is how this problem manifests.

Jean-Paul