Re: Optimising #include creation of FastWriter

On Thu 9 March 2006 23:11, Keats Kirsch wrote:
> If my memory serves (which is dubious) the FastWriter caches an entire
> page before flushing to the output stream.  This was a controversial
> design decision -- there were a lot of complaints about potential
> performance impacts, but no metrics that I'm aware of.  At one point I
> developed a #flush directive to work around this for a particular
> application.

I have a vague memory about this - wasn't it to do with the time before the 
first data is sent to the client that was under discussion rather than the 
overall system throughput?

> In any case, if I'm correct, it should be a simple matter to do the sort
> of checkpoint/rollback mechanism that is being discussed.  

until someone used your #flush directive (or called flush() on the FastWriter 
inside one of their own directives).  Then all your assumptions about 
rollback go out the window - you'd have to restrict access to the flush 
method to prevent this happening...

> But I'm not 
> sure this is a good idea.  It would be adding more dependencies on the
> FastWriter and it seems like an overly complex error handling
> architecture.  We already have the pluggable EEH mechanism which seems
> pretty adequate.  Generally the error "culture" in WM is, if you
> anticipate errors, handle them in code.  Unanticipated errors should
> result in an error template being rendered.
>
> I think the bigger issue is with the FastWriter mechanism as a whole.
> It should be an interface so that you can plug in an implementation that
> serves your needs.  This is much more in line with the flexibility that
> permeates nearly every other aspect of WM.

I've just been having a quick scan through the source for which methods are 
used in FastWriter, and it is pretty much everything across quite a wide 
variety of situations.  If you were therefore going to make an interface from 
it then it would pretty much duplicate the structure of the current 
FastWriter API.  What alterations would you make to the implemenation of 
FastWriter if you did have an extendable structure?

My feeling is that an easier to implement and probably functionally more 
interesting approach would be to make the ByteBufferOutputStream into an 
interface with a Factory method that gets past to the FastWriter on 
construction.  Most of the changes that I could see being useful 
(auto-flushing to the output stream to save memory etc etc) would take place 
at this level in the code and wouldn't touch the FastWriter interface and 
therefore wouldn't mean refactoring pretty much everything else.

Alex

> My 2p.
>
> Keats
>
> Alex Twisleton-Wykeham-Fiennes wrote:
> >On Thu 9 March 2006 17:00, Endre Stølsvik wrote:
> >>On Wed, 8 Mar 2006, Eric B. Ridge wrote:
> >>| I've been thinking back to when I write IncludeDirective and I simply
> >>| cannot remember why it makes a new FastWriter.  Was it because the old
> >>| ParseDirective did that and I was just coping it?  Was it out of sheer
> >>| stupidity?  Or maybe some reason about not wanting to muck with the
> >>| real FastWriter in the event of a parsing/evaluation exception.  I
> >>| dunno.
> >>
> >>You should have commented it in the code.
> >>
> >>..
> >>
> >>One comment about the FastWriter/Writer thingy: A "resetable" writer
> >> would be fantastic, preferrably having unlimited resetable space (just
> >> growing adding byte-arrays into some list as needed).
> >>  I am responsible for a portal, which renders every portlet in its own
> >>context and with its own "StringWriter", so that if it throws an
> >>exception, I can throw it away, and instead render an error-portlet. If
> >>all goes well, I commit the fully rendered portlet to the FastWriter, and
> >>move along.
> >>
> >>However, this would be hugely better if I could just _always_ pass along
> >>the same fastwriter recursing down includes and macros and whatnots.
> >>Before I venture into the next portlet, i do a mark(). Then I let it
> >>render the portlet, with its recursing. Then if an exception turns up,
> >>I'll just do a reset(), and then output an error-template instead, while
> >>if it goes well, i'll do a _flush()_ to commit the stuff all the way to
> >>the browser at the other end of the stream (I feel this should be up to
> >>me, the webapp writer, to decide when to flush). That would have been
> >>absolutely great, and absolutely more performant.
> >
> >It's not going to be extending FastWriter because after a cursory look at
> > the code it appears to be caching small (<4k blocks) of binary data in
> > the FastWriter and then pushing them out to the underlying OutputStream
> > whenever it needs to get it's internal buffer into some kind of known
> > state.
> >
> >Therefore if it had already pushed data out to the OutputStream when you
> > tried to rollback your code to the last point then it would fail because
> > it would no longer have access to or control of the data.
> >
> >To do this properly you would need to maintain a single data structure
> > that holds the entire contents of whatever is written to it in memory and
> > then shifts it out once you've satisfied yourself that it isn't going to
> > need to rollback.
> >
> >I don't think that you are going to do this by extending FastWriter - it
> > just doesn't feel right.  Unfortunately great chunks of WebMacro are
> > expecting FastWriter as their target output stream so at present you are
> > kind of stumped.  Might be interesting to look at what the difference is
> > between Writer and FastWriter and refactor the calling code if it is not
> > too much hassle?
> >
> >However, the more I think about it, the more I feel that this "rolling
> > back" is not a nice idea, for the following reasons:-
> >
> >- the success / failure thing should be a javaspace thing and to only have
> > it occur inside the middle of a portlet feels a bit odd to me.  I kind of
> > feel that when the portlet rendering template is invoked, that the
> > context should already contain a set of calculated portlets (or error
> > portlets) that reflect the state of the system.  Then the rollback is
> > just not necessary.
> >
> >- rolling back the output is only half of the problem.  webmacro code can
> > make changes to the state of the backend java system and to the state of
> > the context.  Just because you roll the output stream back doesn't mean
> > that the java and context are in any way in a consistent state with what
> > they were before the rollback point.  To imply that they are is just
> > tempting fate...
> >
> >Just out of interest - how were you envisioning this rolling back
> > happening? Was it going to be a java space action or would it take place
> > during the template rendering phase?
> >
> >Alex
> >
> >
> >-------------------------------------------------------
> >This SF.Net email is sponsored by xPML, a groundbreaking scripting
> > language that extends applications into web and mobile media. Attend the
> > live webcast and join the prime developer group breaking into this new
> > coding territory!
> > http://sel.as-us.falkag.net/sel?cmd=k&kid0944&bid$1720&dat1642
> >_______________________________________________
> >Webmacro-user mailing list
> >[email protected]
> >https://lists.sourceforge.net/lists/listinfo/webmacro-user
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live
> webcast and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Webmacro-user mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/webmacro-user


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642