Re: Full page load test in stone ridge

Patrick, thanks, good answer.

I'm also more tending to have a talos-like testing on multiple network 
confings (that stoneridge has).  To somehow simulate parallel loads by 
some hard coded test is simply a useless effort to me.  We are dependent 
strongly on how things are organized on the main thread too and there 
are lots of rendering and layout operations that may block necko 
callbacks.  And that is significant and can't be obviously simulated 
just by some small js test.

Page usable is a good idea.  I think we can get information from layout 
that all what's visible has been rendered, like images. That is a start IMO.

-hb-

On 12/1/2012 4:52 PM, Patrick McManus wrote:
> nick, great mail. Thanks.
>
> It sounds like you're asking if we care more about network performance of
> firefox or network performance of standalone necko.
>
> While both are interesting, I feel strongly  that's a no brainer - I care
> about firefox a lot more than the necko library. You cite a bunch of
> reasons (and I can cite more) of where the interaction matters a lot.
> Efforts to replicate that interaction are, as you mention, likely going to
> be fragile and inaccurate and in the end don't really save anything.. all
> we end up doing is trading the testing of something that we care about
> (firefox content/layout) for something we don't (stoneridge docshell
> emulation code).
>
> hopefully 792438 illustrates strongly how much the interaction between
> content and necko matters to overall performance. I don't tihnk there is
> any way of getting around this.
>
> I acknowledge the obvious downside is that a regression (or improvement) in
> this test is not necessarily a necko regression - and that creates bugs
> that are harder to chase (even when they are necko bugs).. Some thought on
> how to make diagnosing these things easier from people better schooled in
> testing would be welcome (maybe the answer to subsequently add a layer of
> necko only tests that can be used as diagnostics but not tracked as
> metrics), but I'm pretty confident what we need to be optimizing for are
> real use cases on realistic networks and right now firefox doesn't have any
> of that at all. So that's what I would want to prioritize.
>
> Even tp5 numbers, using existing talos drivers, would be a huge step
> forward if we actually measured them on a variety of networks.
>
> Let's use 792438 as an example. This work was basically blocked on
> stoneridge until it became undeniable we had to do something sooner.  From
> anecdotal tests I am more than convinced that doing that work is the right
> thing to do - but how do we use stoneridge to pursue that in a more
> disciplined manner? What are the missing pieces to do that meaningfully
> compared to what we have in stoneridge now? I don't think it is a big list,
> but all of them are critical to having a metric that is worth optimizing
> for on that particular use case:
>
>     1. Firefox's loading pattern.. including the speculative stylesheet
>     loader
>     2. a better metric than "page loaded" (i.e. page usable)
>     3. a network model with configurable IW sizes
>     4. a network that models queue sizes for induced latency and loss rather
>     than just stochastic jitter and loss.
>
> And of course without push-to-stoneridge there is no way to do that kind
> metric driven development.
>
> I'm very concerned about boiling the ocean and ending up with nothing. But
> in many cases (pinterest, 136, cnn, etc..) those things all matter greatly
> to perceived performance.
> _______________________________________________
> dev-tech-network mailing list
> [email protected]
> https://lists.mozilla.org/listinfo/dev-tech-network
>