Re: GLX in bad state

Nadav Frum wrote:
>>As we've stated in the past, we are very particular about the patches that
>>we allow into CVS, and we don't like checking things in that have either
>>known issues or that have known paths that might cause issues, unless there's
>>a very compelling reason for it.
> 
>  
> I think I know a bit about the WineX code.  If you have an invalid pointer 
> to an unimplemented function, WineX will crash when that function is 
> called.  Therefore, there are no negative issues with adding a stub.

The issue is not whether adding the stub would cause problems, it was whether
putting the stub in CVS would force him to waste time updating his unfinished
pixel shader patch.  The answer was yes, it did, but he did it anyway.

> Now perhaps you will come forward with some benchmark results.

I wasn't actually using the benchmark mode before, since visually I wasn't
getting more than about 2fps.  Leaving out the pbuffer->texture copy didn't
change that significantly.

Thanks to Ove's suggestion, I'm now getting 19fps.  More on this later.

The machine is an Athlon 2500 with a GeForce 5600 and 1 GB of RAM, my
home gaming box.

>>As Ove stated, this is the overall philosophy of the OpenGL design in
>>general, not ours.Obviously performance is very important to us.
> 
> How is the overall philosophy of the OpenGL design (whatever it may be) 
> relevant to this discussion?

It's relevant because you started the discussion raising the idea that
there should be a new OpenGL extension added to support render-to-texture
in GLX.

>>If you'd like to determine where there might be further performance issues
>>with your updated video memorymanagement code, you might try doing a trace
>>with +timestamp,+x11drv,+ddraw to narrow down what's actually taking up
>>time.
> 
> 
> This is a bad approach for the following reasons:
> 
> 1.  This level of debugging will slow the performance to such an extent 
>     that the overall analysis of performance will become irrelevant.

Not true.  Slowdown will generally be proportional, so it's unlikely to
distort the results.

> 2.  The overall performance is an accumulation of routines which on their 
>     own waste less CPU time than is measurable by the time stamp.

But the problem we see here, as I mentioned, is showing gaps of 200ms and
higher in bits of code within the glx.c code.

In general, that technique is good so long as lots of time is being spent
within a given Win32 API call from the app's perspective.  If an app has a
tight loop calling a Win32 API which takes less than one millisecond, it's
less useful.

> 3.  The performance may be slow because the graphics card is used in an 
>     inefficient way.  Such performance losses will occur when the video 
>     card executes the command and not when the command is issued.

This is impossible to determine with any standard profiler - for that you need
tools that know the hardware intimately.

> 4.  The timestamps will also include time spent executing the game's 
>     binaries and libraries other than x11drv and ddraw.

Yes, but not while we're within an API call.

> 5.  The game's binary may be doing some excess computation in response to 
>     unexpected feedback from WineX.  This will not be revealed.

Possible, but unlikely, and impossible to determine with any other kind of
profiler.

> A much better approach is a profiler like 'oprofile'.  It has the 
> limitation, however, that it does not show what the video card and the 
> games's binary is working on.  The information received from differently 
> spec'ed systems would benefit such an analysis.

Yes, we're quite aware of the different profiling tools available on the
system, having been involved in writing them.  Most Linux profilers are
nice in theory, but less good in practice.  That's why I designed cprof
at Corel, which was implemented by Andrew Lewycky, who later worked at
TransGaming.  Sadly, cprof also turned out to be less good in practice
(it generated too much output data too quickly, on the order of hundreds
of megs a minute).

Regardless, since I wanted to see how it was progressing, I had another
go at oprofile to see how it was doing.  Sadly, it hasn't advanced much
from the last time I looked at it.

oprofile is not yet capable of providing call graph profiles, though they're
working on it.  Once they get it going, we'll probably put some effort into
making sure it can read Win32 stacks as well as Linux stacks so that we can
get some useful data from the Windows side.

Currently. without call graph profiles, you end up with results that tell
you almost nothing useful, like this:

              7806 81.0676 libGLcore.so.1.0.5336
               764  7.9344 c4demo.exe
               200  2.0771 vmlinux
               183  1.9005 libd3dgl.so
               176  1.8278 libx11drv.so
               116  1.2047 libGL.so.1.0.5336
               111  1.1528 zero
               106  1.1008 nvidia

The culprit here is of course libx11drv, but you wouldn't know it from that
output.

oprofile also lacks the ability to extract names from stripped libraries
(ie: libGLCore.so.1.0.5336), so when we ask for a symbolic breakdown, we
simply don't get any of those symbols.  The best we can get, with the application
of a patch I've made to oprofile that gets the dynamic symbol previous in the
image to the point in question is something like this:

samples  %        image name               app name                 symbol name
6102     62.1765  libGLcore.so.1.0.5336    wine                     _nv000054gl
764       7.7848  c4demo.exe               wine                     (no symbols)
536       5.4616  libGLcore.so.1.0.5336    wine                     _nv000061gl
499       5.0846  libGLcore.so.1.0.5336    wine                     _nv000073gl
324       3.3014  libGLcore.so.1.0.5336    wine                     _nv000077gl

This is equally useless, though some asm poking shows that the driver is spending
all of its time copying some buffers around.  Some debugging back to the call point
of the slow routine shows that it's being called from (drumroll)....the GL call to
create a pbuffer, exactly what I said it was half a dozen emails ago.

With this confirmation, and Ove's additional comments on the caching scheme, I had
a go at increasing the cache size from 64 pbuffers to 128.  That instantly gave me
a 10x performance boost, as we were no longer running out of cached pbuffers.

So, what can we conclude:
  1) NVidia's drivers aren't good at allocating new pbuffers for some reason
  2) Over a scene's lifetime c4 is setting up lots of textures for rendering, and
     blowing our cache.  It's not clear how active these are though, as I mentioned
     earler that I'd only seen 7 such textures before.
  3) If some of these are not actively used for rendering during a scene, it's
     possible that our cache policy is to blame.  We may want to switch to a true
     LRU cache.  That could potentially eliminate the problem without requiring
     a significant increase in the cache size.
  4) It may be useful to set the default cache size larger in any case.

and, most importantly,

  6) Performance improvements aren't about how you profile, they're about how you
     analyse the data you have.

Anyhow, I'll leave it to Ove and others to decide whether the cache algorithm
needs significant changes, and about the arguments for and against upping the
cache size further or leaving it alone.

Take care,
  -Gav

-- 
Gavriel State
Co-CEO & CTO
TransGaming Technologies Inc.
[email protected]
http://www.transgaming.com

Let the games begin!