"floccinaucinihilipilification", or "convolutions, blocking, and windows server 2003"

The code at http://www.dspaudio.com/~jaymz/convolution-20040313.tar.gz is
still incomplete, and not at all sanitised for your pleasure.  If you run 
this on your xserver, a big scary monster is guarenteed to come and eat you.  
Feet first.

I figure that some of the things I'm doing in there will be contriversial, so
here is some justifications:

 - fb/convolution/fbConvolutionGeneral.c/handleConvolution(): 

 Most of this code is tied up with not believing the region we've been given by
 fbComposite.  Why would we do this?  Well, try dragging a blurred window, and 
 it will become obvious.  (The cache should be disabled if ram-is-expensive is
 set, but that's not in this particular file right now, becuase i forgot about
 it until just now).

 - fb/convolution/generator/*

 Yes, I know that writing a code generator is a horrific way to optimize.  
 This particular code generator is very preliminary, but it generates 
 significantly faster code than the C it mimics.  There is no vectorisation
 in either case, however, so it still could be significantly faster if we
 used that.  GCC even supports portable vectorisation now, so perhaps I
 should look into that.  (I have a few more optimizations in my local tree,
 which are so horrific that I couldn't bear to present them :-p.  The cool
 one, though, was a little module that generated code, called the system
 compiler, and dlopen()'d it :-p.  Yes, it did help, but it's a maintenance
 nightmare, of course :().

 - fb/convolution/generator/fbGaussianBlur.c

 This is completly bogus and wrong.  The matrix it creates is actually 
 completly incorrect.  I am aware of this.  But there is a faster way to
 do a gaussian blur anyhow, using two passes like FastBlur does.  I just
 need to pull out a graphics book and grab the right formula for the 
 matrix.

 - generally

 Repeat does not work.  At all.  This is simple enough to fix, given the
 WrapClamp code (just add a WrapRepeat that does a modulus instead).  There
 aren't enough flexable ops.  There are ways for clients to crash this, 
 because ExecuteConvolution isn't bounds checking.  ExecuteConvolution
 also doens't honour clipping regions.  I still havn't implented the
 XFixesGrowRegion stuff that I meant to do a couple of weeks ago, because
 I havn't decided on what the "correct" API is for it.

One thing of note about this, is that a lot of the ops available are quite 
complicated ops, which can often block the xserver for seconds at a time.
This is mega painful.

I assume that everything which comes after here isn't new information to
most people on this list.

Obvious statment that is unrealistic: The xserver should never, ever, ever 
block based on a user app.  In the real world, however, some things depend
on other things, and will block, of course.

Contriversial statement: In reality, The xserver should never, ever, ever, 
block the root window based on a uesr app.  No, just making ops really fast
by throwing hardware at the problem does not achieve this.

Story time.  I run a couple of applications which blit a lot, specifically
dosbox, which demands to run at 60/70fps, and vice, which demands to run
at 50fps.  This is not a large problem for my CPU, at least not without
compositing, which this story does not contain any of.

I also run konsole, with a lot of tabs open, of course.  

so I'm running a program in a window in dosbox (amnesia, but the program
is irrelivent).  And then i switch tabs in konsole.  Even though the windows
are not overlapping, dosbox can't blit while konsole is redrawing, and 
so when I switched tabs in konqueror, dosbox dropped 20 frames.  

Additionally, whenever this happens, the mouse stops responding, because
the xserver is blocked.  This isn't so noticable, because you're clicking 
the mouse and not dragging, of course, but i have a theory that the 
choppiness that still exists when dragging windows is related.

The reason I'm interested in, is that some of the things I want to do
server side, convolutions included, hit this hard.

I have several ideas about what to do about it, and I expect all of them
to be flame bait.

Least contraversial: Have two seperate event queues, one for drawing ops,
and one for events.  have a thread process each.  yes, i am aware that
threads are evil, btw.  Completly useless, because you still need to draw,
but if you havea hardwre cursor, at least you can move it, I guess.  Since
my experiments with this were with Xsdl, which doesn't have any acceleration,
I deem this completly useless, but others may not.

More contraversial: Have a seperate thread per application connection.  
Lock drawables based on regions, and all other resources as whole entities.
Have seperate read/write locks, so that you can have multiple readers of a 
region of a drawable, but only one writer.  This means that apps can only
lock other apps if they're overlapping.  This is also completly useless under
the new world order(tm), though, because xcompmgr is a single app, but at least
apps can't rlock the server by doing a complex render op.  I havn't actually
done any experiements with this, so I don't know how it would go in the
real world.  This also breaks your DDX, unless you have a big lock around it.

Outright flamebait: Have a single thread for reading posted ops from apps, 
but then have possibly expensive ops marshall threads from a pool, implenting
the locking described above.  XSync() now waits for all threads to complete
processing.  I have expermented with this somewhat (although again only with
the SDL server), and my experiments indicated that it actually makes a huge 
difference to interactive performance, albiet at a nontrivial benchmark 
performance cost.

I can implent 'lagged convolutions' now, by keeping an eye on the CPU, and only 
delaying the convolution update until the server is idle.  But this isn't cool,
and would still cause blocking on big ops etc etc.  and would reuiqre a redraw,
of course, of the area above it etc etc.  But this does not solve the base 
problem.  Is it worth me persuing this further, or are threads one of those
things that we never ever ever will use(tm)?

I'm still learning to mentally task switch between the windows shop where I 
work (they not only demand that I use IE and Outlook, they also demand 
that I use them on my home desktop with windows 2000.  Fuck that for a 
game of ponies), so progress on anything right now is slow, as refrenced by the
fact that I've sat on the tarball above for two weeks.  Unfortunantly for you
guys, this is one of the projects i've decided to not cut out of my existance
due to my whoring of self to consulting scum (we convinced three schools to move
from mozilla to IE6.  aren't we proud?), so i expect a fun flamewar over all
of this.  But I'll be probably slow to reply during the week, until I master
this whole managing work+free software thing.

	-- jj

-- 
Jaymz Julian - Coder, Visionary, Fat Ass.
"Hannibal is a serial killer. He only likes to kill and eat people. 
 Very few people have `I want to be killed and eaten' on their cards, 
 so Hannibal is out of a job." - http://cards.sf.net