How should the image widget work?

Andy wrote this in another thread I suspect everyone's stopped reading 
by now, but because it's useful I'm going to start another thread to 
discuss it:
> I'd suck in libMagick (from imagemagick) and let that do all the grunt
> work. It should support any still image formats you can shake a stick
> at, and it has all the interesting transformations already. Not sure
> offhand whether to load it server- or client-side. Maybe it would be
> worthwhile to construct a widget that does *both*, and lets user-level
> server configuration pick which way to use (I have some plans for how
> this part will work; it's needed for just about everything and will
> probably be critical in the long run); I can think of scenarios where
> both forms would be useful.

Indeed.  However I can think of scenarios where the server-side version 
is more dangerous.

 From a performance standpoint it's best to keep the image in its 
original form for as long as possible, as this will allow any display 
driver with (full or partial) hardware whatever-it-is decoding 
facilities to take full advantage of them.  This is becoming more of a 
factor as Pixel Shader technology gets more advanced and it becomes 
possible to code your own decode routines to run in the graphics 
processing system rather than consuming general resources.  Even 
ignoring that, most cards from the original Radeon onwards have an 
onboard iDCT circuit which can be used to help decompress JPEGs.  I'm 
not suggesting we try and write such a driver yet, just that the 
architecture we use should allow us to do so in the future.

The biggest obstacle I can see to server-side image rendering is that it 
would be relatively easy for us (or of course the libMagick crew) to 
introduce a bug which allowed one client to alter the image resources of 
another.  This would be a big security problem (as you could 
misrepresent data), and would be less likely to crop up if we had a 
simpler server-side widget and put the complexity in the client.  The 
recent(ish) JPEG-rendering-related security holes in a lot of software 
show how easy it is for this kind of thing to creep in.

I'm not suggesting we're more likely to introduce security holes than 
anyone else, just that we're no less likely to do so.

A compromise would be to use the server-side widget if the uid of the 
client process matches the uid of the server process, that should at 
least stop malicious images interfering with privileged programs.

Thoughts?

Phil