Video drivers in kernel space? (Re: new device driver features)

I have little experience in video hardware, but i've always thought that
keeping the drivers out of the X server, with the kernel, would be great.
Other applications could use them, the X server wouldn't need special
privileges, we could easily have an accelerated framebuffer for embedded
devices (without writing again all the drivers) and for games or other
fullscreen applications that do not use X...

With the current XFree, porting the current drivers is probably not
very practical, but now that you are evaluating to rewrite the
architecture and develop new drivers, it seems a great moment to think
into that question.

The big drawback is that we need a different set of drivers for Linux,
FreeBSD, etc. I'm sure there are other drawbacks that i am not
aware of, but i personally really would like the drivers to go into the
kernel. Is there any possibility for doing this?

-- 
Roberto Gordo - Free Software Engineer
Linalco "Especialistas en Linux y Software Libre"
http://www.linalco.com/  Tel: +34-914561700