Redesign of copy_from_user()

Andreas Schäfer <[email protected]>
Newsgroups gmane.linux.cluster.openmosix.devel
Message-ID <20060226043500.GC16375@wintermute>
Hi list,

as said while investigating the write() flaws I noticed that the
kernel's data transfer from userspace copy_from_user() caused rather
strange traffic signatures. When writing to a file, only one page at a
time got transmitted via the network, so no real streaming is
done. Constantly some control/arbitration packages (of length 74
bytes, i.e. 8 bytes cargo, the rest overhead) have to be exchanged.

This explains the good, but still improvable write performance of
2.7-3.4 ms/s. I understand that copy_from_user() is normally
considered as cheap and so it's ok to call it for every page, even
more so when considering that the slow end of this line is the block
device (i.e. hard disk). So transfer to the hard disk is buffered, as
it's considered slow, and transfer from userspace is not as it's
comparatively fast.

However, with openMosix things change. All of a sudden we have _2_
slow ends as the memory access across the network is impaired by
latency and limited bandwith. 

To improve on this I propose a new "double buffering" schema: upon
write openMosix reserves a little buffer (which might grow or be
shrunk later on) on remote. before running copy_from_user, this buffer
is filled in one go by receiving from deputy. As the buffer is larger
than one page, we can benefit from streaming and transfers will be
sped up. Also by streaming the dependency on ultra low latency links
would be mitigated.

Of course there are downsides to this:
- The implementation of hooking into copy_from_user() is really
  beautiful in it's present state. My proposal would severly butcher
  it by not being elegant and introducing more complexity.
- An additional kernel-internal copy is necessary which raises CPU
  load (but I dunno how much this affects current systems).

Still I'm convinced the advances prevail the disadvantages. We need to
get round of limited write performance, this is even more important
with gigabit ethernet (not the old school fast ethernet I'm having at
home). What do you guys think of that? Any comments welcome.

-Andreas


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.