Redesign of copy_from_user()
Andreas Schäfer <[email protected]>
| Newsgroups | gmane.linux.cluster.openmosix.devel |
|---|---|
| Message-ID | <20060226043500.GC16375@wintermute> |
Hi list, as said while investigating the write() flaws I noticed that the kernel's data transfer from userspace copy_from_user() caused rather strange traffic signatures. When writing to a file, only one page at a time got transmitted via the network, so no real streaming is done. Constantly some control/arbitration packages (of length 74 bytes, i.e. 8 bytes cargo, the rest overhead) have to be exchanged. This explains the good, but still improvable write performance of 2.7-3.4 ms/s. I understand that copy_from_user() is normally considered as cheap and so it's ok to call it for every page, even more so when considering that the slow end of this line is the block device (i.e. hard disk). So transfer to the hard disk is buffered, as it's considered slow, and transfer from userspace is not as it's comparatively fast. However, with openMosix things change. All of a sudden we have _2_ slow ends as the memory access across the network is impaired by latency and limited bandwith. To improve on this I propose a new "double buffering" schema: upon write openMosix reserves a little buffer (which might grow or be shrunk later on) on remote. before running copy_from_user, this buffer is filled in one go by receiving from deputy. As the buffer is larger than one page, we can benefit from streaming and transfers will be sped up. Also by streaming the dependency on ultra low latency links would be mitigated. Of course there are downsides to this: - The implementation of hooking into copy_from_user() is really beautiful in it's present state. My proposal would severly butcher it by not being elegant and introducing more complexity. - An additional kernel-internal copy is necessary which raises CPU load (but I dunno how much this affects current systems). Still I'm convinced the advances prevail the disadvantages. We need to get round of limited write performance, this is even more important with gigabit ethernet (not the old school fast ethernet I'm having at home). What do you guys think of that? Any comments welcome. -Andreas ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642