Re: Patch to align virt and phys spaces on i386

Glenn Brown <[email protected]> Thu, 09 Mar 2006 15:42:44 -0800
Newsgroups gmane.network.etherboot.devel
Message-ID <[email protected]>
Michael,

Thanks for your feedback.  I'd like to fully understand your concerns, 
so I can Do the Right Thing.

SourceForge did not return any hits for "KEEP_IT_REAL" in the [Etherboot 
developers] archives, so I checked out 5.5 and read the KEEP_IT_REAL parts.

Since our NIC interface requires O(1MB) of buffers, there seems to be no 
way our new myri10ge.c driver could be compatible with -DKEEP_IT_REAL, 
where virt_to_phys works only for the <64KB data segment.  Welcome to 
the gritty world of high performance computing hardware.

Could you briefly explain the motivation for KEEP_IT_REAL? Specifically, 
what's the down side of not supporting it, what's it for other than 
strict PXE compliance, how does it relate to EFI now that Intel claims 
EFI has superceded PXE, and where can I get a copy of the PXE spec now 
that Intel no longer distributes it?

 > I'd prefer to keep it down to
 > just one mechanism for simplicity.

I do sympathize with the desire to have a single alignment mechanism for 
simplicity.  I also sympathize with folks trying to port drivers from 
the Solaris+Linux+FreeBSD+Irix+HP-UX+MacOS+VxWorks+NT world, which all 
have this 4K alignment property.  So, do we want a mechanism that allows 
easy porting at the cost of KEEP_IT_REAL compatibility, or a policy that 
forces signifcant driver changes on such ports, even when the driver can 
never work with KEEP_IT_REAL, to minimize KEEP_IT_REAL incompatibilities?

I'm inclined to choose mechanism over policy.  I concede it invites the 
"broken window phenomenon" as far as KEEP_IT_REAL is concerned, and will 
attempted to mitigate that with stategically placed warnings in our 
myri10ge.c driver.  I think driver writers should have a[n informed] 
choice; but I'm biased, of course, since I have a working and fully 
tested driver with the 4KB relocate.patch, and there appears to be no 
benefit to our driver of using a different alignment strategy.

 > I accept that the current approach, in which each driver allocates 
more space than it needs and then slides its buffers to align them, is 
inefficient and duplicative.  In 5.5, all data buffers are part of the 
shared buffer space, so maybe we could implement a mechanism whereby we 
refer to this space via a global pointer that gets maximally aligned by 
shared (non-driver) code.  This would give drivers aligned buffers while 
removing the duplicate code and maintaining the option of PXE compliance.


Our unusual myri10ge.c driver's large memory requirements prevent it 
from using buffers in the .bss segment because it causes a linker error 
(code too large).  Therefore, I'm forced to allot() the memory.  I'm 
sure this is totally incompatible with KEEP_IT_REAL, and I expect our 
large memory requirement would break even __shared, so I believe I have 
no reasonable alternative to allot().

In light of all this, would you discourage me from committing 
reloc.patch?  If so, why?

Thanks for all the information and feedback,
--Glenn


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642