Re: Patch to align virt and phys spaces on i386

On Thu, 9 Mar 2006, Glenn Brown wrote:
> Since our NIC interface requires O(1MB) of buffers, there seems to be no 
> way our new myri10ge.c driver could be compatible with -DKEEP_IT_REAL, 
> where virt_to_phys works only for the <64KB data segment.  Welcome to 
> the gritty world of high performance computing hardware.

Does the NIC really *require* 1MB of buffers, or does that just make it 
easier for you to convert the existing driver?

I have a driver for a high-performance 10GbE card (not yet committed), 
where the Linux driver routinely allocates O(36MB) of receive buffers, yet 
the Etherboot driver requires only 32kB.  This is achieved by keeping the 
RX descriptor ring almost empty; only 16 buffers are pushed in at any one 
time.

> Could you briefly explain the motivation for KEEP_IT_REAL? Specifically, 
> what's the down side of not supporting it,

We break code that expects to be able to use the PXE stack in accordance 
with the PXE spec.  So far, we've broken two different *BSDs; one tried to 
call us in V86 mode, one assumed that we wouldn't change the segment 
limits.

> how does it relate to EFI now that Intel claims EFI has superceded PXE,

I haven't looked at the EFI spec yet.  However, given that we're still 
stuck with having to work around 20-year-old issues such as the A20 line, 
I can't see PXE and all its ugliness vanishing any time soon.

> >  I'd prefer to keep it down to just one mechanism for simplicity.
> I do sympathize with the desire to have a single alignment mechanism for 
> simplicity.  I also sympathize with folks trying to port drivers from 
> the Solaris+Linux+FreeBSD+Irix+HP-UX+MacOS+VxWorks+NT world, which all 
> have this 4K alignment property.  So, do we want a mechanism that allows 
> easy porting at the cost of KEEP_IT_REAL compatibility, or a policy that 
> forces signifcant driver changes on such ports, even when the driver can 
> never work with KEEP_IT_REAL, to minimize KEEP_IT_REAL 
> incompatibilities?

We want just one mechanism.  We want people porting drivers not to just 
copy and paste the source, because that generally results in drivers that 
are very suboptimal for Etherboot's environment.  Forcing significant 
driver change is arguably a good thing.

> Our unusual myri10ge.c driver's large memory requirements prevent it 
> from using buffers in the .bss segment because it causes a linker error 
> (code too large).  Therefore, I'm forced to allot() the memory.  I'm 
> sure this is totally incompatible with KEEP_IT_REAL, and I expect our 
> large memory requirement would break even __shared, so I believe I have 
> no reasonable alternative to allot().

I'd reconsider your memory requirements; are they actually requirements? 
I would be prepared to place reasonable amounts of money on the bet that 
the driver can be reworked to require an order of magnitude less memory.

Hope that helps,

Michael


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642