RE: Ecosystems cost of additional specs (RE: Please re ad - proposed WG termination)
Michael Krause <[email protected]> Wed, 07 Sep 2005 10:45:35 -0700
| Newsgroups | gmane.ietf.ipoib |
|---|---|
| Message-ID | <[email protected]> |
--===============1219312837== Content-Type: multipart/alternative; boundary="=====================_387654317==.ALT" --=====================_387654317==.ALT Content-Type: text/plain; charset="us-ascii"; format=flowed At 08:59 AM 9/7/2005, Bernard King-Smith wrote: >I don't include Windows as a user of IB because it is pretty much a >Ethernet only OS looking at its current market. The approach appears to be IB is provided via a third-party NDIS based driver. Perfectly reasonable which has been used for many technologies including Ethernet. This is what is delivered today by some companies. >This is where technology and marketing get into trouble. If IB is not on >par, then 10 GigE is all people will use. Even though IB has other non-IP >uses where it excells, when you look at total cost, two networks are more >expensive than one. Hence, IPoIB just good enough means no IB for most >customers. I don't agree that 10 GigE can be everything to everyone, and IB >has a place. Make it cost effective, and they will use it. IP is not going >away in the foreseeable future, and many a person/company has proclaimed >the death of IP over some newer technology, only to end up in the memories >of time. The only networks that survived for any length of time supported >IP well (Token ring, FDDI, ATM), even if not always fully utilized by an >application set. I don't see where IB is not on par. On the contrary, many will contend that IB is superior in many ways. The issue is whether there is a compelling reason to have a new spec defined for IP over IB so that it can compensate for the differences in performance between 10 GbE and a x4 IB 2.5 GT/s raw. Some contend this is required; perhaps I am alone in thinking it isn't. There is a complete IP over IB spec suite today and people have implemented a large percentage of it and demonstrated it is interoperable. Sounds like IB is therefore on par with Ethernet. Given 10 GbE Ethernet using PCI-X 265 can hit link rate without resorting to jumbo frames, there really isn't any reason why IB cannot hit its lower link rate using its slightly larger 2K MTU. The issue is in the implementation of HCA which is where I contend is the better solution approach to the problem. However, others contend they would rather spend resources defining / implementing a new spec rather than fixing the problem using / leveraging the approach already proven successful with Ethernet. This is really ROI issue. If people believe there is sufficient ROI, then they will implement it. Others, and again perhaps I'm the only one, would rather see resources spent in getting the RDMA infrastructure completed and view the IP over IB work as complete, i.e. it met the goals set forth by the workgroup since the first BOF occurred several years ago. As for Ethernet being everything to everyone, I think you have can go back to any number of pundits who will state that no matter what comes about, in the end it will be called Ethernet. So, yes today's Ethernet isn't your "father's Ethernet" but it sill inter-operates, still ships hundreds of millions of units per year, will see 10 GbE ship in the 100K's in the next year and ramp to millions in the next 3-4 years (even by conservative estimates), and will still have the lowest cost of entry from a total customer cost perspective for many years to come. This is why if IB is to make in roads, the focus needs to be on the RDMA ecosystem and not on tweaking some corner aspect of IP over IB. 10 GbE RNIC are real. They will ship next year with PCI-X 266 and PCIe points of attach (IB is only PCIe) so there isn't any advantage to IB any longer in that regard. 200ns 10 GbE with good port counts switch chips are real and at very reasonable prices. Companies who know how to deliver very low-latency, high-bandwidth switch and NICs such as Quadrics and Myricom are creating dual-hardware models that will deliver 10 GbE with full support for RDMA and link-rate solutions. These companies are hardly new to the HPC market - being the performance and volume leaders - and they have the ability to quickly translate their implementation prowess into credible offerings. Given their track records, they have immediate customer trust that will be easily leveraged for pilot and production deployments. Given their known quality, this trust can be easily leveraged into the commercial market as they join established players like Cisco, Broadcom, etc. In any case, I think the debate has played itself out and the virtual hum says to proceed. So, I'll go quiet unless someone has a specific issue or wants to continue the debate for some other reason. Mark me down as not supporting the hum which is likely just me in this regard. Mike --=====================_387654317==.ALT Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <body> <font size=3D3>At 08:59 AM 9/7/2005, Bernard King-Smith wrote:<br><br> <blockquote type=3Dcite class=3Dcite cite=3D"">I don't include Windows as a user of IB because it is pretty much a Ethernet only OS looking at its current market.</font></blockquote><br> The approach appears to be IB is provided via a third-party NDIS based driver. Perfectly reasonable which has been used for many technologies including Ethernet. This is what is delivered today by some companies.<br><br> <blockquote type=3Dcite class=3Dcite cite=3D""><font size=3D3>This is where technology and marketing get into trouble. If IB is not on<br> par, then 10 GigE is all people will use. Even though IB has other non-IP<br> uses where it excells, when you look at total cost, two networks are more<br> expensive than one. Hence, IPoIB just good enough means no IB for most<br> customers. I don't agree that 10 GigE can be everything to everyone, and IB<br> has a place. Make it cost effective, and they will use it. IP is not going<br> away in the foreseeable future, and many a person/company has proclaimed<br> the death of IP over some newer technology, only to end up in the memories<br> of time. The only networks that survived for any length of time supported<br> IP well (Token ring, FDDI, ATM), even if not always fully utilized by an<br> application set.</blockquote><br> I don't see where IB is not on par. On the contrary, many will contend that IB is superior in many ways. The issue is whether there is a compelling reason to have a new spec defined for IP over IB so that it can compensate for the differences in performance between 10 GbE and a x4 IB 2.5 GT/s raw. Some contend this is required; perhaps I am alone in thinking it isn't. There is a complete IP over IB spec suite today and people have implemented a large percentage of it and demonstrated it is interoperable. Sounds like IB is therefore on par with Ethernet. Given 10 GbE Ethernet using PCI-X 265 can hit link rate without resorting to jumbo frames, there really isn't any reason why IB cannot hit its lower link rate using its slightly larger 2K MTU. The issue is in the implementation of HCA which is where I contend is the better solution approach to the problem. However, others contend they would rather spend resources defining / implementing a new spec rather than fixing the problem using / leveraging the approach already proven successful with Ethernet. This is really ROI issue. If people believe there is sufficient ROI, then they will implement it. Others, and again perhaps I'm the only one, would rather see resources spent in getting the RDMA infrastructure completed and view the IP over IB work as complete, i.e. it met the goals set forth by the workgroup since the first BOF occurred several years ago. <br><br> As for Ethernet being everything to everyone, I think you have can go back to any number of pundits who will state that no matter what comes about, in the end it will be called Ethernet. So, yes today's Ethernet isn't your "father's Ethernet" but it sill inter-operates, still ships hundreds of millions of units per year, will see 10 GbE ship in the 100K's in the next year and ramp to millions in the next 3-4 years (even by conservative estimates), and will still have the lowest cost of entry from a total customer cost perspective for many years to come. This is why if IB is to make in roads, the focus needs to be on the RDMA ecosystem and not on tweaking some corner aspect of IP over IB. 10 GbE RNIC are real. They will ship next year with PCI-X 266 and PCIe points of attach (IB is only PCIe) so there isn't any advantage to IB any longer in that regard. 200ns 10 GbE with good port counts switch chips are real and at very reasonable prices. Companies who know how to deliver very low-latency, high-bandwidth switch and NICs such as Quadrics and Myricom are creating dual-hardware models that will deliver 10 GbE with full support for RDMA and link-rate solutions. These companies are hardly new to the HPC market - being the performance and volume leaders - and they have the ability to quickly translate their implementation prowess into credible offerings. Given their track records, they have immediate customer trust that will be easily leveraged for pilot and production deployments. Given their known quality, this trust can be easily leveraged into the commercial market as they join established players like Cisco, Broadcom, etc. <br><br> In any case, I think the debate has played itself out and the virtual hum says to proceed. So, I'll go quiet unless someone has a specific issue or wants to continue the debate for some other reason. Mark me down as not supporting the hum which is likely just me in this regard.<br><br> Mike</font></body> </html> --=====================_387654317==.ALT-- --===============1219312837== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPoverIB mailing list [email protected] https://www1.ietf.org/mailman/listinfo/ipoverib --===============1219312837==--