Recent comments about FCoE and iSCSI

Dear All,

The trade press is lately full with comments about the latest and greatest 
reincarnation of Fiber Channel over ethernet.
It made me try and summarize all the long and hot debates that preceded 
the advent of iSCSI.
Although FCoE proponents make it look like no debate preceded iSCSI that 
was not so - FCoE was considered even then and was dropped as a dumb idea.

Here is a summary (as afar as I can remember) of the main arguments. They 
are not bad arguments even in retrospect and technically FCoE doesn't look 
better than it did then.

Feel free to use this material in a nay form. I expect this group to 
seriously  expand my arguments and make them public - in personal or 
collective form.

And do not forget - it is a technical dispute - although we all must have 
some doubts about the way it is pursued.

Regards,
Julo

---------------------------------------------------------------------

What a piece of nostalgia :-)

Around 1997 when a team at IBM Research (Haifa and Almaden) started 
looking at connecting storage to servers using the "regular network" (the 
ubiquitous LAN) we considered many alternatives (another team even had a 
look at ATM - still a computer network candidate at the time). I won't get 
you over all of our rationale (and we went over some of them again at the 
end of 1999 with a team from CISCO before we convened the first IETF BOF 
in 2000 at Adelaide that resulted in iSCSI and all the rest) but some of 
the reasons we choose to drop Fiber Channel over raw Ethernet where 
multiple:

Fiber Channel Protocol (SCSI over Fiber Channel Link) is "mildly" 
effective because:
it implements endpoints in a dedicated engine (Offload)
it has no transport layer (recovery is done at the application layer under 
the assumption that the error rate will be very low)
the network is limited in physical span and logical span (number of 
switches)
flow-control/congestion control is achieved with a mechanism adequate for 
a limited span network (credits). The packet loss rate is almost nil and 
that allows FCP to avoid using a transport (end-to-end) layer
FCP she switches are simple (addresses are local and the memory 
requirements cam be limited through the credit mechanism)
However FCP endpoints are inherently costlier than simple NICs ? the cost 
argument (initiators are more expensive)
The credit mechanisms is highly unstable for large networks (check switch 
vendors planning docs for the network diameter limits) ? the scaling 
argument
The assumption of low losses due to errors might radically change when 
moving from 1 to 10 Gb/s ? the scaling argument
Ethernet has no credit mechanism and any mechanism with a similar effect 
increases the end point cost. Building a transport layer in the protocol 
stack has always been the preferred choice of the networking community ? 
the community argument
The "performance penalty" of a complete protocol stack has always been 
overstated (and overrated). Advances in protocol stack implementation and 
finer tuning of the congestion control mechanisms make conventional TCP/IP 
performing well even at 10 Gb/s and over. Moreover the multicore 
processors that become dominant on the computing scene have enough compute 
cycles available to make any "offloading" possible as a mere code 
restructuring exercise (see the stack reports from Intel, IBM etc.)
Building on a complete stack makes available a wealth of operational and 
management mechanisms built over the years by the networking community 
(routing, provisioning, security, service location etc.) ? the community 
argument
Higher level storage access over an IP network is widely available and 
having both block and file served over the same connection with the same 
support and management structure is compelling ? the community argument
Highly efficient networks are easy to build over IP with optimal (shortest 
path) routing while Layer 2 networks use bridging and are limited by the 
logical tree structure that bridges must follow. The effort to combine 
routers and bridges (rbridges) is promising to change that but it will 
take some time to finalize (and we don't know exactly how it will 
operate). Untill then the scale of Layer 2 network is going to seriously 
limited ? the scaling argument


As a side argument ? a performance comparison made in 1998 showed SCSI 
over TCP (a predecessor of the later iSCSI) to perform better than FCP at 
1Gbs for block sizes typical for OLTP (4-8KB). That was what convinced us 
to take the path that lead to iSCSI ? and we used plain vanilla x86 
servers with plain-vanilla NICs and Linux (with similar measurements 
conducted on Windows).
The networking and storage community acknowledged those arguments and 
developed iSCSI and the companion protocols for service discovery, boot 
etc.

The community also acknowledged the need to support existing 
infrastructure and extend it in a reasonable fashion and developed 2 
protocols iFCP (to support hosts with FCP drivers and IP connections to 
connect to storage by a simple conversion from FCP to TCP packets) FCPIP 
to extend the reach of FCP through IP (connects FCP islands through TCP 
links). Both have been 
implemented and their foundation is solid.

The current attempt of developing a "new-age" FCP over an Ethernet link is 
going against most of the arguments that have given us iSCSI etc.

It ignores the networking layering practice, build an application protocol 
directly above a link and thus limits scaling, mandates elements at the 
link layer and application layer that make applications more expensive and 
leaves aside the whole "ecosystem" that accompanies TCP/IP (and not 
Ethernet).

In some related effort (and at a point also when developing iSCSI) we 
considered also moving away from SCSI (like some "no standardized" but 
popular in some circles software did ? e.g., NBP) but decided against. 
SCSI is a mature and well understood access architecture for block storage 
and is implemented by many device vendors. Moving away from it would not 
have been justified at the time.

_______________________________________________
Ips mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/ips