Re: OpenSSI & Infiniband

OpenSSI 1.9.1 was running, IPoIB or IB verbs, over the SourceForge IB
stack about a year and 1/2 ago. Many useful lessons were learned from
the experience.
 
OpenSSI 1.9.2 was running over the openIB.org IPoIB (svn.3796) as of
about a year ago. I started on the openIB.org IB verbs port but was way
layed, hence it was never finished. Possibly this summer I will be able
to reconstitute the project.
 
Lessons learned:
 
PXE/Etherboot over Infiniband doesn't exist, hence a mechanism had to be
invented to PXE boot over Ethernet and then switch to an Infiniband
interconnect. Mods were required to the initrd ramdisk along with the
initial /linux script which implies mods to mkinitrd script, along with
a shadow /etc/boottab.ib. All tractable problems which demonstrated a
need for more interconnect device independence - natural evolution.
 
Kernel CI config file needed work as the options for different CI
interconnect device choices didn't exist.
 
Running openSSI over InfiniBand IPoIB was pretty straightforward. A few
tweaks to some kernel header files, along with the boot ramdisk config
was all.
 
openSSI over Infiniband native verbs was much more interesting,
including RDMA reads at the CI layer; TCP/IP rates between nodes was
around 200MB/sec with the SourceForge stack - openIB.org transfer rates
will be much higher!
 
The introduction of multiple CI devices opens the door for a single
cluster built from multiple interconnects. CI routing needs to be
addressed.
 
Bottom line, openSSI over Infiniband is certainly possible as the
openSSI framework has proven flexible; some assembly required.
 
stan.
 

________________________________

From: [email protected]
[mailto:[email protected]] On Behalf Of
Hugo Segura
Sent: Saturday, June 10, 2006 5:16 AM
To: [email protected]
Subject: [SSI-users] OpenSSI & Infiniband


Hi Guys :

Is OpenSSI ready for Infiniband? Does anybody done that?

Any good info/url?

best regards,

Hugo Segura

_______________________________________________
Ssic-linux-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ssic-linux-users