Setting Up IP over IB

We have 1 dual port HCA card installed. Assuming port 2 (ib1) is up and port 1 (ib0) is down.

Step 1

In /etc/sysconfig/network-scripts, create:

$ cat ifcfg-ib1
DEVICE=ib1
#HWADDR=80:00:00:49:FE:80:00:00:00:00:00:00:00:02:C9:03:00:0F:D8:5E
TYPE=InfiniBand
UUID=48c746a0-5536-408f-ae8f-c36be67c66c7
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPADDR=192.168.11.1
NETMASK=255.255.255.0

Step 2

$ /etc/init.d/openibd restart
$ /etc/init.d/network restart

Step 3

Verify the connectivity:

$ ifconfig
# Check if ib1 is up and has IP assigned

$ ibdev2netdev

Set up another node but with different IP, e.g. 192.168.11.2. From this new node you can ping the older node by
$ ping 192.168.11.1