How to specify GID index for RDMA?

Hello All,
 
I’ve been trying to get an fio workload over RDMA working, where a limitation is that there is only ipv6 available for now, and that I have to specify GID index 2 in order to be able to use ipv4. For example, when using ib_write_bw, it looks like this:
 
softnic@ktu-vm1:~$ ib_write_bw -x 2 10.0.0.4 --port 18515
---------------------------------------------------------------------------------------
                    RDMA_Write BW Test
Dual-port       : OFF           Device         : roceP30832p0s0
Number of qps   : 1             Transport type : IB
Connection type : RC            Using SRQ      : OFF
PCIe relax order: ON
ibv_wr* API     : OFF
TX depth        : 128
CQ Moderation   : 1
Mtu             : 1024[B]
Link type       : Ethernet
GID index       : 2
Max inline data : 0[B]
rdma_cm QPs : OFF
Data ex. method : Ethernet
---------------------------------------------------------------------------------------
local address: LID 0000 QPN 0x0024 PSN 0xd4e107 RKey 0x00cd00 VAddr 0x007240fdfd2000
GID: 38:03:16:176:05:67:00:60:00:00:00:01:00:114:204:90
remote address: LID 0000 QPN 0x0024 PSN 0xb3e770 RKey 0x00cd00 VAddr 0x007bb8d3c51000
GID: 38:03:16:176:05:67:00:58:00:00:00:01:00:59:18:224
---------------------------------------------------------------------------------------
#bytes     #iterations    BW peak[MiB/sec]    BW average[MiB/sec]   MsgRate[Mpps]
Conflicting CPU frequency values detected: 2300.000000 != 3599.929000. CPU Frequency is not max.
65536      5000             10828.63            10828.55              0.173257
---------------------------------------------------------------------------------------
 
Here, -x specifies the GID index.
 
Now, when using fio, I couldn’t find a similar option in the documentation. The job files I used are:
 
Server:
[global]
ioengine=rdma
port=18515
bs=1m
size=100g
 
[receiver]
rw=read
iodepth=16
 
This appears to work:
 
receiver: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=rdma, iodepth=16
fio-3.28
Starting 1 process
fio: waiting for connection
 
On the client:

[global]
ioengine=rdma
hostname=10.0.0.5
port=18515
verb=write
bs=1m
size=100g
 
[sender]
rw=write
iodepth=1
iodepth_batch_complete=1
 
This eventually shows:
 
softnic@ktu-vm2:~$ fio fio_rdma_client.job
sender: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=rdma, iodepth=1
fio-3.28
Starting 1 process
Jobs: 1 (f=0): [W(1)][-.-%][eta 00m:00s]
 
After a while, I get:
 
fio: event is RDMA_CM_EVENT_UNREACHABLE instead of RDMA_CM_EVENT_ESTABLISHED
fio: wait for RDMA_CM_EVENT_ESTABLISHED
 
Given the limitations mentioned above, is it possible to get RDMA working in this scenario?
 
Rob