So here's a further update:<div><br></div><div>I tried uninstalling MLNX_OFED-1.5.1 and using the CentOS5.5 OFED, which is 1.4.x based. The SDP connection worked without a manual drbd-setup invocation. However, as Lars believed, the performance didn't appear affected by the connect() problem. SDP on the CentOS OFED was about 1-2% worse than the MLNX_OFED-1.5.1. SDP on both was about 65-70% of IPoIB throughput in DRBD. So SDP is performing at about two thirds of IPoIB for DRBD. Overall, SDP and IPoIB of the MLNX_OFED-1.5.1 outperformed the same protocol in the CentOS OFED by about 1-3% from what I saw.</div>
<div><br></div><div>Another interesting problem is that you can't bond back-to-back dual-port IB cards in 2 servers. Each port is a separate fabric when not connected to a switch, and ib-bonding only works if they are on the same fabric (ie switch). This is the opposite of the Dolphin Express cards, which combine the ports into a single fabric when back-to-back.</div>
<div><br></div><div>Any chance you would take a feature request to support multiple network interfaces for DRBD devices, even if active-passive only? =)</div><div><br></div><div>Cheers,</div><div>-JR</div>