Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi,
Following your comments, you should be using
address sdp 192.168.90.1:7778 <http://192.168.20.1:7778>;
Instead of:
address sdp 192.168.20.1:7778 <http://192.168.20.1:7778>;
We have IB working in centos 5.5 too, without problems. Let me know if
it's not this!
Have fun.
Em 17-09-2010 00:29, J. Ryan Earl escreveu:
> Hello,
>
> I've recently setup an InfiniBand 40Gbit interconnect between two
> nodes to run DRBD on top of some pretty fast storage. I am able to
> get DRBD to work over Ethernet and IPoIB, however, when I try to
> enable SDP for the lower latency, lower overhead communication I'm
> getting connection errors:
>
> block drbd0: conn( Unconnected -> WFConnection )
> block drbd0: connect failed, err = -22
> block drbd0: connect failed, err = -22
> block drbd0: connect failed, err = -22
>
> I have the MLNX_OFED installed on CentOS5.5 with SDP active:
>
> # rpm -qa|grep sdp
> libsdp-devel-1.1.100-0.1.g920ea31
> sdpnetstat-1.60-0.2.g8844f04
> libsdp-1.1.100-0.1.g920ea31
> libsdp-1.1.100-0.1.g920ea31
> libsdp-devel-1.1.100-0.1.g920ea31
> libsdp-debuginfo-1.1.100-0.1.g920ea31
>
> [root at node02 log]# netperf -f g -H 192.168.20.1 -c -C
> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.20.1
> (192.168.20.1) port 0 AF_INET
> Recv Send Send Utilization Service
> Demand
> Socket Socket Message Elapsed Send Recv Send
> Recv
> Size Size Size Time Throughput local remote local
> remote
> bytes bytes bytes secs. 10^9bits/s % S % S us/KB
> us/KB
>
> 87380 65536 65536 10.00 16.15 1.74 4.61 0.211
> 0.562
>
> [root at node02 log]# LD_PRELOAD="libsdp.so" netperf -f g -H 192.168.20.1
> -c -C
> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.20.1
> (192.168.20.1) port 0 AF_INET
> Recv Send Send Utilization Service
> Demand
> Socket Socket Message Elapsed Send Recv Send
> Recv
> Size Size Size Time Throughput local remote local
> remote
> bytes bytes bytes secs. 10^9bits/s % S % S us/KB
> us/KB
>
> 87380 65536 65536 10.01 24.67 3.18 3.28 0.253
> 0.262
>
> There is a significant (50-100%) increase in bandwidth and decrease in
> latency using SDP instead of IPoIB, so even though IPoIB works I'd
> like to use the SDP method.
>
> Here's my node stanzas:
>
> <snip>
> # 90-subnet = Ethernet, 20-subnet = InfiniBand
> on node01 {
> address 192.168.20.1:7778 <http://192.168.20.1:7778>;
> # address sdp 192.168.20.1:7778 <http://192.168.20.1:7778>;
> # address 192.168.90.1:7778 <http://192.168.90.1:7778>;
> }
> on node02 {
> address 192.168.20.2:7778 <http://192.168.20.2:7778>;
> # address sdp 192.168.20.2:7778 <http://192.168.20.2:7778>;
> # address 192.168.90.2:7778 <http://192.168.90.2:7778>;
> }
> </snip>
>
> If I uncomment on the SDP-based address, I get connection error -22
> and haven't been able to surmise what that means. Any ideas? Running
> the lastest DRBD:
>
> # rpm -qa|grep drbd
> drbd83-8.3.8.1-1
> kmod-drbd83-8.3.8.1-1
>
> Any help would be appreciated,
> -JR
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
--
Igor Neves<igor.neves at 3gnt.net>
3GNTW - Tecnologias de Informação, Lda
SIP: igor at 3gnt.net
MSN: igor at 3gnt.net
JID: igor at 3gnt.net
PSTN: 00351 252377120
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20100917/84dd8933/attachment.htm>