<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi,<br>
<br>
Following your comments, you should be using <br>
address sdp <a moz-do-not-send="true"
href="http://192.168.20.1:7778">192.168.90.1:7778</a>;<br>
<br>
Instead of:<br>
address sdp <a moz-do-not-send="true"
href="http://192.168.20.1:7778">192.168.20.1:7778</a>;<br>
<br>
We have IB working in centos 5.5 too, without problems. Let me know
if it's not this!<br>
<br>
Have fun.<br>
<br>
Em 17-09-2010 00:29, J. Ryan Earl escreveu:
<blockquote
cite="mid:AANLkTimyy7+8y=YzSDXQo3dNjSvf4OSbb6+JCSs4khby@mail.gmail.com"
type="cite">Hello,
<div><br>
</div>
<div>I've recently setup an InfiniBand 40Gbit interconnect between
two nodes to run DRBD on top of some pretty fast storage. I am
able to get DRBD to work over Ethernet and IPoIB, however, when
I try to enable SDP for the lower latency, lower overhead
communication I'm getting connection errors:</div>
<div><br>
</div>
<div>
<div>
<div>block drbd0: conn( Unconnected -> WFConnection )</div>
<div>block drbd0: connect failed, err = -22</div>
<div>block drbd0: connect failed, err = -22</div>
<div>block drbd0: connect failed, err = -22</div>
</div>
<div><br>
</div>
<div>I have the MLNX_OFED installed on CentOS5.5 with SDP
active:</div>
<div><br>
</div>
<div># rpm -qa|grep sdp</div>
<div>libsdp-devel-1.1.100-0.1.g920ea31</div>
<div>sdpnetstat-1.60-0.2.g8844f04</div>
<div>
libsdp-1.1.100-0.1.g920ea31</div>
<div>libsdp-1.1.100-0.1.g920ea31</div>
<div>libsdp-devel-1.1.100-0.1.g920ea31</div>
<div>libsdp-debuginfo-1.1.100-0.1.g920ea31</div>
</div>
<div><br>
</div>
<div>
<div>[root@node02 log]# netperf -f g -H 192.168.20.1 -c -C</div>
<div>TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
192.168.20.1 (192.168.20.1) port 0 AF_INET</div>
<div>Recv Send Send Utilization
Service Demand</div>
<div>Socket Socket Message Elapsed Send Recv
Send Recv</div>
<div>Size Size Size Time Throughput local
remote local remote</div>
<div>bytes bytes bytes secs. 10^9bits/s % S % S
us/KB us/KB</div>
<div><br>
</div>
<div> 87380 65536 65536 10.00 16.15 1.74 4.61
0.211 0.562</div>
</div>
<div><br>
</div>
<div>
<div>[root@node02 log]# LD_PRELOAD="libsdp.so" netperf -f g -H
192.168.20.1 -c -C</div>
<div>TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
192.168.20.1 (192.168.20.1) port 0 AF_INET</div>
<div>Recv Send Send Utilization
Service Demand</div>
<div>Socket Socket Message Elapsed Send Recv
Send Recv</div>
<div>Size Size Size Time Throughput local
remote local remote</div>
<div>bytes bytes bytes secs. 10^9bits/s % S % S
us/KB us/KB</div>
<div><br>
</div>
<div> 87380 65536 65536 10.01 24.67 3.18 3.28
0.253 0.262</div>
</div>
<div><br>
</div>
<div>There is a significant (50-100%) increase in bandwidth and
decrease in latency using SDP instead of IPoIB, so even though
IPoIB works I'd like to use the SDP method.</div>
<div><br>
</div>
<div>Here's my node stanzas:</div>
<div><br>
</div>
<div><snip></div>
<div> # 90-subnet = Ethernet, 20-subnet = InfiniBand</div>
<div>
<div> on node01 {</div>
<div> address <a moz-do-not-send="true"
href="http://192.168.20.1:7778">192.168.20.1:7778</a>;</div>
<div># address sdp <a moz-do-not-send="true"
href="http://192.168.20.1:7778">192.168.20.1:7778</a>;</div>
<div># address <a moz-do-not-send="true"
href="http://192.168.90.1:7778">192.168.90.1:7778</a>;</div>
<div> }</div>
<div> on node02 {</div>
<div> address <a moz-do-not-send="true"
href="http://192.168.20.2:7778">192.168.20.2:7778</a>;</div>
<div># address sdp <a moz-do-not-send="true"
href="http://192.168.20.2:7778">192.168.20.2:7778</a>;</div>
<div># address <a moz-do-not-send="true"
href="http://192.168.90.2:7778">192.168.90.2:7778</a>;</div>
<div> }</div>
</div>
<div></snip></div>
<div><br>
</div>
<div>If I uncomment on the SDP-based address, I get connection
error -22 and haven't been able to surmise what that means. Any
ideas? Running the lastest DRBD:</div>
<div><br>
</div>
<div>
<div># rpm -qa|grep drbd</div>
<div>drbd83-8.3.8.1-1</div>
<div>kmod-drbd83-8.3.8.1-1</div>
</div>
<div><br>
</div>
<div>Any help would be appreciated,</div>
<div>-JR</div>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
drbd-user mailing list
<a class="moz-txt-link-abbreviated" href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a>
<a class="moz-txt-link-freetext" href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Igor Neves <a class="moz-txt-link-rfc2396E" href="mailto:igor.neves@3gnt.net"><igor.neves@3gnt.net></a>
3GNTW - Tecnologias de Informação, Lda
SIP: <a class="moz-txt-link-abbreviated" href="mailto:igor@3gnt.net">igor@3gnt.net</a>
MSN: <a class="moz-txt-link-abbreviated" href="mailto:igor@3gnt.net">igor@3gnt.net</a>
JID: <a class="moz-txt-link-abbreviated" href="mailto:igor@3gnt.net">igor@3gnt.net</a>
PSTN: 00351 252377120
</pre>
</body>
</html>