[DRBD-user] connect error -22 with SDP/InfiniBand

J. Ryan Earl oss at jryanearl.us
Fri Sep 17 23:19:23 CEST 2010

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Hi again,

On Fri, Sep 17, 2010 at 3:11 PM, Lars Ellenberg
<lars.ellenberg at linbit.com>wrote:

> On Fri, Sep 17, 2010 at 02:12:35PM -0500, J. Ryan Earl wrote:
>  > Is this the fix to which you refer?
> > http://www.mail-archive.com/general@lists.openfabrics.org/msg10615.html
> That's certainly relevant as well,
> but it would have returned EAFNOSUPPORT, which is 97.

I'm going through the code and that patch is already included anyway.

> I doubt that this has anything to do with performance, btw,
> it is just the address lookup during connect.

Oh, that blows then.

> My guess is if you strace a netcat in userland, using your sdp preload
> thingy, you'll likely see that it only creates the socket as
> AF_INET_SDP, but all the rest of the network functions keep using
> AF_INET, so no-one ever noticed. If that is intentional, we'd have to
> adjust that in DRBD. If not, it needs to be fixed in the sdp stack.

It seems to intercept AF_INET.  The LD_PRELOAD doesn't change the strace
except for opening the shared object.

> DRBD over SDP performance tuning is a bit tricky,
> and no, I don't remember the details, it's been a while.
> I think cpu usage dropped considerable, thats a plus. But neither
> single write latency nor sequential throughput of a single connection
> improved much or even degraded respective to IPoIB.  If you have
> multiple DRBD, thus multiple connections, the cumulative throughput
> scaled better, though.

Actually CPU usage went up.  The wall time was 50% longer with 10% more CPU

> But please go ahead and tune your stack, on your hardware, which may be
> more capable than the test lab hardware we used.
> Depending on your hardware, and the quality recommendations of
> your SDP tuning expert, your findings may be different.

I'm not sure how to tune SDP, but I'll look into it.  ssocks when I tried
didn't seem to need tuning.  With 2 ports, I was getting better performance
than with IB and SDP, though it had other issues.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20100917/b2ee3f72/attachment.htm>

More information about the drbd-user mailing list