[DRBD-user] Directly connected GigE ports bonded together no switch

Bart Coninckx bart.coninckx at telenet.be
Wed Aug 10 21:34:48 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On 08/10/11 19:04, Herman wrote:
>> On 2011-08-09 16:46, Herman wrote:
>>> Sorry if this is covered elsewhere.
>>>
>>> I know the Linux Bonding FAQ is supposed to talk about this, but I
>>> didn't see anything specific in it on what parameters to use.
>>>
>>> Basically, I want to bond two GigE ports between two servers which are
>>> connected with straight cables with no switch and use them for DRBD.
>>>
>>> I tried the various bonding modes with "miimon=100", but none of them
>>> worked. Say the eth1 ports on both servers were cabled together, and the
>>> same for eth5.  Then,  I could create the bond with eth1 and eth5.
>>> However, if I downed one of the ports on one server, say eth1, it would
>>> failover on that server to eth5, but the other server would not
>>> failover  to eth5.
>>>
>>> Eventually, I decided to use "arp_interval=100" and "arp_ip_target=<ip
>>> of other bonded pair>"  instead of "miimon=100".  This seems to work as
>>> I expected, with the bond properly failing over.
>>>
>>> Is this the right way to do this kind of bonding?
>>>
>>> Also, right now I'm using "mode=active-backup".  Would one of the other
>>> modes allow higher throughput and still allow automatic failover and
>>> transparency to DRBD?
>>
>> use balance-rr and e.g. miimon=100, that should do fine
>>
>> Regards,
>> Andreas
>
> Andreas and Andi,
>
> Thanks for your suggestions to use balance-rr.  I did try balance-rr
> with miimon=100; however, it didn't seem to work the way I wanted it to.
> Perhaps the way I was testing it isn't proper for miimon?
>
> I attempted to make one of the two links fail by doing "ifconfig eth3
> down"  This appeared to work find on the server I ran that on.  I could
> still ping the other server.  However, from the 2nd server, when I ping
> the 1st, I lost every other packet.
> Checking /proc/networking/bonding/bond2 showed that it still thought
> that both links were up.
>
> Is this because miimon still thinks a port is good if there is a cable
> and a powered NIC on both ends, and it doesn't care if th other NIC
> isn't responding?
>
> And arp monitoring works because it actually checks the reachability of
> the target IP.
>
> If this is the case, maybe arp monitoring is more reliable for direct
> connections since NIC failure (which may fail but still have link up) is
> more likely than cable failure?  Maybe I don't have a good understanding
> of this.
>
> In addition, I tried to use scp to test the throughput through the
> bonded link, but I actually got almost the same results via
> active-backup as with balance-rr.  Am I doing something wrong?
>
> Thanks,
> Herman
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user

I noticed only improvement on SLES11 after tuning the tcp_reordering 
parameter.

B.




More information about the drbd-user mailing list