Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Wed, 16 Sep 2009 13:25:47 -0400 Fortier,Vincent [Montreal] wrote: > > -----Message d'origine----- > > De : drbd-user-bounces at lists.linbit.com > > [mailto:drbd-user-bounces at lists.linbit.com] De la part de > > Diego Remolina > > Envoyé : 16 septembre 2009 09:17 > > > > > > linux bonding of _two_ nics in "balance-rr" mode, after some tuning > > > of the network stack sysctls, should give you about 1.6 to 1.8 x the > > > throughput of a single link. > > > For a single TCP connection (as DRBDs bulk data socket is), bonding > > > more than two will degrade throughput again, mostly due to packet > > > reordering. > > > > I've tried several bonding modes and with balance-rr the most > > I got was about 1.2Gbps using netperf tests. IIRC, the other > > issue of balance-rr is that there can be retransmission which > > slows down the transfers. > > > > Any specific information or howto accomplish the 1.6 to 1.8 x > > would be really appreciated. > > > > I am currently replicating two drbd devices over separate > > bonds in active backup mode (two bonds with 2 Gigabit > > interfaces each using mode=1 miimon=100). > > I've done some testing with 2 giga NIC's in different bonding modes and > balance-rr was the slowest one. > > I'd love to know how to achieve that speed because the way it is now it > will most probably be active-backup or simply a single nic setup. > I'd love to know how you managed that, seriously. It may be the type of NICs and kernel features like offloads that may or may not impair your performance here, but for me with 2 or in the current case 4 interfaces balance-rr always scaled as expected and stated above. balance-xor (note that this basically means one interface for a direct connected cluster) iperf results: --- Client connecting to 10.0.0.2, TCP port 5001 TCP window size: 27.4 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.0.1 port 45376 connected with 10.0.0.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.16 GBytes 992 Mbits/sec --- The same for balance-rr: --- Client connecting to 10.0.0.2, TCP port 5001 TCP window size: 27.4 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.0.1 port 36877 connected with 10.0.0.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 4.09 GBytes 3.51 Gbits/sec --- I suppose the degradation Lars is talking about up there is visible here, because compared to just 2 NICs it surely is less efficient. OTOH the additional 1.5Gb/s bandwidth are nice to have either way. Here is balance-rr with just 2 NICs in comparison: --- Client connecting to 10.0.0.2, TCP port 5001 TCP window size: 27.4 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.0.1 port 45944 connected with 10.0.0.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 2.30 GBytes 1.98 Gbits/sec --- Alas as stated in http://article.gmane.org/gmane.comp.linux.drbd/18259 despite of all that bandwidth goodness I still only about 115MB/s speeds with drbd instead of the 200MB/s (180MB/s would be nice enough ^o^) that the underlying RAID is capable of. Oh and doing a: ifdown bond0; echo 2 > /sys/class/net/bond0/bonding/mode; ifup bond0 changes the bonding mode just fine without the need for reboots. Regards, Christian -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/