<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<META content="MSHTML 6.00.6001.18063" name=GENERATOR></HEAD>
<BODY>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2>Again, I believe that in 802.3ad when transmitting to the
same destination, packets will go over the same link.</FONT></SPAN></DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV align=left><SPAN class=405135818-19062008><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2>I think we tested
ftp throughput between two servers (connected via a cisco switch) using the
various bonding methods to see if any performance gains were there. I think the
answer was no because it was point to point however I note with interest the
snippet below which suggests the two different ports need to be in different
vlans. Not quite sure why that would be but I'll take it on face value
:)</FONT></SPAN></SPAN></DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2>The bottom line is that no aggregation of multiple "slow"
links will ever beat a single "fast" link. And there are
compromises/caveats/complexity along the way.</FONT></SPAN></DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2>We use link bonding for resilience across cisco 3750 stacks
as two switches can be treated as a single logical entity and when you drop a
switch or link there is almost no packet loss.</FONT></SPAN></DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2>10Gb-e is getting cheaper all the time. Intel are currently
offering a 2 for 1 deal on the 10Gb-e adapters we bought in for testing, so we
paid 700 GBP for a pair. Configuration was a simple matter of dropping in the
cards, building the kernel module and a wee bit of modprobe/kudzu , configure of
eth2 and bingo - in business.</FONT></SPAN></DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV align=left><SPAN class=405135818-19062008><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV><BR>
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader align=left>
<HR>
<FONT face=Tahoma size=2><B>From:</B> drbd-user-bounces@lists.linbit.com
[mailto:drbd-user-bounces@lists.linbit.com] <B>On Behalf Of </B>Sören
Malchow<BR><B>Sent:</B> 19 June 2008 17:05<BR><B>To:</B> Ralf
Gross<BR><B>Cc:</B> drbd-user-bounces@lists.linbit.com;
drbd-user@lists.linbit.com<BR><B>Subject:</B> Re: [DRBD-user] 10Gb ethernet
?<BR></FONT><BR></DIV>
<DIV></DIV><BR><FONT face=sans-serif size=2>Hi,</FONT> <BR><BR><FONT face=sans-serif size=2>i am sorry if i miss the point, but i did not read the
older messages.</FONT> <BR><BR><FONT face=sans-serif size=2>But why not use
802.3ad aka dynamic link aggregation, we have that successfully configured
with HP and Nortel switches ( also with machines running drbd over those links
).</FONT> <BR><BR><FONT face=sans-serif size=2>As far as i know cisco switches
also support 802.3ad.</FONT> <BR><BR><FONT face=sans-serif size=2>Regards</FONT> <BR><FONT face=sans-serif size=2>Soeren</FONT>
<BR><BR><BR><BR><BR>
<TABLE width="100%">
<TBODY>
<TR valign=top>
<TD width="40%"><FONT face=sans-serif size=1><B>Ralf Gross
<Ralf-Lists@ralfgross.de></B> </FONT><BR><FONT face=sans-serif size=1>Sent by: drbd-user-bounces@lists.linbit.com</FONT>
<P><FONT face=sans-serif size=1>19.06.2008 17:49</FONT> </P>
<TD width="59%">
<TABLE width="100%">
<TBODY>
<TR valign=top>
<TD>
<DIV align=right><FONT face=sans-serif size=1>To</FONT></DIV>
<TD><FONT face=sans-serif size=1>drbd-user@lists.linbit.com</FONT>
<TR valign=top>
<TD>
<DIV align=right><FONT face=sans-serif size=1>cc</FONT></DIV>
<TD>
<TR valign=top>
<TD>
<DIV align=right><FONT face=sans-serif size=1>Subject</FONT></DIV>
<TD><FONT face=sans-serif size=1>Re: [DRBD-user] 10Gb ethernet
?</FONT></TR></TBODY></TABLE><BR>
<TABLE>
<TBODY>
<TR valign=top>
<TD>
<TD></TR></TBODY></TABLE><BR></TR></TBODY></TABLE><BR><BR><BR><TT><FONT size=2>Lars Ellenberg schrieb:<BR>> On Thu, Jun 19, 2008 at 01:03:11PM
+0100, Lee Christie wrote:<BR>> > In any event, I'm no expert on channel
bonding, but in a 2-server<BR>> > configuration, where the Ips and MAC
addresses are fixed at either end,<BR>> > how can you use all 4 channels
? I was always under the impression that<BR>> > the bonding used an
algorithm based on src/dest IP/Mac to choose which<BR>> > link to send
data down, so in a point to point config it would always be<BR>> > the
same link. <BR>> <BR>> "balance-rr" aka mode 0 for linux bonding
schedules packets round robin<BR>> over the available
links.<BR><BR>balance-rr will not help in a cisco environment, because the
switch<BR>will still use the same ports and does not perform rr load
balancing.<BR>The only way to get rr working with cisco switches was to use
2<BR>different vlans.<BR><BR>eth0 <---- vlan x ----> eth0<BR>eth1
<---- vlan y ----> eth1<BR><BR>At least this is my experience and our
CCNP's (or what they are<BR>called) told me the same.<BR><BR>With the above
trick I was able to get ~1.6x GbE throughput with<BR>the netpipe benchmark
(after tuning the reorder kernel parameter). I<BR>didn't used the conenction
for drbd, I tried to speed up our backup.<BR>But the funny thing was, it
slowed down. Even the ftpd or samba was<BR>slower over this link as it was
with the xor mode or just one GbE NIC.<BR><BR>> but still, for a single tcp
connection, given some tcp_reorder tuning,<BR>> the strong gain you get
from 2x 1GbE (1.6 to 1.8 * that of one channel)<BR>> degrades again to
effectively less than one channel if you try to use 4x.<BR>> <BR>>
again, "more" is not always "better".<BR>> for the usage pattern of drbd
(single tcp connection with bulk data) the<BR>> throughput-optimum linux
bonding seems to be 2x, with 3x you are back to<BR>> around the same
throughput as 1x, with 4x you are even worse than 1x,<BR>> because packet
reordering over bonded GbE and tcp congestion control<BR>> don't work well
together for sinlge tcp links.<BR><BR>Very
true.<BR><BR>Ralf<BR>_______________________________________________<BR>drbd-user
mailing
list<BR>drbd-user@lists.linbit.com<BR>http://lists.linbit.com/mailman/listinfo/drbd-user<BR></FONT></TT><BR></BLOCKQUOTE></BODY><!--[object_id=#titaninternet.co.uk#]--><FONT face=Tahoma size=2><FONT color=#0000ff>
<P align=center>
<HR>
</P>
<P align=left>This email may contain legally privileged and/or confidential information. It is solely for and is confidential for use by the addressee. Unauthorised recipients must preserve, observe and respect this confidentiality. If you have received it in error please notify us and delete it from your computer. Do not discuss, distribute or otherwise copy it. </FONT></FONT></P>
<P align=left><FONT face=Tahoma size=2><FONT color=#0000ff>Unless expressly stated to the contrary this e-mail is not intended to, and shall not, have any contractually binding effect on the Company and its clients. We accept no liability for any reliance placed on this e-mail other than to the intended recipient. If the content is not about the business of this Company or its clients then the message is neither from nor sanctioned by the Company. </FONT></FONT></P>
<P align=left><FONT face=Tahoma size=2><FONT color=#0000ff>We accept no liability or responsibility for any changes made to this e-mail after it was sent or any viruses transmitted through this e-mail or any attachment. It is your responsibility to satisfy yourself that this e-mail or any attachment is free from viruses and can be opened without harm to your systems.</FONT></FONT></P>
<P align=left><FONT face=Tahoma color=#0000ff size=2></FONT> </P></HTML>