<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>Hi,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I’m attempting to build a redundant storage solution
using drbd replication,<o:p></o:p></p>
<p class=MsoNormal>however I can't seem to get the performance I would have
expected.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I have the two servers on an Infiniband link, which has
tested at about 170MB/s<o:p></o:p></p>
<p class=MsoNormal>(Megabytes not Megabits) with netcat (CPU bound).<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Both disks write at just under 100MB/s (tested via
dd). Yet when attempting to<o:p></o:p></p>
<p class=MsoNormal>write to the drbd device on the primary I consistently get
30MB/s. I assume<o:p></o:p></p>
<p class=MsoNormal>the problem must be something I've configured incorrectly,
but for the life of<o:p></o:p></p>
<p class=MsoNormal>me I can't figure out what it is.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>My configuration can be found at: http://pastie.org/737624<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I have tried Google, and read and played with the tuning recommendations<o:p></o:p></p>
<p class=MsoNormal>in the drbd manual, with no success. In an effort to
demonstrate the <o:p></o:p></p>
<p class=MsoNormal>problem is not network speeds, or the secondary, I
have brought down the<o:p></o:p></p>
<p class=MsoNormal>secondary, and still see the same performance:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>[root@lager ~]# cat /proc/drbd<o:p></o:p></p>
<p class=MsoNormal>version: 8.3.2 (api:88/proto:86-90)<o:p></o:p></p>
<p class=MsoNormal>GIT-hash: dd7985327f146f33b86d4bff5ca8c94234ce840e build by
mockbuild@v20z-x86-64.home.local, 2009-08-29 14:07:55<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal> 1: cs:WFConnection ro:Primary/Unknown
ds:UpToDate/DUnknown C r----<o:p></o:p></p>
<p class=MsoNormal> ns:1048576 nr:0 dw:0 dr:1048576 al:0
bm:64 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>[root@lager ~]# dd if=/dev/zero of=/dev/drbd1 bs=512M
count=2<o:p></o:p></p>
<p class=MsoNormal>2+0 records in<o:p></o:p></p>
<p class=MsoNormal>2+0 records out<o:p></o:p></p>
<p class=MsoNormal>1073741824 bytes (1.1 GB) copied, 32.6865 seconds, 32.8 MB/s<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>[root@lager ~]# drbdadm down r0<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>[root@lager ~]# dd if=/dev/zero of=/dev/sda2 bs=512M count=2<o:p></o:p></p>
<p class=MsoNormal>2+0 records in<o:p></o:p></p>
<p class=MsoNormal>2+0 records out<o:p></o:p></p>
<p class=MsoNormal>1073741824 bytes (1.1 GB) copied, 11.1106 seconds, 96.6 MB/s<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>If anyone has any suggestions as to how I can troubleshoot
this further it<o:p></o:p></p>
<p class=MsoNormal>would be greatly appreciated.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Thanks!<o:p></o:p></p>
<p class=MsoNormal>Walton<o:p></o:p></p>
</div>
</body>
</html>