<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">That’s a very difficult way to go about setting up internal metadata. Normally, just we create the metadata on the raw device (/dev/sdb) and then create the
filesystem on the drbd device (/dev/sql_data1). No math!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">You did not appear to specify a syncer rate. I thought the default was much higher than 2040K, but that’s the target for the sync operation. Why not set the
synch rate up to some reasonable percentage (most all for initial sync, maybe 30% of your bandwidth thereafter) of the available bandwidth. You say “low” without defining it. The displays appear syncher rate constrained.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Also, you don’t have to do a full sync on initially empty disks. That’s in the doc under clear-bitmap and/or new-current-uuid.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">You can modify the syncher rate while running, or in the config files and then “adjust” the resources.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Dan<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> drbd-user-bounces@lists.linbit.com [mailto:drbd-user-bounces@lists.linbit.com]
<b>On Behalf Of </b>Lafaille Christophe<br>
<b>Sent:</b> Thursday, September 05, 2013 9:37 AM<br>
<b>To:</b> drbd-user@lists.linbit.com<br>
<b>Subject:</b> [DRBD-user] csums-alg seems not working on my cluster....<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi All,<br>
<br>
I need to use very low bandwith network between 2 machines using drbd and I try using csums-alg/verify-alg.<br>
<br>
But I've same duration with or without csums-alg !<br>
<br>
Execution with csums-alg:<o:p></o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">[root@sms246105 drbd.d]# cat /proc/drbd</span></tt><br>
<tt><span style="font-size:10.0pt">version: 8.4.2 (api:1/proto:86-101)</span></tt><br>
<tt><span style="font-size:10.0pt">GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@rh63_build, 2013-01-10 09:57:53</span></tt><br>
<br>
<tt><span style="font-size:10.0pt"> 1: cs:SyncTarget ro:Secondary/Secondary ds:Inconsistent/UpToDate C r-----</span></tt><br>
<tt><span style="font-size:10.0pt"> ns:0 nr:512 dw:512 dr:147968 al:0 bm:9 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:4904384</span></tt><br>
<tt><span style="font-size:10.0pt"> [>....................] sync'ed: 3.0% (4788/4932)M</span></tt><br>
<tt><span style="font-size:10.0pt"> finish: 0:15:17 speed: 5,332 (5,284) want: 2,040 K/sec</span></tt><o:p></o:p></p>
<p class="MsoNormal"><br>
Execution without csums-alg:<o:p></o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">[root@sms246105 drbd.d]# cat /proc/drbd</span></tt><br>
<tt><span style="font-size:10.0pt">version: 8.4.2 (api:1/proto:86-101)</span></tt><br>
<tt><span style="font-size:10.0pt">GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@rh63_build, 2013-01-10 09:57:53</span></tt><br>
<br>
<tt><span style="font-size:10.0pt"> 1: cs:SyncTarget ro:Secondary/Secondary ds:Inconsistent/UpToDate C r-----</span></tt><br>
<tt><span style="font-size:10.0pt"> ns:0 nr:53760 dw:53760 dr:0 al:0 bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:4819904</span></tt><br>
<tt><span style="font-size:10.0pt"> [>....................] sync'ed: 1.2% (4704/4756)M</span></tt><br>
<tt><span style="font-size:10.0pt"> finish: 0:14:52 speed: 5,376 (5,376) want: 2,040 K/sec</span></tt><o:p></o:p></p>
<p class="MsoNormal">I don't know where is the problem... is csums-alg usable only in a more recent version of DRBD (like 8.4.3 or 8.4.4) ?<br>
I've built drbd packages from source, perhaps I need to specify an option in order to have csums-alg facility (I'll check for this) ?<br>
<br>
I've put csums-alg in "net" section and in some web pages I've found a "syncer" section with csums-alg (seems no more available in 8.4.x versions).<br>
==> what's the good place ?<br>
<br>
On both machines, I do this sequence:<o:p></o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt"># /etc/init.d/drbd stop</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt># delete all partition on /dev/sdb and create a 5GB (for my tests, real size is around 300GB) partitions with fdisk</tt><br>
<tt># partprobe /dev/sdb</tt></span><span style="font-size:7.5pt;font-family:"Courier New""><br>
</span><tt><span style="font-size:10.0pt">#</span></tt><tt><b><span style="font-size:7.5pt">
</span></b></tt><tt><span style="font-size:10.0pt">dd if=/dev/zero of=dev/sdb1 bs=4096 ==> to initialize disk content</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt># mkfs.ext3 -j -m 0 -b 4096 /dev/sdb1</tt><br>
<tt># PARTSIZE=`sfdisk -s /dev/sdb1 | xargs -i echo "{} 1024 / 1024 / p" | dc`</tt><br>
<tt># NEWSIZE=$[${PARTSIZE}-2]</tt><br>
<tt># resize2fs /dev/sdb1 ${NEWSIZE}G</tt><br>
<tt># e2fsck -f /dev/sdb1</tt><br>
<tt># /etc/init.d/drbd start</tt><br>
<tt># /sbin/drbdadm create-md sqldata</tt><br>
<tt># /sbin/drbdadm up sqldata</tt><br>
<tt>On one machine: # /sbin/drbdadm --force primary sqldata</tt></span><o:p></o:p></p>
<p class="MsoNormal">The file /etc/drbd.d/sqldata.res :<o:p></o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">resource sqldata {</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt> device /dev/drbd_sqldata minor 1;</tt><br>
<tt> disk /dev/sdb1;</tt><br>
<tt> meta-disk internal;</tt><br>
<tt> on sms246104 {</tt><br>
<tt> address 135.117.246.104:7788;</tt><br>
<tt> }</tt><br>
<tt> on sms246105 {</tt><br>
<tt> address 135.117.246.105:7788;</tt><br>
<tt> }</tt><br>
<tt>}</tt></span><o:p></o:p></p>
<p class="MsoNormal">The file /etc/drbd.d/global_common.conf :<o:p></o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">global {</span></tt><br>
<tt><span style="font-size:10.0pt"> usage-count yes;</span></tt><br>
<tt><span style="font-size:10.0pt"> dialog-refresh 1;</span></tt><br>
<tt><span style="font-size:10.0pt"> minor-count 5;</span></tt><br>
<tt><span style="font-size:10.0pt">}</span></tt><br>
<tt><span style="font-size:10.0pt">common {</span></tt><br>
<tt><span style="font-size:10.0pt"> handlers {</span></tt><br>
<tt><span style="font-size:10.0pt"> pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";</span></tt><br>
<tt><span style="font-size:10.0pt"> pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";</span></tt><br>
<tt><span style="font-size:10.0pt"> local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";</span></tt><br>
<tt><span style="font-size:10.0pt"> split-brain "/usr/lib/drbd/notify-split-brain.sh root";</span></tt><br>
<tt><span style="font-size:10.0pt"> }</span></tt><br>
<br>
<tt><span style="font-size:10.0pt"> startup {</span></tt><br>
<tt><span style="font-size:10.0pt"> wfc-timeout 15;</span></tt><br>
<tt><span style="font-size:10.0pt"> }</span></tt><br>
<br>
<tt><span style="font-size:10.0pt"> options {</span></tt><br>
<tt><span style="font-size:10.0pt"> }</span></tt><br>
<br>
<tt><span style="font-size:10.0pt"> disk {</span></tt><br>
<tt><span style="font-size:10.0pt"> on-io-error detach;</span></tt><br>
<tt><span style="font-size:10.0pt"> c-plan-ahead 20;</span></tt><br>
<tt><span style="font-size:10.0pt"> c-fill-target 50k;</span></tt><br>
<tt><span style="font-size:10.0pt"> c-min-rate 250k;</span></tt><br>
<tt><span style="font-size:10.0pt"> c-max-rate 2M;</span></tt><br>
<tt><span style="font-size:10.0pt"> }</span></tt><br>
<br>
<tt><span style="font-size:10.0pt"> net {</span></tt><br>
<tt><span style="font-size:10.0pt"> timeout 60;</span></tt><br>
<tt><span style="font-size:10.0pt"> ping-int 6;</span></tt><br>
<tt><span style="font-size:10.0pt"> after-sb-0pri discard-younger-primary;</span></tt><br>
<tt><span style="font-size:10.0pt"> after-sb-1pri discard-secondary;</span></tt><br>
<tt><span style="font-size:10.0pt"> after-sb-2pri call-pri-lost-after-sb;</span></tt><br>
<tt><span style="font-size:10.0pt"> ping-timeout 60;</span></tt><br>
<tt><span style="font-size:10.0pt"> protocol C;</span></tt><br>
<tt><span style="font-size:10.0pt"> cram-hmac-alg sha1;</span></tt><br>
<tt><span style="font-size:10.0pt"> shared-secret "TestHA";</span></tt><br>
<tt><span style="font-size:10.0pt"> csums-alg sha1;</span></tt><br>
<tt><span style="font-size:10.0pt"> verify-alg sha1;</span></tt><br>
<tt><span style="font-size:10.0pt"> }</span></tt><br>
<tt><span style="font-size:10.0pt">}</span></tt><o:p></o:p></p>
<p class="MsoNormal"><br>
Traces in /var/log/kern.log :<o:p></o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: events: mcg drbd: 2</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: drbd: initialized. Version: 8.4.2 (api:1/proto:86-101)</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: drbd: GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@rh63_build, 2013-01-10 09:57:53</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: drbd: registered as block device major 147</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: d-con sqldata: Starting worker thread (from drbdsetup [21781])</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: disk( Diskless -> Attaching )</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: d-con sqldata: Method to ensure write ordering: flush</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: max BIO size = 524288</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: drbd_bm_resize called with capacity == 10486656</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: resync bitmap: bits=1310832 words=40964 pages=41</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: size = 5120 MB (5243328 KB)</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: bitmap READ of 41 pages took 1 jiffies</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: recounting of set bits took additional 0 jiffies</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: 5051 MB (1293168 bits) marked out-of-sync by on disk bit-map.</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: disk( Attaching -> UpToDate ) pdsk( DUnknown -> Outdated )</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: block drbd1: attached to UUIDs 646C4E1151078FBF:DAB8D60C65E253A0:0000000000000004:0000000000000000</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: d-con sqldata: conn( StandAlone -> Unconnected )</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: d-con sqldata: Starting receiver thread (from drbd_w_sqldata [21782])</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: d-con sqldata: receiver (re)started</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:16 sms246104 kernel: d-con sqldata: conn( Unconnected -> WFConnection )</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: d-con sqldata: Handshake successful: Agreed network protocol version 101</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: d-con sqldata: Peer authenticated using 20 bytes HMAC</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: d-con sqldata: conn( WFConnection -> WFReportParams )</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: d-con sqldata: Starting asender thread (from drbd_r_sqldata [21786])</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: drbd_sync_handshake:</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: self 646C4E1151078FBE:DAB8D60C65E253A0:0000000000000004:0000000000000000 bits:1293168 flags:0</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: peer DAB8D60C65E253A0:0000000000000000:0000000000000000:0000000000000000 bits:1293168 flags:0</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: uuid_compare()=1 by rule 70</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: Becoming sync source due to disk states.</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS ) pdsk( Outdated -> Inconsistent )</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total 25; compression: 100.0%</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total 25; compression: 100.0%</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: helper command: /sbin/drbdadm before-resync-source minor-1</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: helper command: /sbin/drbdadm before-resync-source minor-1 exit code 0 (0x0)</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: conn( WFBitMapS -> SyncSource )</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: Began resync as SyncSource (will sync 5172672 KB [1293168 bits set]).</span></tt><br>
<tt><span style="font-size:10.0pt">Sep 5 13:03:37 sms246104 kernel: block drbd1: updated sync UUID 646C4E1151078FBE:DAB9D60C65E253A0:DAB8D60C65E253A0:0000000000000004</span></tt><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
I hope someone will be able to help me...<br>
<br>
Perhaps, I'm totally wrong on csums-alg usage !<br>
<br>
Regards<br>
<br>
<br>
<br>
<br>
<o:p></o:p></p>
</div>
</div>
</body>
</html>