<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi Oleksiy,</p>
<p>Here is my entire current config:<br>
</p>
<p>global { usage-count no; }<br>
common { protocol C; }<br>
resource r0 {<br>
disk {<br>
on-io-error detach;<br>
no-disk-flushes ;<br>
no-disk-barrier;<br>
c-plan-ahead 0;<br>
c-fill-target 24M;<br>
c-min-rate 80M;<br>
c-max-rate 720M;<br>
}<br>
net {<br>
max-buffers 36k;<br>
sndbuf-size 1024k;<br>
rcvbuf-size 2048k;<br>
}<br>
on node1 {<br>
device /dev/drbd0;<br>
disk /dev/sdb1;<br>
address 192.168.200.1:7788;<br>
meta-disk internal;<br>
}<br>
on node2 {<br>
device /dev/drbd0;<br>
disk /dev/sdb1;<br>
address 192.168.200.2:7788;<br>
meta-disk internal;<br>
}<br>
}<br>
</p>
<p>...and the speed is still dreadfully slow, even though the link
can easily do 150MB/s:<br>
<br>
cat /proc/drbd<br>
version: 8.4.10 (api:1/proto:86-101)<br>
srcversion: 17A0C3A0AF9492ED4B9A418 <br>
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C
r-----<br>
ns:46712 nr:0 dw:0 dr:47528 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1
wo:d oos:3898265768<br>
[>....................] sync'ed: 0.1% (3806900/3806944)M<br>
finish: 3867:19:37 speed: 264 (260) K/sec</p>
<p>Anything else to try before giving up on DRBD?<br>
<br>
Thanks,<br>
Adam<br>
</p>
<br>
<div class="moz-cite-prefix">On 11/10/18 07:45, Oleksiy Evin wrote:<br>
</div>
<blockquote type="cite"
cite="mid:c380fd0c7c45c69f7b6f5110bedca830cdfa2db6.camel@onefc.com">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div>You may try to disable dynamic sync rate by
setting "c-plan-ahead" to 0 and increase "max-buffers". That's
the only way which helped me to get reasonable sync rate with
100GigE connection.</div>
<div><br>
</div>
<div>net {</div>
<div> ...</div>
<div> max-buffers 32K;</div>
<div> # max-epoch-size 18K;</div>
<div>}</div>
<div><br>
</div>
<div>disk {</div>
<div> ... </div>
<div> c-plan-ahead 0;</div>
<div>}</div>
<div><br>
</div>
<div>You can find some more info here: </div>
<div><a
href="https://serverfault.com/questions/740311/drbd-terrible-sync-performance-on-10gige/740370"
moz-do-not-send="true">https://serverfault.com/questions/740311/drbd-terrible-sync-performance-on-10gige/740370</a></div>
<div><br>
</div>
<div><br>
</div>
<div>//OE</div>
<div><br>
</div>
<div>-----Original Message-----</div>
<div><b>From</b>: Adam Weremczuk <<a
href="mailto:Adam%20Weremczuk%20%3cadamw@matrixscience.com%3e"
moz-do-not-send="true">adamw@matrixscience.com</a>></div>
<div><b>To</b>: <a href="mailto:drbd-user@lists.linbit.com"
moz-do-not-send="true">drbd-user@lists.linbit.com</a></div>
<div><b>Subject</b>: [DRBD-user] slow sync speed</div>
<div><b>Date</b>: Wed, 10 Oct 2018 14:57:02 +0100</div>
<div><br>
</div>
<pre>Hi all,</pre>
<pre>
</pre>
<pre>I'm trying out DRBD Pacemaker HA Cluster on Proxmox 5.2</pre>
<pre>
</pre>
<pre>I have 2 identical servers connected with 2 x 1 Gbps links in bond_mode </pre>
<pre>balance-rr.</pre>
<pre>
</pre>
<pre>The bond is working fine; I get a transfer rate of 150 MB/s with scp.</pre>
<pre>
</pre>
<pre>Following this guide: </pre>
<pre><a href="https://www.theurbanpenguin.com/drbd-pacemaker-ha-cluster-ubuntu-16-04/" moz-do-not-send="true">https://www.theurbanpenguin.com/drbd-pacemaker-ha-cluster-ubuntu-16-04/</a> </pre>
<pre>was going smoothly up until:</pre>
<pre>
</pre>
<pre>drbdadm -- --overwrite-data-of-peer primary r0/0</pre>
<pre>
</pre>
<pre>cat /proc/drbd</pre>
<pre>version: 8.4.10 (api:1/proto:86-101)</pre>
<pre>srcversion: 17A0C3A0AF9492ED4B9A418</pre>
<pre> 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----</pre>
<pre> ns:10944 nr:0 dw:0 dr:10992 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f </pre>
<pre>oos:3898301536</pre>
<pre> [>....................] sync'ed: 0.1% (3806932/3806944)M</pre>
<pre> finish: 483:25:13 speed: 2,188 (2,188) K/sec</pre>
<pre>
</pre>
<pre>The transfer rate is horribly slow and at this pace it's going to take </pre>
<pre>20 days for two 4 TB volumes to sync!</pre>
<pre>
</pre>
<pre>That's almost 15 times slower comparing with the guide video (8:30): </pre>
<pre><a href="https://www.youtube.com/watch?v=WQGi8Nf0kVc" moz-do-not-send="true">https://www.youtube.com/watch?v=WQGi8Nf0kVc</a></pre>
<pre>
</pre>
<pre>The volumes have been zeroed and contain no live data yet.</pre>
<pre>
</pre>
<pre>My sdb disks are logical drives (hardware RAID) set up as RAID50 with </pre>
<pre>the defaults:</pre>
<pre>
</pre>
<pre>Strip size: 128 KB</pre>
<pre>Access policy: RW</pre>
<pre>Read policy: Normal</pre>
<pre>Write policy: Write Back with BBU</pre>
<pre>IO policy: Direct</pre>
<pre>Drive Cache: Disable</pre>
<pre>Disable BGI: No</pre>
<pre>
</pre>
<pre>Performance looks good when tested with hdparm:</pre>
<pre>
</pre>
<pre>hdparm -tT /dev/sdb1</pre>
<pre>
</pre>
<pre>/dev/sdb1:</pre>
<pre> Timing cached reads: 15056 MB in 1.99 seconds = 7550.46 MB/sec</pre>
<pre> Timing buffered disk reads: 2100 MB in 3.00 seconds = 699.81 MB/sec</pre>
<pre>
</pre>
<pre>The volumes have been zeroed and contain no live data yet.</pre>
<pre>
</pre>
<pre>It seems to be a problem with default DRBD settings.</pre>
<pre>
</pre>
<pre>Can anybody recommend optimal tweaks specific to my environment?</pre>
<pre>
</pre>
<pre>Regards,</pre>
<pre>Adam</pre>
<pre>
</pre>
<pre>_______________________________________________</pre>
<pre>drbd-user mailing list</pre>
<pre><a href="mailto:drbd-user@lists.linbit.com" moz-do-not-send="true">drbd-user@lists.linbit.com</a></pre>
<pre><a href="http://lists.linbit.com/mailman/listinfo/drbd-user" moz-do-not-send="true">http://lists.linbit.com/mailman/listinfo/drbd-user</a></pre>
<pre>
</pre>
</blockquote>
<br>
</body>
</html>