<html dir="ltr"><head></head><body style="text-align:left; direction:ltr;"><div>You may try to disable dynamic sync rate by setting "c-plan-ahead" to 0 and increase "max-buffers". That's the only way which helped me to get reasonable sync rate with 100GigE connection.</div><div><br></div><div>net {</div><div> ...</div><div> max-buffers 32K;</div><div> # max-epoch-size 18K;</div><div>}</div><div><br></div><div>disk {</div><div> ... </div><div> c-plan-ahead 0;</div><div>}</div><div><br></div><div>You can find some more info here: </div><div><a href="https://serverfault.com/questions/740311/drbd-terrible-sync-performance-on-10gige/740370">https://serverfault.com/questions/740311/drbd-terrible-sync-performance-on-10gige/740370</a></div><div><br></div><div><br></div><div>//OE</div><div></div><div><br></div><div>-----Original Message-----</div><div><b>From</b>: Adam Weremczuk <<a href="mailto:Adam%20Weremczuk%20%3cadamw@matrixscience.com%3e">adamw@matrixscience.com</a>></div><div><b>To</b>: <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a></div><div><b>Subject</b>: [DRBD-user] slow sync speed</div><div><b>Date</b>: Wed, 10 Oct 2018 14:57:02 +0100</div><div><br></div><pre>Hi all,</pre><pre><br></pre><pre>I'm trying out DRBD Pacemaker HA Cluster on Proxmox 5.2</pre><pre><br></pre><pre>I have 2 identical servers connected with 2 x 1 Gbps links in bond_mode </pre><pre>balance-rr.</pre><pre><br></pre><pre>The bond is working fine; I get a transfer rate of 150 MB/s with scp.</pre><pre><br></pre><pre>Following this guide: </pre><pre><a href="https://www.theurbanpenguin.com/drbd-pacemaker-ha-cluster-ubuntu-16-04/">https://www.theurbanpenguin.com/drbd-pacemaker-ha-cluster-ubuntu-16-04/</a> </pre><pre>was going smoothly up until:</pre><pre><br></pre><pre>drbdadm -- --overwrite-data-of-peer primary r0/0</pre><pre><br></pre><pre>cat /proc/drbd</pre><pre>version: 8.4.10 (api:1/proto:86-101)</pre><pre>srcversion: 17A0C3A0AF9492ED4B9A418</pre><pre> 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----</pre><pre> ns:10944 nr:0 dw:0 dr:10992 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f </pre><pre>oos:3898301536</pre><pre> [>....................] sync'ed: 0.1% (3806932/3806944)M</pre><pre> finish: 483:25:13 speed: 2,188 (2,188) K/sec</pre><pre><br></pre><pre>The transfer rate is horribly slow and at this pace it's going to take </pre><pre>20 days for two 4 TB volumes to sync!</pre><pre><br></pre><pre>That's almost 15 times slower comparing with the guide video (8:30): </pre><pre><a href="https://www.youtube.com/watch?v=WQGi8Nf0kVc">https://www.youtube.com/watch?v=WQGi8Nf0kVc</a></pre><pre><br></pre><pre>The volumes have been zeroed and contain no live data yet.</pre><pre><br></pre><pre>My sdb disks are logical drives (hardware RAID) set up as RAID50 with </pre><pre>the defaults:</pre><pre><br></pre><pre>Strip size: 128 KB</pre><pre>Access policy: RW</pre><pre>Read policy: Normal</pre><pre>Write policy: Write Back with BBU</pre><pre>IO policy: Direct</pre><pre>Drive Cache: Disable</pre><pre>Disable BGI: No</pre><pre><br></pre><pre>Performance looks good when tested with hdparm:</pre><pre><br></pre><pre>hdparm -tT /dev/sdb1</pre><pre><br></pre><pre>/dev/sdb1:</pre><pre> Timing cached reads: 15056 MB in 1.99 seconds = 7550.46 MB/sec</pre><pre> Timing buffered disk reads: 2100 MB in 3.00 seconds = 699.81 MB/sec</pre><pre><br></pre><pre>The volumes have been zeroed and contain no live data yet.</pre><pre><br></pre><pre>It seems to be a problem with default DRBD settings.</pre><pre><br></pre><pre>Can anybody recommend optimal tweaks specific to my environment?</pre><pre><br></pre><pre>Regards,</pre><pre>Adam</pre><pre><br></pre><pre>_______________________________________________</pre><pre>drbd-user mailing list</pre><pre><a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a></pre><pre><a href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a></pre><pre><br></pre></body></html>