<div dir="ltr"><div>RAID controller is OK yes.<br><br></div>Here is a 4 steps example of the issue :<br><br><br><br>### 1 - initial state :<br><br>Source :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 0<br>- eth1 incoming MB/s : 0<br>- eth1 outgoing MB/s : 0<br>Target :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 0<br>- eth1 incoming MB/s : 0<br>- eth1 outgoing MB/s : 0<br><br><br><br>### 2 - dd if=/dev/zero of=bigfile :<br><br>Source :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 670<br>- eth1 incoming MB/s : 1<br>- eth1 outgoing MB/s : 670<br>Target :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 670<br>- eth1 incoming MB/s : 670<br>- eth1 outgoing MB/s : 1<br><br><br><br>### 3 - disable the link between the 2 nodes :<br><br>Source :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 670<br>- eth1 incoming MB/s : 0<br>- eth1 outgoing MB/s : 0<br>Target :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 0<br>- eth1 incoming MB/s : 0<br>- eth1 outgoing MB/s : 0<br><br><br><br>### 4 - re-enable the link between the 2 nodes :<br><br>Source :<br>- sdb read MB/s : ~20<br>- sdb write MB/s : ~670<br>- eth1 incoming MB/s : 1<br>- eth1 outgoing MB/s : 670<br>Target :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 670<br>- eth1 incoming MB/s : 670<br>- eth1 outgoing MB/s : 1<br>DRBD resource :<br> 1: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----<br> ns:62950732 nr:1143320132 dw:1206271712 dr:1379744185 al:9869 bm:6499 lo:2 pe:681 ua:1 ap:0 ep:1 wo:d oos:11883000<br> [>...................] sync'ed: 6.9% (11604/12448)M<br> finish: 0:34:22 speed: 5,756 (6,568) want: 696,320 K/sec<br><br><br><br>### values I would have expected in step 4 :<br><br>Source :<br>- sdb read MB/s : ~400 (because of c-min-rate 400M)<br>- sdb write MB/s : ~370<br>- eth1 incoming MB/s : 1<br>- eth1 outgoing MB/s : 670<br>Target :<br>- sdb read MB/s : 0<br>- sdb write MB/s : 670<br>- eth1 incoming MB/s : 670<br>- eth1 outgoing MB/s : 1<br><div><div><div class="gmail_extra"><br></div><div class="gmail_extra">Why resync is totally ignored and application (dd here in the example) still consumes all available IOs / bandwidth ?<br><br><br><br></div><div class="gmail_extra">Thank you,<br><br></div><div class="gmail_extra">Ben<br></div><div class="gmail_extra"><br><br><br><div class="gmail_quote">2015-05-25 16:50 GMT+02:00 A.Rubio <span dir="ltr"><<a href="mailto:aurusa@etsii.upv.es" target="_blank">aurusa@etsii.upv.es</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Cache settings an I/O in RAID controller is optimal ??? Write-back, write-through, cache enablad, I/O direct, ...<br>
<br>
El 25/05/15 a las 11:50, Ben RUBSON escribió:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="h5">
The link between nodes is a 10Gb/s link.<br>
The DRBD resource is a RAID-10 array which is able to resync at up to 800M (as you can see I have lowered it to 680M in my configuration file).<br>
<br>
The "issue" here seems to be a prioritization "issue" between application IOs and resync IOs.<br>
Perhaps I miss-configured something ?<br>
Goal is to have resync rate up to 680M, with a minimum of 400M, even if there are application IOs.<br>
This in order to be sure to complete the resync even if there are a lot of write IOs from the application.<br>
<br>
With my simple test below, this is not the case, dd still writes at its best throughput, lowering resync rate which can’t reach 400M at all.<br>
<br>
Thank you !<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Le 25 mai 2015 à 11:18, A.Rubio <<a href="mailto:aurusa@etsii.upv.es" target="_blank">aurusa@etsii.upv.es</a>> a écrit :<br>
<br>
the link between nodes is ??? 1Gb/s ??? , 10Gb/s ??? ...<br>
<br>
the Hard Disks are ??? SATA 7200rpm ???, 10000rpm ???, SAS ???,<br>
SSD ???...<br>
<br>
400M to 680M with a 10Gb/s link and SAS 15.000 rpm is OK but less ...<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Le 12 avr. 2014 à 17:23, Ben RUBSON <<a href="mailto:ben.rubson@gmail.com" target="_blank">ben.rubson@gmail.com</a>> a écrit :<br>
<br>
Hello,<br>
<br>
Let's assume the following configuration :<br>
disk {<br>
c-plan-ahead 0;<br>
resync-rate 680M;<br>
c-min-rate 400M;<br>
}<br>
<br>
Both nodes are uptodate, and on the primary, I have a test IO burst running, using dd.<br>
<br>
I then cut replication link for a few minutes so that secondary node will be several GB behind primary node.<br>
<br>
I then re-enable replication link.<br>
What I expect here according to the configuration is that secondary node will fetch missing GB at a 400 MB/s throughput.<br>
DRBD should then prefer resync IOs over application (dd here) IOs.<br>
<br>
However, it does not seems to work.<br>
dd still writes at its best throughput, meanwhile reads are made from the primary disk between 30 and 60 MB/s to complete the resync.<br>
Of course this is not the expected behaviour.<br>
<br>
Did I miss something ?
</blockquote></blockquote></div></div></blockquote></blockquote></div><br></div></div></div></div>