<div dir="ltr">Is there anything that will force the OOS to push what is out of sync?<div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 21, 2019 at 11:00 AM Digimer <<a href="mailto:lists@alteeve.ca">lists@alteeve.ca</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Tuning is quite instance-specific. I would always suggest starting by<br>
commenting out all tuning, see how it behaves, then tune. Premature<br>
optimization never is.<br>
<br>
digimer<br>
<br>
On 2019-10-21 1:31 p.m., G C wrote:<br>
> Would any of these values being changed help or would it need to be the<br>
> actual speed between the two nodes that needs to be increased?<br>
> <br>
> disk {<br>
> on-io-error detach;<br>
> c-plan-ahead 10;<br>
> c-fill-target 24M;<br>
> c-min-rate 80M;<br>
> c-max-rate 720M;<br>
> }<br>
> net {<br>
> protocol A;<br>
> max-buffers 36k;<br>
> sndbuf-size 1024k;<br>
> rcvbuf-size 2048k;<br>
> }<br>
> <br>
> Thank you<br>
> <br>
> <br>
> <br>
> On Mon, Oct 21, 2019 at 10:10 AM Digimer <<a href="mailto:lists@alteeve.ca" target="_blank">lists@alteeve.ca</a><br>
> <mailto:<a href="mailto:lists@alteeve.ca" target="_blank">lists@alteeve.ca</a>>> wrote:<br>
> <br>
> I assumed it wasn't paused, but that confirms it.<br>
> <br>
> Protocol A allows for out of sync to grow. It says "when the data in on<br>
> the network buffer to send to the peer, consider the write complete". As<br>
> such, data that hasn't made it over to the peer causes oos to climb. If<br>
> you have a steady write rate that is faster than your transmit<br>
> bandwidth, then seeing fairly steady OOS makes sense.<br>
> <br>
> To "fix" it, you need to increase the connection speed to the peer node.<br>
> Or, less likely, if the peer's disk is slower than the bandwidth<br>
> connecting it, speed up the disk write speed.<br>
> <br>
> In either case, what you are seeing is not a surprise, and it's not a<br>
> problem with DRBD. The only other option is to use protocol C, so that a<br>
> write isn't complete until it reaches the peer, but that will slow down<br>
> the write performance of the primary node to be whatever speed you have<br>
> to the peer. That's likely unacceptable.<br>
> <br>
> In short, you have a hardware/resource issue.<br>
> <br>
> digimer<br>
> <br>
> On 2019-10-21 12:19 p.m., G C wrote:<br>
> > version: 8.4.10<br>
> > Ran the resume-sync all and received:<br>
> > 0: Failure: (135) Sync-pause flag is already cleared<br>
> > Command 'drbdsetup-84 resume-sync 0' terminated with exit code 10<br>
> ><br>
> > Protocol used is 'A', our systems are running on a cloud environment.<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > On Mon, Oct 21, 2019 at 9:09 AM Digimer <<a href="mailto:lists@alteeve.ca" target="_blank">lists@alteeve.ca</a><br>
> <mailto:<a href="mailto:lists@alteeve.ca" target="_blank">lists@alteeve.ca</a>><br>
> > <mailto:<a href="mailto:lists@alteeve.ca" target="_blank">lists@alteeve.ca</a> <mailto:<a href="mailto:lists@alteeve.ca" target="_blank">lists@alteeve.ca</a>>>> wrote:<br>
> ><br>
> > 8.9.2 is the utils version, what is the kernel module version?<br>
> > (8.3.x/8.4.x/9.0.x)?<br>
> ><br>
> > It's possible something paused sync, but I doubt it. You can try<br>
> > 'drbdadm resume-sync all'. The oos number should change<br>
> constantly, any<br>
> > time a block changes it should go up and every time a block<br>
> syncs it<br>
> > should go down.<br>
> ><br>
> > What protocol are you using? A, B or C?<br>
> ><br>
> > digimer<br>
> <br>
> <br>
> -- <br>
> Digimer<br>
> Papers and Projects: <a href="https://alteeve.com/w/" rel="noreferrer" target="_blank">https://alteeve.com/w/</a><br>
> "I am, somehow, less interested in the weight and convolutions of<br>
> Einstein’s brain than in the near certainty that people of equal talent<br>
> have lived and died in cotton fields and sweatshops." - Stephen Jay<br>
> Gould<br>
> <br>
<br>
<br>
-- <br>
Digimer<br>
Papers and Projects: <a href="https://alteeve.com/w/" rel="noreferrer" target="_blank">https://alteeve.com/w/</a><br>
"I am, somehow, less interested in the weight and convolutions of<br>
Einstein’s brain than in the near certainty that people of equal talent<br>
have lived and died in cotton fields and sweatshops." - Stephen Jay Gould<br>
</blockquote></div>