<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">Thank you for your answer, and don't worry, we clearly understand you !</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">What I want is the following :</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">When the secondary node is outdated, I want it to be uptodate as soon as possible, even if new blocks arrive on the primary node, which would of course also need to be duplicated.</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">My priority is to recover the replication delay.</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">So, when the secondary node is outdated, I want DRBD to reduce / delay application IOs on the primary node, so that the replication link is almost dedicated to recover the delay, to make the secondary node uptodate as soon as possible.</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">Once secondary node is uptodate, replication link is fully back for new blocks, application data rate can then reach again the link bandwidth.</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">This is why I tried to use :</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">c-plan-ahead 0;</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">resync-rate 680M; --> my replication link can go up to 680M</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">c-min-rate 400M; --> I want to have at least 400M to recover a delay</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">However, it does not work ; application write data rate keeps reaching 600/680M even if secondary node is outdated / we have to recover replication delay.</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">I would have expected application data rate to be reduced to 280M (680-400) and 400M of bandwidth used to recover the delay.</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">I may be wrong in my settings / in my understanding of how DRBD works in this specific case.</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">Thank you again,</span><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><br style="color: rgb(34, 34, 34); font-family: arial, sans-serif; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: small; background-color: rgb(255, 255, 255);" class="">Ben</span><div class=""><font color="#222222" face="arial, sans-serif" size="2" class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></font></div><div class=""><font color="#222222" face="arial, sans-serif" size="2" class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></font></div><div class=""><font color="#222222" face="arial, sans-serif" size="2" class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></font><div><blockquote type="cite" class=""><div class="">Le 8 juin 2015 à 21:00, A.Rubio <<a href="mailto:aurusa@etsii.upv.es" class="">aurusa@etsii.upv.es</a>> a écrit :</div><br class="Apple-interchange-newline"><div class="">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
Hello. <br class="">
<br class="">
I didnt reply your email because:<br class="">
<br class="">
I'm a DRBD user. I'm not in the DRBD Team.<br class="">
<br class="">
My english is poor.<br class="">
<br class="">
And I dont understand what is your problem, because in the email you
send the DRBD sync is aprox 680MB/s.<br class="">
<br class="">
RAID 10 to 800MB/s is a theoretical or tested value ?<br class="">
<br class="">
Do you test link connection ? What it is the real speed of your
connection ? I think that 680MB/s is good. The theoretical value in
a 10Gb/s is 1280MB/s but real depends<br class="">
<br class="">
You can test speed link connection with<br class="">
<br class="">
<a class="moz-txt-link-freetext" href="http://eggdrop.ch/projects/bwbench/">http://eggdrop.ch/projects/bwbench/</a><br class="">
<br class="">
What is you want ?<br class="">
<br class="">
more speed in sync ??<br class="">
<br class="">
<br class="">
<div class="moz-cite-prefix">El 08/06/15 a las 16:47, Ben RUBSON
escribió:<br class="">
</div>
<blockquote cite="mid:CAGUFuyAdqt3_5hzcMBWw7tXJOkK+x0EZ5bWWY5L8ZHcW0V7aVg@mail.gmail.com" type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">Hello,<br class="">
<br class="">
</div>
I am confused to ask again, but could you help me
with this please ?<br class="">
</div>
I really don't know how to go further, if the behavior
I would like to have is supported by DRBD or not...<br class="">
<br class="">
</div>
DRBD team ?<br class="">
</div>
Any support would really be appreciated.<br class="">
<br class="">
</div>
Thank you again,<br class="">
<br class="">
</div>
Best regards,<br class="">
<br class="">
</div>
Ben<br class="">
<br class="">
<br class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="gmail_extra"><br class="">
<div class="gmail_quote">2015-05-28 17:05
GMT+02:00 Ben RUBSON <span dir="ltr" class=""><<a moz-do-not-send="true" href="mailto:ben.rubson@gmail.com" target="_blank" class="">ben.rubson@gmail.com</a>></span>:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div dir="ltr" class="">So,<br class="">
<br class="">
I played during hours with the dynamic
resync rate controller.<br class="">
Here are my settings :<br class="">
<br class="">
c-plan-ahead 10; //10ms between my 2
nodes, but minimum 1 second
recommended here : <a moz-do-not-send="true" href="https://blogs.linbit.com/p/128/drbd-sync-rate-controller/" target="_blank" class="">https://blogs.linbit.com/p/128/drbd-sync-rate-controller/</a><br class="">
resync-rate 680M; //mainly ignored<br class="">
c-min-rate 400M;<br class="">
c-max-rate 680M;<br class="">
c-fill-target 20M; //my BDP is 6.8M,
guides say to use a value between 1x
and 3x BDP<br class="">
<br class="">
Resync can achieve up to 680M when
there are no application IOs on the
source.<br class="">
However, as soon as there are
application IOs (writes with dd in my
tests), resync slows down to some
MB/s...<br class="">
I played with c-plan-ahead and
c-fill-target without success.<br class="">
I also tested c-delay-target.<br class="">
I tried to set unplug-watermark to 16.<br class="">
My IO scheduler is already the
deadline one...<br class="">
<br class="">
Well, I'm a little bit lost, I can't
achieve to get resync with a minimum
rate of 400M when there are
application IOs...<br class="">
<br class="">
Here, Lars says :<br class="">
<a moz-do-not-send="true" href="http://lists.linbit.com/pipermail/drbd-user/2011-August/016739.html" target="_blank" class="">http://lists.linbit.com/pipermail/drbd-user/2011-August/016739.html</a><br class="">
The dynamic resync rate controller
basically tries to use as much as
c-max-rate bandwidth, but will
automatically throttle, if<br class="">
- application IO on the device is
detected (READ or WRITE), AND the
estimated current resync speed is
above c-min-rate<br class="">
- the amount of in-flight resync
requests exceeds c-fill-target<br class="">
<br class="">
However, does DRBD throttle
application IOs when resync rate is
lower than c-min-rate ?<br class="">
According to my tests I'm not so sure.<br class="">
<br class="">
<br class="">
<div class="">
<div class="h5">
<div class="gmail_extra"><br class="">
<div class="gmail_quote">2015-05-26
15:06 GMT+02:00 A.Rubio <span dir="ltr" class=""><<a moz-do-not-send="true" href="mailto:aurusa@etsii.upv.es" target="_blank" class="">aurusa@etsii.upv.es</a>></span>:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000" class=""> Have you
test these values ?<br class="">
<br class="">
<a moz-do-not-send="true" href="https://drbd.linbit.com/users-guide/s-throughput-tuning.html" target="_blank" class="">https://drbd.linbit.com/users-guide/s-throughput-tuning.html</a><br class="">
<br class="">
<br class="">
<div class="">El 26/05/15 a las
13:16, Ben RUBSON
escribió:<br class="">
</div>
<blockquote type="cite" class="">
<div class="">
<div class="">
<div dir="ltr" class="">
<div class="">RAID
controller is OK
yes.<br class="">
<br class="">
</div>
Here is a 4 steps
example of the
issue :<br class="">
<br class="">
<br class="">
<br class="">
### 1 - initial
state :<br class="">
<br class="">
Source :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 0<br class="">
- eth1 incoming
MB/s : 0<br class="">
- eth1 outgoing
MB/s : 0<br class="">
Target :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 0<br class="">
- eth1 incoming
MB/s : 0<br class="">
- eth1 outgoing
MB/s : 0<br class="">
<br class="">
<br class="">
<br class="">
### 2 - dd
if=/dev/zero
of=bigfile :<br class="">
<br class="">
Source :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 670<br class="">
- eth1 incoming
MB/s : 1<br class="">
- eth1 outgoing
MB/s : 670<br class="">
Target :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 670<br class="">
- eth1 incoming
MB/s : 670<br class="">
- eth1 outgoing
MB/s : 1<br class="">
<br class="">
<br class="">
<br class="">
### 3 - disable
the link between
the 2 nodes :<br class="">
<br class="">
Source :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 670<br class="">
- eth1 incoming
MB/s : 0<br class="">
- eth1 outgoing
MB/s : 0<br class="">
Target :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 0<br class="">
- eth1 incoming
MB/s : 0<br class="">
- eth1 outgoing
MB/s : 0<br class="">
<br class="">
<br class="">
<br class="">
### 4 - re-enable
the link between
the 2 nodes :<br class="">
<br class="">
Source :<br class="">
- sdb read
MB/s : ~20<br class="">
- sdb write
MB/s : ~670<br class="">
- eth1 incoming
MB/s : 1<br class="">
- eth1 outgoing
MB/s : 670<br class="">
Target :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 670<br class="">
- eth1 incoming
MB/s : 670<br class="">
- eth1 outgoing
MB/s : 1<br class="">
DRBD resource :<br class="">
1: cs:SyncTarget
ro:Secondary/Primary
ds:Inconsistent/UpToDate
C r-----<br class="">
ns:62950732
nr:1143320132
dw:1206271712
dr:1379744185
al:9869 bm:6499
lo:2 pe:681 ua:1
ap:0 ep:1 wo:d
oos:11883000<br class="">
[>...................]
sync'ed: 6.9%
(11604/12448)M<br class="">
finish:
0:34:22 speed:
5,756 (6,568)
want: 696,320
K/sec<br class="">
<br class="">
<br class="">
<br class="">
### values I would
have expected in
step 4 :<br class="">
<br class="">
Source :<br class="">
- sdb read
MB/s : ~400
(because of
c-min-rate 400M)<br class="">
- sdb write
MB/s : ~370<br class="">
- eth1 incoming
MB/s : 1<br class="">
- eth1 outgoing
MB/s : 670<br class="">
Target :<br class="">
- sdb read
MB/s : 0<br class="">
- sdb write
MB/s : 670<br class="">
- eth1 incoming
MB/s : 670<br class="">
- eth1 outgoing
MB/s : 1<br class="">
<div class="">
<div class="">
<div class="gmail_extra"><br class="">
</div>
<div class="gmail_extra">Why
resync is
totally
ignored and
application
(dd here in
the example)
still consumes
all available
IOs /
bandwidth ?<br class="">
<br class="">
<br class="">
<br class="">
<div class="gmail_quote">2015-05-25
16:50
GMT+02:00
A.Rubio <span dir="ltr" class=""><<a moz-do-not-send="true" href="mailto:aurusa@etsii.upv.es" target="_blank" class="">aurusa@etsii.upv.es</a>></span>:<br class="">
<blockquote class="gmail_quote" style="margin:0px
0px 0px
0.8ex;border-left:1px
solid
rgb(204,204,204);padding-left:1ex">Cache
settings an
I/O in RAID
controller is
optimal ???
Write-back,
write-through,
cache enablad,
I/O direct,
...<br class="">
<br class="">
El 25/05/15 a
las 11:50, Ben
RUBSON
escribió:<br class="">
<blockquote class="gmail_quote" style="margin:0px
0px 0px
0.8ex;border-left:1px
solid
rgb(204,204,204);padding-left:1ex">
<div class="">
<div class=""> The link
between nodes
is a 10Gb/s
link.<br class="">
The DRBD
resource is a
RAID-10 array
which is able
to resync at
up to 800M (as
you can see I
have lowered
it to 680M in
my
configuration
file).<br class="">
<br class="">
The "issue"
here seems to
be a
prioritization
"issue"
between
application
IOs and resync
IOs.<br class="">
Perhaps I
miss-configured
something ?<br class="">
Goal is to
have resync
rate up to
680M, with a
minimum of
400M, even if
there are
application
IOs.<br class="">
This in order
to be sure to
complete the
resync even if
there are a
lot of write
IOs from the
application.<br class="">
<br class="">
With my simple
test below,
this is not
the case, dd
still writes
at its best
throughput,
lowering
resync rate
which can’t
reach 400M at
all.<br class="">
<br class="">
Thank you !<br class="">
<br class="">
<blockquote class="gmail_quote" style="margin:0px
0px 0px
0.8ex;border-left:1px
solid
rgb(204,204,204);padding-left:1ex">
Le 25 mai 2015
à 11:18,
A.Rubio <<a moz-do-not-send="true" href="mailto:aurusa@etsii.upv.es" target="_blank" class="">aurusa@etsii.upv.es</a>>
a écrit :<br class="">
<br class="">
the link
between nodes
is ??? 1Gb/s
??? , 10Gb/s
??? ...<br class="">
<br class="">
the Hard Disks
are ??? SATA
7200rpm ???,
10000rpm ???,
SAS ???,<br class="">
SSD ???...<br class="">
<br class="">
400M to 680M
with a 10Gb/s
link and SAS
15.000 rpm is
OK but less
...<br class="">
<br class="">
<blockquote class="gmail_quote" style="margin:0px
0px 0px
0.8ex;border-left:1px
solid
rgb(204,204,204);padding-left:1ex">
Le 12 avr.
2014 à 17:23,
Ben RUBSON
<<a moz-do-not-send="true" href="mailto:ben.rubson@gmail.com" target="_blank" class="">ben.rubson@gmail.com</a>>
a écrit :<br class="">
<br class="">
Hello,<br class="">
<br class="">
Let's assume
the following
configuration
:<br class="">
disk {<br class="">
c-plan-ahead
0;<br class="">
resync-rate
680M;<br class="">
c-min-rate
400M;<br class="">
}<br class="">
<br class="">
Both nodes are
uptodate, and
on the
primary, I
have a test IO
burst running,
using dd.<br class="">
<br class="">
I then cut
replication
link for a few
minutes so
that secondary
node will be
several GB
behind primary
node.<br class="">
<br class="">
I then
re-enable
replication
link.<br class="">
What I expect
here according
to the
configuration
is that
secondary node
will fetch
missing GB at
a 400 MB/s
throughput.<br class="">
DRBD should
then prefer
resync IOs
over
application
(dd here) IOs.<br class="">
<br class="">
However, it
does not seems
to work.<br class="">
dd still
writes at its
best
throughput,
meanwhile
reads are made
from the
primary disk
between 30 and
60 MB/s to
complete the
resync.<br class="">
Of course this
is not the
expected
behaviour.<br class="">
<br class="">
Did I miss
something ?</blockquote></blockquote></div></div></blockquote></blockquote></div></div></div></div></div></div></div></blockquote></div></blockquote></div></div></div></div></div></blockquote></div></div></div></div></div></div></div></div></div></div></div></div></blockquote>
</div>
</div></blockquote></div><br class=""></div></body></html>