Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Tue, Mar 29, 2011 at 05:09:03PM +0200, "Lionel Sausin, de la part de l'équipe informatique Numérigraphe" wrote:
> Please let me answer myself, in case others look for the same answers later.
> The following is only assumptions however, so please somebody
> correct me if I'm wrong.
>
> >Sometime we have big activity spikes, like someone writing a 100GB
> >video work.
> >The network is slow so we may want to let the secondary node fall
> >behind, and we may use drbd-proxy
> >I gave the new features in v8.3.10 a try without drbd-proxy, and I
> >humbly admit I don't get the whole picture, so please let me ask a
> >few questions.
> >
> >First, is drbd-proxy required if we want to use the new congestion
> >settings in v3.8.10?
No. But they may not bee too useful without, the behaviour will likely
appear to be erratic (even though it is not).
Also, please be aware that a system that is catching up
from "Behind" mode is "Inconsistent" locally, just as a SyncTarget is,
until the connection mode becomes "Connected" once again.
> From what I understand, it is not required, but the congestion
> features are mostly useless without it.
> >If it's not required, then I wonder if it would be useful in our
> >setup. With video data, compression won't help, and when I push
> >large files to the drbd resource, "free" tells me the overall
> >buffers get as large as 7GB.
"free" buffer/page chache settings are no indication of whether things
are dirty or not, or currently under write-out or not.
This is something completely differnt.
You may want to look at /proc/meminfo
for a more detailed view.
> >What more does drbd-proxy do?
Something completely different ;-)
>
> The 7GB were actually not at all in DRBD's buffers, but in the
> kernel's dirty pages - we are used to having a very high
> vm.dirty_ratio on storage servers, to use lots of RAM as write
> cache.
> From what I can tell, drbd-proxy would provide the following bonus
> vs dirty pages:
> - smoother behavior: insensitive to sync() and various kernel
> optimizations trying to keep the dirty page count low
Dirty pages are those pages that have not yet been written to stable
storage, neither local nor remote.
These are modifications living in volatile main RAM only.
In the event of crash/power out age,
these modifications are lost.
And that has nothing to do with replication, or RAID, or storage
backend. If you just did not submit things to the block layer yet,
there is just nothing the block layer can do to preserve these changes.
[application]
[filesystem layer]
[page cache (buffer cache)]
--------------
things that are accounted in "free" as buffers/cached live above this
line, both clean (identical as on stable storage, as far as the OS
knows; useful as "read cache"), and dirty (yet to be submitted and
confirmed to be on stable storage).
--------------
[block layer] <--- that is where DRBD lives
[DRBD]
+--- ["local" storage]
|
N|
E| <--- this is where the DRBD proxy lives
T|
|
[remote DRBD]
`-- ["remote" storage]
> - compression
> >Another point that I don't get is how drbd uses the
> >"congestion-fill" parameter. If I set it low (like 1K), the
> >secondary node falls behind as soon as there is write activity, as
> >I expected. But if I set it to 1G and I rsync a 10GB file full of
> >zeros to the FS on the resource, the secondary node never falls
> >behind and rsync slows down to the network speed. Am I doing it
> >wrong?
> Actually the TCP buffer never gets bigger than a few MB, so my guess
> is that any value of congestion-fill bigger than that will never
> make the secondary node fall behind.
> I don't see the point in allowing values as big as 10GB, so I may
> still be missing something.
Possibly the proxy ;-)
Don't hesitate to give us a call.
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com