[DRBD-user] degraded write performace

[Duane Cox]

> A basic rule with DRBD:
> _NEVER_ access (read or write) the underlying block device while drbd is
> supposed to be in control of it.

Agreed.

> A general rule is that you should not use the underlying block device that
> DRBD has been setup for after you setup DRBD, with out all traffic passing
> through DRBD.  The exception to this rule is when you are in full disaster
> recovery, i.e., the drive has physically been moved to another machine and
you
> NEED  the data off of it because some one put .45 rounds through the
original
> machines.

Agreed.

> to test DRBD speed you should be running your test tool against /dev/drbdX
for
> the device. Also you should be aware that by writing to the lower level
> device, if you wrote enough data you may have munged DRBD's meta-data, but
> most certainly have munged DRBD's understanding of what data needs synced
> between the machines when they reconnect.

Yes, I am aware.

> Assumptions:
> A) the BM tool is issuing an appropriate Sync command, so their is no
extra
> data hanging around in the block device's buffer.

BM?  I'll assume you mean ./dm in the benchmark dir?  if so I did supply
the -y flag.  I assume it's working.

> B) you DON'T get the SAME/SIMILAR degradation when drbd is not running
during
> the tests.

That is correct, see latest email posted to list (one prior to this)

> C) nothing else is happening on the machine at the time.
> (if I read the data correctly your runs
> take < 2 minutes, if you start backing up
> applications like X and sendmail you can
> see big effects in that short a period,
> and they can take a while to clear.)

Nothing else is installed.  This is an LFS distro.  (X, sendmail, etc. are
not even installed)

> D) the BM tool writes to the same physical location on the disk each time.

BM?  Assuming you mean ./dm in the benchmark dir then how do I know where
it's writing?  based on the -b and -s flags?
If yes, then I issued the same command again and again.

> E) the disk is not doing any dynamic relocation of the data (ala Disk On a
Chip)

This is a single 36G 15K U160 seagate ST336754LC

> F) your test set is big enough that a) the kernel block drivers are not
> buffering a significant amount b) the physical drive is not buffering a
> significant amount, i.e., try ~10-20GB.

I tried 30G at one point, seen similiar results, just takes so darn long to
run.  I went back to 3G.

> G) your test set does not create a (big) memory buffer itself to then dump
to
> disk, i.e, cause other apps to swap out and then swap back in while you
are
> running the later tests.

I'm not sure about this, but how would that apply when "drbd detach all"
resolves the issue.

> Tis a bit strange (that it should go down when writing to the same
physical
> location on the disk).

I know, what's going on???