[DRBD-user] DRBD 8.2.6 - reason for full resync?!

Lars Ellenberg lars.ellenberg at linbit.com
Tue Jun 24 13:00:28 CEST 2008

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On Tue, Jun 24, 2008 at 12:27:30PM +0200, Andrei Neagoe wrote:
> Hi,
> I was trying today to play with drbd's settings and benchmark the results in
> order to obtain the best performance.
> Here is my test setup:
> 2 identical machines with sas storage boxes. Each machine has two 2TB device
> (in my case /dev/sdb and /dev/sdc) that I mirror over drbd and on top of them
> there's LVM set up. The nodes share a gbit link dedicated for drbd traffic.
> After the initial sync which took something around 20 hours to finish, I've
> created the LVM volume and formatted using ext3 FS. Then I started to play
> around with params like al-extents, unplug-watermark, maxbuffers, max-epoch by
> changing the  values and doing a drbdadm adjust all on each node (of course
> after copying the config file accordingly). In the begining it went pretty
> well, maximum value attained by dd test over drbd was 28.9 MB/s:
> [root at erebus testing]# dd if=/dev/zero of=test.dat bs=1G count=1 oflag=dsync
> 1+0 records in
> 1+0 records out
> 1073741824 bytes (1.1 GB) copied, 37.1114 seconds, 28.9 MB/s
> The configuration used is described in the end. After a couple more tests, I
> noticed a big impact on performance, getting around 19-20 MB/s so I checked /
> proc/drbd to see what's going on. Surprisingly, it was doing a full resync on
> one of the disks. Problem is, I don't understand why, as normally it should
> only resync discrepancies.

if you change anything in the config file that changes "disk"
parameters (like on-io-error, size, fencing, use-bmbv, ...),
which causes drbdadm adjust to think it needs to detach/attach, and you
do that while being primary, you get a full sync.

this is unfortunate, and there should probably
be a dialog to warn you about it.

if you detach a Primary, then reattach, it will receive a full sync.
you need to make it secondary first, if you want to avoid that.
detaching, then reattaching a secondary will only receive an
"incremental" resync, which typically is a few KB or nothing at all,
depending on the timing.

if this is not what happened for you, read the kernel log,
typically drbd tells you why a resync was necessary.

: Lars Ellenberg                           http://www.linbit.com :
: DRBD/HA support and consulting             sales at linbit.com :
: LINBIT Information Technologies GmbH      Tel +43-1-8178292-0  :
: Vivenotgasse 48, A-1120 Vienna/Europe     Fax +43-1-8178292-82 :
please don't Cc me, but send to list -- I'm subscribed

More information about the drbd-user mailing list