[DRBD-user] Following Up

Florian Haas florian.haas at linbit.com
Wed Nov 14 10:45:00 CET 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


I am afraid you are overestimating the asynchronicity of protocol A.

> For example, Im maintaining a 2TB device but have my drbd on the WAN link
> which is 3Meg pipe (2 bonded T1s) obviously the data will never be able to
> keep up with the writes during the day but at night when the business slows
> to a crawl I should be able to 'catch up' as long as the system doesn't
> lose track of where it is.

When you have DRBD configured to use protocol A, these two events occur 
simultaneously on every write to the device:
1. Local disk write;
2. Replication packet handoff to the TCP send buffer.

Immediately after it reaches the network send buffer, the application learns 
that the I/O request is done and can proceed (this is documented in the 
drbd.conf man page).

However, I/O will inevitably stall when the send buffer is filled up to the 
lid. Since the replication packet gets cleared from the send buffer only 
after TCP acknowledgment, if DRBD runs on a narrow pipe, under I/O load this 
will happen rather quickly. And when that happens, your performance will 
suffer. DRBD doesn't play catch-up to the extent that you would like. 
Protocol A will allow your Secondary to lag behind your Primary by perhaps a 
few megabytes' worth of I/O if you tune your network stack, but that's about 
it. Anything above that will bite a significant chunk off your Primary's 

> As a followup, Im also seeing a lot of stalled on the LAN using Protocol A
> and I set syncer to 100k or 150k

Also, do not misunderstand the implications of setting your syncer rate. The 
syncer rate only affects initial sync and background resync, _not_ your 
normal replication traffic. Normal replication traffic always uses whatever 
bandwidth is available. 

> When I goto drbdadm down all.   I get the
> drbd is not loaded and I have to do a reboot in order to get control of it
> again. (A kill would probably work too)

If you're getting the "is the module loaded?" message on drbdadm down, DRBD 
may have oopsed. So you may be dealing with a bug. Can you post a dmesg 
excerpt so we can look into this more closely? When you do, don't forget to 
state your exact DRBD and kernel versions, too.


: Florian G. Haas                               Tel +43-1-8178292-60  :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82  :
: Vivenotgasse 48, A-1120 Vienna, Austria       http://www.linbit.com :

This e-mail is solely for use by the intended recipient(s). Information 
contained in this e-mail and its attachments may be confidential, privileged 
or copyrighted. If you are not the intended recipient you are hereby formally 
notified that any use, copying, disclosure or distribution of the contents of 
this e-mail, in whole or in part, is prohibited. Also please notify 
immediately the sender by return e-mail and delete this e-mail from your 
system. Thank you for your co-operation.

More information about the drbd-user mailing list