[DRBD-user] How Long Can a Primary Stay Disconnected from a Secondary?

Florian Haas florian.haas at linbit.com
Thu Mar 3 19:37:31 CET 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On 03/03/2011 06:20 PM, Digimer wrote:
> On 03/03/2011 12:09 PM, Florian Haas wrote:
>> On 2011-03-03 16:13, Digimer wrote:
>>> On 03/03/2011 10:05 AM, Robinson, Eric wrote:
>>>> I am thinking about taking a secondary node offline for maintenance
>>>> during production hours. I assume it cannot stay offline forever because
>>>> eventually the primary would run out of space to record changes that
>>>> need to be sync'd. Is there a way to know in advance how long it is safe
>>>> to stay disconnected without resync'ing?
>>> As I understand it, dirty blocks are recorded in memory. I am sure there
>>> is an upper limit to the counters, but I would hazard a guess that you
>>> would run out of RAM first.
>> Gaaah, we'd never be crash safe if we recorded this in memory only. One
>> node fails, we track the sync, the other node fails, boom the sync
>> information is gone. No, DRBD is a wee bit smarter than that. :)
>> You can overwrite your whole device while disconnected. DRBD will simply
>> eventually have flipped all the bits in the bitmap. Which it keeps on
>> stable storage, obviously.
>> Florian
> Today I learned...
> I swear that I read that dirty blocks were kept in memory somewhere...
> Was this perhaps true some time in the past, or did I have a very vivid
> and inaccurate daydream at some point?

I'm afraid I can't comment on your daydreams. :)

But bitmap persistence is admittedly mildly involved: the bitmap always
lives in memory -- well actually not always, but only when in
disconnected mode or during resync, but that's a given. Now, when an
Activity Log extent goes cold, DRBD writes *the corresponding portion*
of the bitmap to the metadata area on disk.

If the primary crashes while disconnected, and then recovers, DRBD knows
which AL extents were hot at the time of the crash, sets _all_ the bits
in the bitmap that correspond to hot AL extents, and then adds the
bitmap information it can read from persistent metadata (which
corresponds to areas that were cold).

And if you think this through, you'll quickly realize that that's quite
clever, because AL granularity is a thousand fold less than the
granularity of the bitmap. So rather than having to go to disk whenever
we flip a bit (which would kill performance), we do so only when we
expire an AL extent. Which occurs a lot less often, and is also
configurable. And we still get the same persistence and crash safety.

Does this help?


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20110303/426604f6/attachment.pgp>

More information about the drbd-user mailing list