[DRBD-user] How Long Can a Primary Stay Disconnected from a Secondary?
linux at alteeve.com
Thu Mar 3 19:49:28 CET 2011
On 03/03/2011 01:37 PM, Florian Haas wrote:
> On 03/03/2011 06:20 PM, Digimer wrote:
>> On 03/03/2011 12:09 PM, Florian Haas wrote:
>>> On 2011-03-03 16:13, Digimer wrote:
>>>> On 03/03/2011 10:05 AM, Robinson, Eric wrote:
>>>>> I am thinking about taking a secondary node offline for maintenance
>>>>> during production hours. I assume it cannot stay offline forever because
>>>>> eventually the primary would run out of space to record changes that
>>>>> need to be sync'd. Is there a way to know in advance how long it is safe
>>>>> to stay disconnected without resync'ing?
>>>> As I understand it, dirty blocks are recorded in memory. I am sure there
>>>> is an upper limit to the counters, but I would hazard a guess that you
>>>> would run out of RAM first.
>>> Gaaah, we'd never be crash safe if we recorded this in memory only. One
>>> node fails, we track the sync, the other node fails, boom the sync
>>> information is gone. No, DRBD is a wee bit smarter than that. :)
>>> You can overwrite your whole device while disconnected. DRBD will simply
>>> eventually have flipped all the bits in the bitmap. Which it keeps on
>>> stable storage, obviously.
>> Today I learned...
>> I swear that I read that dirty blocks were kept in memory somewhere...
>> Was this perhaps true some time in the past, or did I have a very vivid
>> and inaccurate daydream at some point?
> I'm afraid I can't comment on your daydreams. :)
> But bitmap persistence is admittedly mildly involved: the bitmap always
> lives in memory -- well actually not always, but only when in
> disconnected mode or during resync, but that's a given. Now, when an
> Activity Log extent goes cold, DRBD writes *the corresponding portion*
> of the bitmap to the metadata area on disk.
> If the primary crashes while disconnected, and then recovers, DRBD knows
> which AL extents were hot at the time of the crash, sets _all_ the bits
> in the bitmap that correspond to hot AL extents, and then adds the
> bitmap information it can read from persistent metadata (which
> corresponds to areas that were cold).
> And if you think this through, you'll quickly realize that that's quite
> clever, because AL granularity is a thousand fold less than the
> granularity of the bitmap. So rather than having to go to disk whenever
> we flip a bit (which would kill performance), we do so only when we
> expire an AL extent. Which occurs a lot less often, and is also
> configurable. And we still get the same persistence and crash safety.
> Does this help?
Actually, it does help oodles. Thanks for taking the time to write that
out. You know, you should do a presentation on DRBD internals... ;)
E-Mail: digimer at alteeve.com
Node Assassin: http://nodeassassin.org
More information about the drbd-user