Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Wed, Jun 04, 2008 at 08:42:50PM +0200, Simon Carey-Smith wrote:
> Does anyone have a simple explanation of the difference between these 3
> commands, and in which situations to use them ?
a data set may be
inconsistent (invalid, not recoverable to any sane state, not usable)
- drbd refuses to let you access inconsistent data
consistent, which may be
- up to date, or
- possibly outdated (we may or may not be up to date, we just don't
know yet, because we cannot talk to the peer), or
- outdated (we know we are out of date, because we know by other
means that the other node has been changed without being able to
replicate those changes to us)
when drbd creates new meta data,
it is initialized to "inconsistent, all dirty"
when drbd connects two nodes,
the state of the data sets is compared.
it is slightly more complicated than the following, but basically:
if both nodes are inconsistent, nothing is synced.
if one node is inconsistent, and the other is consistent,
the inconsistent one becomes sync target.
if both are consistent, and our algorithm detects that one is a direct
successor of the other, the "newer", more up-to-date one becomes sync
source.
if both are consistent, but our algorithm detects that they have
_both_ been modified in diverging, incompatible ways, we recognize
that there must have been a period of "resource internal split brain".
the default behaviour after split brain recovery is to disconnect, and
refuse to connect again. because you have two consistent data sets
that are different, you have to chose which one you want to use from
now on. this can be somewhat automated (dangerous!) by the "after
split brain handlers".
on resync, transfered are only blocks that have been changed
(marked dirty) since the last time the nodes have been in sync.
> > "drbdadm invalidate $r"
forcefully sets the data set to "inconsistent",
and marks all block dirty - thus initiating a full sync,
making this node sync target.
use this if you have reason to believe that the information in the dirty
bitmap does not match the changed blocks on disk, e.g. when you
"accidentally" bypassed drbd (mounted directly), or when you replaced
the drbd data store device but kept the external drbd meta data device.
> > "drbdadm --over-write-data-of-peer $r"
overrides DRBDs notion of the state, forcefully sets this node to
"consistent". if it was already consistent, this becomes a no-op.
if the other node is (still) inconsistent, this node will become
sync source, effectively overwriting the data of the peer.
if the other node is consistent as well, nothing will change,
so you cannot use this to recover from diverging data sets.
use this once, right after you newly created the drbd meta data area,
on one node only, to pick the sync source of the initial sync.
> > "drbdadm -- --discard-my-data connect $r"
if "after split brain" was detected, and therefore the nodes refuse to
talk to each other, you can pick one to become sync target.
this forcefully marks this node as inconsistent, but in contrast to the
"invalidate" it does not mark any additional blocks as dirty, so we only
need to transfer the union of marked-as-dirty blocks.
use this to recover from diverging data sets.
--
: Lars Ellenberg http://www.linbit.com :
: DRBD/HA support and consulting sales at linbit.com :
: LINBIT Information Technologies GmbH Tel +43-1-8178292-0 :
: Vivenotgasse 48, A-1120 Vienna/Europe Fax +43-1-8178292-82 :
__
please don't Cc me, but send to list -- I'm subscribed