[DRBD-user] Help: Replacing A Dead Node

Lars Ellenberg lars.ellenberg at linbit.com
Mon Jul 28 23:20:19 CEST 2008

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On Mon, Jul 28, 2008 at 12:42:01PM -0700, Mike Sweetser - Adhost wrote:
> I have a two-server DRBD primary/secondary configuration, running
> Heartbeat.  Recently, the primary server had a drive failure and had to
> be rebuilt, leaving the secondary server running the cluster (and quite
> well, I may add).
> The time has come to bring the original server back into the fold, which
> means bringing it back into the DRBD fold, but WITHOUT damaging the
> existing data.  This is my plan to do so - am I on the right track?  Am
> I offbase?  Is this the right procedure?
> * Copy /etc/drbd.conf from the working node to the new node
> * Create the metadata on the new node with: drbd create-md all

that should be perfectly enough to ensure that after

> * Attach the DRBD device on the new node: drbdadm attach all

it is now StandAlone Secondary Inconsistent.
since an Inconsistent device cannot become SyncSource
(and a Primary device will refuse to become SyncTarget)
just make sure your good data is Primary still,
and say "drbdadm adjust all" now.

you probably should not start the cluster manager yet, though, unless
you are sure that your sedtup knows how to deal with a degraded drbd.

> * Make sure that the new node is in secondary mode: drbdadm secondary all

if you did not make it Primary, and there is no cluster manager,
and you just created fresh meta data that marks it as Inconsistent,
there is no way it possibly could be Primary...

> * Connect the new node to the working node, discarding the existing data
> so that it's overwritten by the good data: drbdadm -- --discard-my-data
> connect all

the "--discard-my-data" will normally be ignored, it is only used
to resolve the situation after split-brain, when the data sets have
diverged already, and drbd needs help to decide which modification
should be thrown away.

it is not usefull in this situation.

> Is this correct?

it would have worked,
  re-init meta data (drbdadm create-md)
  drbdadm attach
  verify that it is StandAlone Secondary Inconsistent now,

  verify that the good data is still in Primary mode

  drbdadm adjust (possibly on the good node as well)
should be enough to get it fully synced.

> Will I have to take down Heartbeat while I reconnect?

you can keep heartbeat running on the "Primary", no problem.

but while it is possible to configure heartbeat + drbd to behave nicely
even while drbd is degraded, it is easy to get it wrong.

I'd recommend to keep heartbeat off
on the newly to be integrated node, until it is ready.

: Lars Ellenberg                           http://www.linbit.com :
: DRBD/HA support and consulting             sales at linbit.com :
: LINBIT Information Technologies GmbH      Tel +43-1-8178292-0  :
: Vivenotgasse 48, A-1120 Vienna/Europe     Fax +43-1-8178292-82 :
please don't Cc me, but send to list -- I'm subscribed

More information about the drbd-user mailing list