[DRBD-user] drbd resource fencing - 2nd try with more information

Dominik Klein dk at in-telegence.net
Tue Dec 4 10:48:43 CET 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hi Florian, drbd-users

I see I have been very short on info here. Sorry for that.

So I want to learn about resource fencing in DRBD. I read the recent 
thread about it and read about the different modes DRBD offers for 
fencing. As I dont have a STONITH device, I went for resource-only.

Here's my configuration, what I did and what I got.

Nodes: dktest1debian, dktest2debian
OS: Debian Etch 32 bit
DRBD: 8.0.7
Heartbeat: 2.1.12-24
Kernel 2.6.18-4-686
Network: eth0 10.250.250.0/24 for drbd and heartbeat
	 eth1 10.2.50.0/24 for normal networking and heartbeat

ha.cf:
keepalive 2
deadtime 30
warntime 10
ucast eth1 10.2.50.100
ucast eth0 10.250.250.100
node    dktest1debian
node    dktest2debian
ping 10.2.50.32
ping 10.2.50.2
ping 10.2.50.34
ping 10.2.50.250
ping 10.2.50.11
respawn root /usr/lib/heartbeat/pingd -p /var/run/pingd.pid -d 5s -m 100
respawn hacluster /usr/lib/heartbeat/dopd
apiauth dopd gid=haclient uid=hacluster
use_logd        yes
crm on

drbd.conf:
global {
   usage-count no;
}
common {
  handlers {
    outdate-peer "/usr/lib/heartbeat/drbd-peer-outdater";
  }
}
resource drbd2 {
   protocol C;
   startup {
     wfc-timeout  15;
     degr-wfc-timeout 120;
   }
   disk {
     on-io-error   detach;
     fencing       resource-only;
   }
   net {
     after-sb-0pri disconnect;
     after-sb-1pri disconnect;
     after-sb-2pri disconnect;
     rr-conflict disconnect;
     max-buffers      20480;
     max-epoch-size   16384;
     unplug-watermark 20480;
   }
   syncer {
     rate 140M;
   }
   on dktest1debian {
     device     /dev/drbd2;
     disk       /dev/sda3;
     address    10.250.250.100:7790;
     meta-disk  internal;
   }
   on dktest2debian {
     device    /dev/drbd2;
     disk      /dev/sda3;
     address   10.250.250.101:7790;
     meta-disk internal;
   }
}

Now I do:
reboot both nodes
rm /var/lib/heartbeat/crm/* on both nodes

So we start off real clean.

/etc/init.d/heartbeat start on both nodes

Wait to see online/online and that a DC has been chosen, dopd is started.

At this point, I have no resources configured and Linux-HA is running 
with all defaults (no STONITH).

Now I promote drbd2 on dktest1debian.

After that I unplug the DRBD link (eth0)

Then in the logs I see:

Dec  4 10:27:39 dktest1debian drbd-peer-outdater: [2674]: debug: drbd 
peer: dktest2debian
Dec  4 10:27:39 dktest1debian drbd-peer-outdater: [2674]: debug: drbd 
resource: drbd2
Dec  4 10:27:39 dktest1debian drbd-peer-outdater: [2674]: ERROR: 
cl_free: Bad magic number in object at 0xbfc405e8
Dec  4 10:27:39 dktest1debian drbd-peer-outdater: [2674]: info: Dumping 
cl_malloc item @ 0xbfc405e8, bucket address: 0xbfc405d8
Dec  4 10:27:39 dktest1debian /usr/lib/heartbeat/dopd: [2643]: debug: 
Connecting channel
Dec  4 10:27:39 dktest1debian /usr/lib/heartbeat/dopd: [2643]: debug: 
Client outdater (0x8055bc8) connected
Dec  4 10:27:39 dktest1debian /usr/lib/heartbeat/dopd: [2643]: debug: 
invoked: outdater
Dec  4 10:27:39 dktest1debian /usr/lib/heartbeat/dopd: [2643]: debug: 
Processed 0 messages
Dec  4 10:27:39 dktest1debian /usr/lib/heartbeat/dopd: [2643]: debug: 
Deleting outdater (0x8055bc8) from mainloop
Dec  4 10:27:39 dktest1debian /usr/lib/heartbeat/dopd: [2643]: debug: 
connection from client closed

What does this mean (bad magic number)?

On dktest2debian I see:

/proc/drbd:
version: 8.0.7 (api:86/proto:86)
GIT-hash: cf14288833afe95db396075f8530a5960d29e498 build by 
root at dktest2debian, 2007-12-04 09:11:56

  2: cs:WFConnection st:Secondary/Unknown ds:UpToDate/DUnknown C r---
     ns:0 nr:9775216 dw:9775216 dr:0 al:0 bm:597 lo:0 pe:0 ua:0 ap:0
         resync: used:0/31 hits:610355 misses:597 starving:0 dirty:0 
changed:597
         act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0

So the resource is still upToDate.

I tested to manually make it primary but this was refused:

drbdadm primary drbd2
State change failed: (-2) Refusing to be Primary without at least one 
UpToDate disk
Command 'drbdsetup /dev/drbd2 primary' terminated with exit code 11

After that, I can see the resource is outdated.

cat /proc/drbd
version: 8.0.7 (api:86/proto:86)
GIT-hash: cf14288833afe95db396075f8530a5960d29e498 build by 
root at dktest2debian, 2007-12-04 09:11:56

  2: cs:WFConnection st:Secondary/Unknown ds:Outdated/DUnknown C r---
     ns:0 nr:9775216 dw:9775216 dr:0 al:0 bm:597 lo:0 pe:0 ua:0 ap:0
         resync: used:0/31 hits:610355 misses:597 starving:0 dirty:0 
changed:597
         act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0

Maybe you can help me now. Again - sorry for the ****-mail yesterday - 
there was really nothing in there.

Regards
Dominik



More information about the drbd-user mailing list