<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 07/01/2015 10:05 PM, Digimer wrote:<br>
<blockquote cite="mid:55941DC9.3020302@alteeve.ca" type="cite">
<pre wrap="">You need to setup proper, working fencing.
Configure stonith in pacemaker, test it by crashing each node (echo c >
/proc/sysrq-trigger) and watch that they get rebooted. Once that works,
configure DRBD to use 'fencing resource-and-stonith;' and use the
'crm-{un,}fence-peer.sh' fence/unfence handlers.
digimer
On 01/07/15 11:21 AM, Muhammad Sharfuddin wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hello,
First I setup a perfect dual primary setup, then I configured the
pacemaker cluster resource to start the drbd resource. As soon cluster
starts
the drbd resource splait-brain occurs, please let me know what I am
doing wrong.
Here is the drbd configuration:
global_common.conf:
global { usage-count no; }
common {
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;
reboot -f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;
reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh;
/usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger
; halt -f";
split-brain "/usr/lib/drbd/notify-split-brain.sh root";
}
startup { wfc-timeout 0; degr-wfc-timeout 120; become-primary-on both; }
disk { on-io-error detach; al-extents 3389; }
net {
allow-two-primaries; after-sb-0pri discard-zero-changes;
after-sb-1pri discard-secondary; after-sb-2pri disconnect;
max-buffers 8000; max-epoch-size 8000;
sndbuf-size 0; verify-alg md5;
ping-int 2; ping-timeout 2;
connect-int 2; timeout 5; ko-count 5;
}
}
r0.res:
resource r0 {
device /dev/drbd_r0 minor 0;
meta-disk internal;
on node1 {
disk "/dev/disk/by-path/pci-0000:00:10.0-scsi-0:0:1:0-part1";
address 172.16.241.131:7780;
}
on node2 {
disk "/dev/disk/by-path/pci-0000:00:10.0-scsi-0:0:1:0-part1";
address 172.16.241.132:7780;
}
syncer { rate 100M; }
}
below are the cluster drbd resource configuration:
primitive p-drbd ocf:linbit:drbd \
params drbd_resource="r0" \
op monitor interval="50" role="Master" timeout="30" \
op monitor interval="60" role="Slave" timeout="30" \
op start interval="0" timeout="240" \
op stop interval="0" timeout="100"
ms ms-drbd p-drbd \
meta master-max="2" clone-max="2" notify="true" interleave="true"
when cluster starts the drbd resource, /var/log/messages:
Jul 1 19:04:40 node2 cibadmin[4754]: notice: crm_log_args: Invoked:
cibadmin -p -R -o resources
Jul 1 19:04:41 node2 kernel: [ 494.932537] events: mcg drbd: 3
Jul 1 19:04:41 node2 kernel: [ 494.943147] drbd: initialized. Version:
8.4.3 (api:1/proto:86-101)
Jul 1 19:04:41 node2 kernel: [ 494.943151] drbd: GIT-hash:
89a294209144b68adb3ee85a73221f964d3ee515 build by phil@fat-tyre,
2013-02-05 15:35:49
Jul 1 19:04:41 node2 kernel: [ 494.943153] drbd: registered as block
device major 147
Jul 1 19:04:42 node2 kernel: [ 495.981244] d-con r0: Starting worker
thread (from drbdsetup [4801])
Jul 1 19:04:42 node2 kernel: [ 495.981560] block drbd0: disk( Diskless
-> Attaching )
Jul 1 19:04:42 node2 kernel: [ 495.982168] d-con r0: Method to ensure
write ordering: flush
Jul 1 19:04:42 node2 kernel: [ 495.982174] block drbd0: max BIO size =
1048576
Jul 1 19:04:42 node2 kernel: [ 495.982179] block drbd0: drbd_bm_resize
called with capacity == 4192056
Jul 1 19:04:42 node2 kernel: [ 495.982201] block drbd0: resync bitmap:
bits=524007 words=8188 pages=16
Jul 1 19:04:42 node2 kernel: [ 495.982204] block drbd0: size = 2047 MB
(2096028 KB)
Jul 1 19:04:42 node2 kernel: [ 495.983736] block drbd0: bitmap READ of
16 pages took 1 jiffies
Jul 1 19:04:42 node2 kernel: [ 495.983757] block drbd0: recounting of
set bits took additional 0 jiffies
Jul 1 19:04:42 node2 kernel: [ 495.983760] block drbd0: 0 KB (0 bits)
marked out-of-sync by on disk bit-map.
Jul 1 19:04:42 node2 kernel: [ 495.983767] block drbd0: disk(
Attaching -> UpToDate )
Jul 1 19:04:42 node2 kernel: [ 495.983771] block drbd0: attached to
UUIDs 62EE6E5BA23AC477:37CECFD41B2C30A4:1B8441319CED9865:1B8341319CED9865
Jul 1 19:04:42 node2 attrd[4231]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-p-drbd (1000)
Jul 1 19:04:42 node2 attrd[4231]: notice: attrd_perform_update: Sent
update 24: master-p-drbd=1000
Jul 1 19:04:42 node2 attrd[4231]: notice: attrd_perform_update: Sent
update 27: master-p-drbd=1000
Jul 1 19:04:42 node2 crmd[4233]: notice: process_lrm_event: LRM
operation p-drbd_start_0 (call=68, rc=0, cib-update=18, confirmed=true) ok
Jul 1 19:04:42 node2 kernel: [ 495.993653] d-con r0: conn( StandAlone
-> Unconnected )
Jul 1 19:04:42 node2 kernel: [ 496.044937] d-con r0: Starting receiver
thread (from drbd_w_r0 [4802])
Jul 1 19:04:42 node2 kernel: [ 496.045820] d-con r0: receiver (re)started
Jul 1 19:04:42 node2 kernel: [ 496.045830] d-con r0: conn( Unconnected
-> WFConnection )
Jul 1 19:04:42 node2 crmd[4233]: notice: process_lrm_event: LRM
operation p-drbd_notify_0 (call=71, rc=0, cib-update=0, confirmed=true) ok
Jul 1 19:04:42 node2 crmd[4233]: notice: process_lrm_event: LRM
operation p-drbd_notify_0 (call=74, rc=0, cib-update=0, confirmed=true) ok
Jul 1 19:04:42 node2 crmd[4233]: notice: process_lrm_event: LRM
operation p-drbd_promote_0 (call=77, rc=0, cib-update=19, confirmed=true) ok
Jul 1 19:04:42 node2 kernel: [ 496.197480] block drbd0: role(
Secondary -> Primary )
Jul 1 19:04:42 node2 attrd[4231]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-p-drbd (10000)
Jul 1 19:04:42 node2 attrd[4231]: notice: attrd_perform_update: Sent
update 31: master-p-drbd=10000
Jul 1 19:04:42 node2 crmd[4233]: notice: process_lrm_event: LRM
operation p-drbd_notify_0 (call=80, rc=0, cib-update=0, confirmed=true) ok
Jul 1 19:04:42 node2 crmd[4233]: notice: process_lrm_event: LRM
operation p-drbd_monitor_50000 (call=83, rc=8, cib-update=20,
confirmed=false) master
Jul 1 19:04:42 node2 crmd[4233]: notice: process_lrm_event:
node2-p-drbd_monitor_50000:83 [ ]
Jul 1 19:04:42 node2 kernel: [ 496.342704] d-con r0: Handshake
successful: Agreed network protocol version 101
Jul 1 19:04:42 node2 kernel: [ 496.342890] d-con r0: conn(
WFConnection -> WFReportParams )
Jul 1 19:04:42 node2 kernel: [ 496.342893] d-con r0: Starting asender
thread (from drbd_r_r0 [4821])
Jul 1 19:04:42 node2 kernel: [ 496.356028] block drbd0:
drbd_sync_handshake:
Jul 1 19:04:42 node2 kernel: [ 496.356033] block drbd0: self
62EE6E5BA23AC477:37CECFD41B2C30A4:1B8441319CED9865:1B8341319CED9865
bits:0 flags:0
Jul 1 19:04:42 node2 kernel: [ 496.356035] block drbd0: peer
20FA2D65F94F24B7:37CECFD41B2C30A5:1B8441319CED9865:1B8341319CED9865
bits:0 flags:0
Jul 1 19:04:42 node2 kernel: [ 496.356038] block drbd0:
uuid_compare()=100 by rule 90
Jul 1 19:04:42 node2 kernel: [ 496.356041] block drbd0: helper
command: /sbin/drbdadm initial-split-brain minor-0
Jul 1 19:04:42 node2 kernel: [ 496.358760] block drbd0: helper
command: /sbin/drbdadm initial-split-brain minor-0 exit code 0 (0x0)
Jul 1 19:04:42 node2 kernel: [ 496.358776] block drbd0: Split-Brain
detected but unresolved, dropping connection!
Jul 1 19:04:42 node2 kernel: [ 496.358811] block drbd0: helper
command: /sbin/drbdadm split-brain minor-0
Jul 1 19:04:42 node2 notify-split-brain.sh[4966]: invoked for r0/0 (drbd0)
Jul 1 19:04:42 node2 kernel: [ 496.385210] d-con r0: meta connection
shut down by peer.
Jul 1 19:04:42 node2 kernel: [ 496.385225] d-con r0: conn(
WFReportParams -> NetworkFailure )
Jul 1 19:04:42 node2 kernel: [ 496.385228] d-con r0: asender terminated
Jul 1 19:04:42 node2 kernel: [ 496.385229] d-con r0: Terminating drbd_a_r0
Jul 1 19:04:42 node2 kernel: [ 496.389939] block drbd0: helper
command: /sbin/drbdadm split-brain minor-0 exit code 0 (0x0)
Jul 1 19:04:42 node2 kernel: [ 496.389961] d-con r0: conn(
NetworkFailure -> Disconnecting )
Jul 1 19:04:42 node2 kernel: [ 496.389964] d-con r0: error receiving
ReportState, e: -5 l: 0!
Jul 1 19:04:42 node2 kernel: [ 496.390147] d-con r0: Connection closed
Jul 1 19:04:42 node2 kernel: [ 496.390174] d-con r0: conn(
Disconnecting -> StandAlone )
Jul 1 19:04:42 node2 kernel: [ 496.390176] d-con r0: receiver terminated
Jul 1 19:04:42 node2 kernel: [ 496.390177] d-con r0: Terminating drbd_r_r0
--
Regards,
Muhammad Sharfuddin
_______________________________________________
drbd-user mailing list
<a class="moz-txt-link-abbreviated" href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a>
<a class="moz-txt-link-freetext" href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<font size="-1">Thanks for quick help, sbd stonith already
configured properly, and per your recommendation<br>
I googled for the handler and found following working perfectly:<br>
</font><br>
<pre>fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
after-resync-target "/usr/lib/drbd/crm-unfence-peer.sh";</pre>
<font size="-1">Thanks a lot ;-)<br>
<br>
--</font><br>
<div class="moz-signature"><font size="1">Regards,<br>
<br>
</font>
<font size="2">Muhammad Sharfuddin<br>
</font></div>
</body>
</html>