Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi.
Please ignore the previous email since this one has updated information.
I just get up and running a cluster with DRBD + OCFS2 + Pacemaker on a OpenSUSE 12.1.
Im testing the setup of it, so I start to stop all the resouces from the top to the bottom by the
crm shell.
The stop task went through without any trouble and the DRBD resources were take down clean. When I
try to start the DRBD resource both servers are kept tryng to get DRBD up With the DRBD resouce
going up and down and so on.
Sometimes I need to stop the resource on the crm and use "drbdadm up export" on both servers then
start the resource on the crm. This way everything goes ok.
Trying to see what was going on, i used "cat /proc/drbd" and i could see this:
version: 8.3.11 (api:88/proto:86-96)
srcversion: 21CA73FE6D7D9C67B0C6AB2
0: cs:StandAlone ro:Secondary/Unknown ds:Attaching/DUnknown r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:1 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
jupiter:~ #
Message from syslogd at jupiter at Mar 22 13:17:55 ...
kernel:[55900.328287] Oops: 0000 [#116] PREEMPT SMP
Message from syslogd at jupiter at Mar 22 13:17:55 ...
kernel:[55900.351721] Stack:
Message from syslogd at jupiter at Mar 22 13:17:55 ...
kernel:[55900.351731] Call Trace:
Message from syslogd at jupiter at Mar 22 13:17:55 ...
kernel:[55900.351754] Code: 24 18 4c 8b 6c 24 20 4c 8b 74 24 28 4c 8b 7c 24 30 48 83 c4 38 c3 66 90
55 53 89 f3 48 83 ec 18 48 8b b7 a0 00 00 00 48 8b 46 20
Message from syslogd at jupiter at Mar 22 13:17:55 ...
kernel:[55900.351771] CR2: 0000000000000030
cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: 21CA73FE6D7D9C67B0C6AB2
0: cs:Unconfigured
jupiter:~ #
Message from syslogd at jupiter at Mar 22 13:17:58 ...
kernel:[55903.426097] Oops: 0000 [#117] PREEMPT SMP
Message from syslogd at jupiter at Mar 22 13:17:58 ...
kernel:[55903.523440] Stack:
Message from syslogd at jupiter at Mar 22 13:17:58 ...
kernel:[55903.523452] Call Trace:
Message from syslogd at jupiter at Mar 22 13:17:58 ...
kernel:[55903.523480] Code: 24 18 4c 8b 6c 24 20 4c 8b 74 24 28 4c 8b 7c 24 30 48 83 c4 38 c3 66 90
55 53 89 f3 48 83 ec 18 48 8b b7 a0 00 00 00 48 8b 46 20
Message from syslogd at jupiter at Mar 22 13:17:58 ...
kernel:[55903.523500] CR2: 0000000000000030
cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: 21CA73FE6D7D9C67B0C6AB2
0: cs:WFConnection ro:Secondary/Unknown ds:Consistent/DUnknown C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
The last time I tryed Icould get this:
jupiter:~ # cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: 21CA73FE6D7D9C67B0C6AB2
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
ns:0 nr:0 dw:0 dr:672 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[===================>] sync'ed:100.0% (0/0)K
finish: 0:00:00 speed: 0 (0) K/sec
jupiter:~ # cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
srcversion: 21CA73FE6D7D9C67B0C6AB2
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:0 nr:0 dw:0 dr:672 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Thats weird, since I have the DRBD resouces configured as dual primary.
On the /var/log/messages, while the problem is hapenning , this was logged :
Mar 22 13:18:16 jupiter kernel: [55920.551847] block drbd0: helper command: /sbin/drbdadm
before-resync-source minor-0 exit code 0 (0x9)
Mar 22 13:18:16 jupiter kernel: [55920.551855] block drbd0: conn( WFBitMapS -> SyncSource ) pdsk(
Consistent -> Inconsistent )
Mar 22 13:18:16 jupiter kernel: [55920.551862] block drbd0: Began resync as SyncSource (will sync 0
KB [0 bits set]).
Mar 22 13:18:16 jupiter kernel: [55920.551872] block drbd0: updated sync UUID
AB844D96DE579455:78933A381DE3DA01:78923A381DE3DA01:E2205A8E9753F099
Mar 22 13:18:17 jupiter kernel: [55922.066881] block drbd0: Resync done (total 1 sec; paused 0 sec;
0 K/sec)
Mar 22 13:18:17 jupiter kernel: [55922.093250] block drbd0: updated UUIDs
AB844D96DE579455:0000000000000000:78933A381DE3DA01:78923A381DE3DA01
Mar 22 13:18:17 jupiter kernel: [55922.120245] block drbd0: conn( SyncSource -> Connected ) pdsk(
Inconsistent -> UpToDate )
Mar 22 13:18:18 jupiter kernel: [55922.673924] block drbd0: bitmap WRITE of 7243 pages took 506
jiffies
Mar 22 13:18:18 jupiter kernel: [55922.715472] block drbd0: 0 KB (0 bits) marked out-of-sync by on
disk bit-map.
Mar 22 13:20:07 jupiter cib: [10090]: info: cib_stats: Processed 1164 operations (2001.00us average,
0% utilization) in the last 10min
jupiter:~ # tail -200 /var/log/messages
Mar 22 13:18:12 jupiter crmd: [10094]: info: do_te_invoke: Processing graph 424
(ref=pe_calc-dc-1332433092-3616) derived from /var/lib/pengine/pe-input-1424.bz2
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 32 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 68: notify
resDRBD:0_pre_notify_demote_0 on artemis
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 70: notify
resDRBD:1_pre_notify_demote_0 on jupiter (local)
Mar 22 13:18:12 jupiter crmd: [10094]: info: do_lrm_rsc_op: Performing
key=70:424:0:8534b432-496c-4036-bcbb-09460824c604 op=resDRBD:1_notify_0 )
Mar 22 13:18:12 jupiter lrmd: [10091]: info: rsc:resDRBD:1 notify[1214] (pid 18325)
Mar 22 13:18:12 jupiter pengine: [10093]: notice: process_pe_message: Transition 424: PEngine Input
stored in: /var/lib/pengine/pe-input-1424.bz2
Mar 22 13:18:12 jupiter lrmd: [10091]: info: operation notify[1214] on resDRBD:1 for client 10094:
pid 18325 exited with return code 0
Mar 22 13:18:12 jupiter crmd: [10094]: info: send_direct_ack: ACK'ing resource op resDRBD:1_notify_0
from 70:424:0:8534b432-496c-4036-bcbb-09460824c604: lrm_invoke-lrmd-1332433092-3619
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_te_message: Processing (N)ACK
lrm_invoke-lrmd-1332433092-3619 from jupiter
Mar 22 13:18:12 jupiter crmd: [10094]: info: match_graph_event: Action resDRBD:1_notify_0 (70)
confirmed on jupiter (rc=0)
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_lrm_event: LRM operation resDRBD:1_notify_0
(call=1214, rc=0, cib-update=0, confirmed=true) ok
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_te_message: Processing (N)ACK
lrm_invoke-lrmd-1332433092-742 from artemis
Mar 22 13:18:12 jupiter crmd: [10094]: info: match_graph_event: Action resDRBD:0_notify_0 (68)
confirmed on artemis (rc=0)
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 33 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 30 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 6: demote
resDRBD:0_demote_0 on artemis
Mar 22 13:18:12 jupiter crmd: [10094]: info: match_graph_event: Action resDRBD:0_demote_0 (6)
confirmed on artemis (rc=0)
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 31 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 34 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 69: notify
resDRBD:0_post_notify_demote_0 on artemis
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 71: notify
resDRBD:1_post_notify_demote_0 on jupiter (local)
Mar 22 13:18:12 jupiter crmd: [10094]: info: do_lrm_rsc_op: Performing
key=71:424:0:8534b432-496c-4036-bcbb-09460824c604 op=resDRBD:1_notify_0 )
Mar 22 13:18:12 jupiter lrmd: [10091]: info: rsc:resDRBD:1 notify[1215] (pid 18348)
Mar 22 13:18:12 jupiter lrmd: [10091]: info: RA output: (resDRBD:1:notify:stdout)
Mar 22 13:18:12 jupiter lrmd: [10091]: info: operation notify[1215] on resDRBD:1 for client 10094:
pid 18348 exited with return code 0
Mar 22 13:18:12 jupiter crmd: [10094]: info: send_direct_ack: ACK'ing resource op resDRBD:1_notify_0
from 71:424:0:8534b432-496c-4036-bcbb-09460824c604: lrm_invoke-lrmd-1332433092-3623
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_te_message: Processing (N)ACK
lrm_invoke-lrmd-1332433092-3623 from jupiter
Mar 22 13:18:12 jupiter crmd: [10094]: info: match_graph_event: Action resDRBD:1_notify_0 (71)
confirmed on jupiter (rc=0)
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_lrm_event: LRM operation resDRBD:1_notify_0
(call=1215, rc=0, cib-update=0, confirmed=true) ok
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_te_message: Processing (N)ACK
lrm_invoke-lrmd-1332433092-743 from artemis
Mar 22 13:18:12 jupiter crmd: [10094]: info: match_graph_event: Action resDRBD:0_notify_0 (69)
confirmed on artemis (rc=0)
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 35 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 20 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 61: notify
resDRBD:0_pre_notify_stop_0 on artemis
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 62: notify
resDRBD:1_pre_notify_stop_0 on jupiter (local)
Mar 22 13:18:12 jupiter crmd: [10094]: info: do_lrm_rsc_op: Performing
key=62:424:0:8534b432-496c-4036-bcbb-09460824c604 op=resDRBD:1_notify_0 )
Mar 22 13:18:12 jupiter lrmd: [10091]: info: rsc:resDRBD:1 notify[1216] (pid 18376)
Mar 22 13:18:12 jupiter lrmd: [10091]: info: operation notify[1216] on resDRBD:1 for client 10094:
pid 18376 exited with return code 0
Mar 22 13:18:12 jupiter crmd: [10094]: info: send_direct_ack: ACK'ing resource op resDRBD:1_notify_0
from 62:424:0:8534b432-496c-4036-bcbb-09460824c604: lrm_invoke-lrmd-1332433092-3626
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_te_message: Processing (N)ACK
lrm_invoke-lrmd-1332433092-3626 from jupiter
Mar 22 13:18:12 jupiter crmd: [10094]: info: match_graph_event: Action resDRBD:1_notify_0 (62)
confirmed on jupiter (rc=0)
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_lrm_event: LRM operation resDRBD:1_notify_0
(call=1216, rc=0, cib-update=0, confirmed=true) ok
Mar 22 13:18:12 jupiter crmd: [10094]: info: process_te_message: Processing (N)ACK
lrm_invoke-lrmd-1332433092-744 from artemis
Mar 22 13:18:12 jupiter crmd: [10094]: info: match_graph_event: Action resDRBD:0_notify_0 (61)
confirmed on artemis (rc=0)
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 21 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_pseudo_action: Pseudo action 18 fired and confirmed
Mar 22 13:18:12 jupiter crmd: [10094]: info: te_rsc_command: Initiating action 1: stop
resDRBD:0_stop_0 on artemis
Mar 22 13:18:13 jupiter kernel: [55917.530955] block drbd0: State change failed: Refusing to be
Primary while peer is not outdated
Mar 22 13:18:13 jupiter kernel: [55917.555668] block drbd0: state = { cs:Connected
ro:Primary/Secondary ds:UpToDate/UpToDate r--- }
Mar 22 13:18:13 jupiter kernel: [55917.580546] block drbd0: wanted = { cs:TearDown
ro:Primary/Unknown ds:UpToDate/DUnknown s--- }
Mar 22 13:18:13 jupiter kernel: [55917.630162] block drbd0: peer( Secondary -> Unknown ) conn(
Connected -> TearDown ) pdsk( UpToDate -> Outdated )
Mar 22 13:18:13 jupiter kernel: [55917.655640] block drbd0: new current UUID
AB844D96DE579455:78923A381DE3DA01:E2205A8E9753F099:E21F5A8E9753F099
Mar 22 13:18:13 jupiter kernel: [55917.679495] block drbd0: asender terminated
Even getting the DRBD up by hand, sometimes the same errors show up as you can see below:
Mar 22 14:02:37 jupiter kernel: [58576.878810] Process drbdadm (pid: 18801, threadinfo
ffff880212c3c000, task ffff88021e7fa5c0)
Mar 22 14:02:37 jupiter kernel: [58576.878811] Stack:
Mar 22 14:02:37 jupiter kernel: [58576.878812] 00007fff79101258 0000000000000003 ffff88021e7fa5c0
ffff88021e464800
Mar 22 14:02:37 jupiter kernel: [58576.878815] 0000000000000000 ffffffff811653e5 00007fd00d7b01b0
0000000000413d10
Mar 22 14:02:37 jupiter kernel: [58576.878817] ffff880212c3c000 0000000000000030 ffff88021e464800
0000000000000000
Mar 22 14:02:37 jupiter kernel: [58576.878820] Call Trace:
Mar 22 14:02:37 jupiter kernel: [58576.878828] [<ffffffff811653e5>] do_vfs_ioctl+0x85/0x2e0
Mar 22 14:02:37 jupiter kernel: [58576.878831] [<ffffffff811656d8>] sys_ioctl+0x98/0xa0
Mar 22 14:02:37 jupiter kernel: [58576.878840] [<ffffffff815a4c12>] system_call_fastpath+0x16/0x1b
Mar 22 14:02:37 jupiter kernel: [58576.878844] [<00007fd00d4e4f67>] 0x7fd00d4e4f66
Mar 22 14:02:37 jupiter kernel: [58576.878845] Code: 24 18 4c 8b 6c 24 20 4c 8b 74 24 28 4c 8b 7c 24
30 48 83 c4 38 c3 66 90 55 53 89 f3 48 83 ec 18 48 8b b7 a0 00 00 00 48 8b 46 20
Mar 22 14:02:37 jupiter kernel: [58576.878856] RIP [<ffffffff81468753>] sock_ioctl+0x13/0x2b0
Mar 22 14:02:37 jupiter kernel: [58576.878858] RSP <ffff880212c3dec8>
Mar 22 14:02:37 jupiter kernel: [58576.878859] CR2: 0000000000000030
Mar 22 14:02:37 jupiter kernel: [58576.878872] ---[ end trace 527c00f5943cff74 ]---
Mar 22 14:02:37 jupiter kernel: [58576.878971] block drbd0: helper command: /sbin/drbdadm
before-resync-target minor-0 exit code 0 (0x9)
Mar 22 14:02:37 jupiter kernel: [58576.878976] block drbd0: conn( WFSyncUUID -> SyncTarget )
Mar 22 14:02:37 jupiter kernel: [58576.878980] block drbd0: Began resync as SyncTarget (will sync 0
KB [0 bits set]).
Mar 22 14:02:38 jupiter kernel: [58576.987141] block drbd0: Resync done (total 1 sec; paused 0 sec;
0 K/sec)
Mar 22 14:02:38 jupiter kernel: [58576.987145] block drbd0: updated UUIDs
1818AD4FA40CF704:0000000000000000:AB884D96DE579454:AB874D96DE579455
Mar 22 14:02:38 jupiter kernel: [58576.987149] block drbd0: conn( SyncTarget -> Connected ) disk(
Inconsistent -> UpToDate )
Mar 22 14:02:38 jupiter kernel: [58577.996227] block drbd0: helper command: /sbin/drbdadm
after-resync-target minor-0
Mar 22 14:02:38 jupiter kernel: [58577.996674] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000030
Mar 22 14:02:38 jupiter kernel: [58577.996676] IP: [<ffffffff81468753>] sock_ioctl+0x13/0x2b0
Mar 22 14:02:38 jupiter kernel: [58577.996680] PGD 217235067 PUD 21dbe3067 PMD 0
Mar 22 14:02:38 jupiter kernel: [58577.996682] Oops: 0000 [#128] PREEMPT SMP
Mar 22 14:02:38 jupiter kernel: [58577.996684] CPU 7
Mar 22 14:02:38 jupiter kernel: [58577.996684] Modules linked in: af_packet drbd md5 sctp dlm
configfs sha1_generic lru_cache 8021q garp stp llc bonding mperf microcode joydev serio_raw sr_mod
cdrom pcspkr sg iTCO_wdt iTCO_vendor_support dcdbas i7core_edac edac_core acpi_power_meter pciehp
pci_hotplug bnx2 button autofs4 xfs processor thermal_sys ata_generic mpt2sas scsi_transport_sas
raid_class [last unloaded: ocfs2_stackglue]
Mar 22 14:02:38 jupiter kernel: [58577.996703]
Mar 22 14:02:38 jupiter kernel: [58577.996705] Pid: 18803, comm: drbdadm Tainted: G D
3.1.9-1.4-desktop #1 Dell Inc. PowerEdge R610/0F0XJ6
Mar 22 14:02:38 jupiter kernel: [58577.996709] RIP: 0010:[<ffffffff81468753>] [<ffffffff81468753>]
sock_ioctl+0x13/0x2b0
Mar 22 14:02:38 jupiter kernel: [58577.996714] RSP: 0018:ffff88021e7f9ec8 EFLAGS: 00010292
Mar 22 14:02:38 jupiter kernel: [58577.996716] RAX: 0000000000000000 RBX: 0000000000005401 RCX:
00007fff468a4a78
Mar 22 14:02:38 jupiter kernel: [58577.996718] RDX: 00007fff468a4a78 RSI: ffff88022ea3f580 RDI:
ffff88021e464800
Mar 22 14:02:38 jupiter kernel: [58577.996721] RBP: 0000000000000000 R08: ffffffff81468740 R09:
00007fb14307a1b0
Mar 22 14:02:38 jupiter kernel: [58577.996723] R10: 00007fff468a4890 R11: 0000000000000202 R12:
0000000000005401
Mar 22 14:02:38 jupiter kernel: [58577.996726] R13: 00007fff468a4a78 R14: 0000000000000000 R15:
0000000000000000
Mar 22 14:02:38 jupiter kernel: [58577.996729] FS: 00007fb143269700(0000) GS:ffff88022f2e0000(0000)
knlGS:0000000000000000
Mar 22 14:02:38 jupiter kernel: [58577.996732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 22 14:02:38 jupiter kernel: [58577.996734] CR2: 0000000000000030 CR3: 000000021ff54000 CR4:
00000000000006e0
Mar 22 14:02:38 jupiter kernel: [58577.996737] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
Mar 22 14:02:38 jupiter kernel: [58577.996739] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
Mar 22 14:02:38 jupiter kernel: [58577.996742] Process drbdadm (pid: 18803, threadinfo
ffff88021e7f8000, task ffff880217e00240)
Mar 22 14:02:38 jupiter kernel: [58577.996745] Stack:
Mar 22 14:02:38 jupiter kernel: [58577.996746] 00007fff468a4c38 0000000000000003 ffff880217e00240
ffff88021e464800
Mar 22 14:02:38 jupiter kernel: [58577.996750] 0000000000000000 ffffffff811653e5 00007fb14307a1b0
0000000000413d10
Mar 22 14:02:38 jupiter kernel: [58577.996755] ffff88021e7f8000 0000000000000030 ffff88021e464800
0000000000000000
Mar 22 14:02:38 jupiter kernel: [58577.996759] Call Trace:
Mar 22 14:02:38 jupiter kernel: [58577.996764] [<ffffffff811653e5>] do_vfs_ioctl+0x85/0x2e0
Mar 22 14:02:38 jupiter kernel: [58577.996768] [<ffffffff811656d8>] sys_ioctl+0x98/0xa0
Mar 22 14:02:38 jupiter kernel: [58577.996773] [<ffffffff815a4c12>] system_call_fastpath+0x16/0x1b
Mar 22 14:02:38 jupiter kernel: [58577.996777] [<00007fb142daef67>] 0x7fb142daef66
Mar 22 14:02:38 jupiter kernel: [58577.996779] Code: 24 18 4c 8b 6c 24 20 4c 8b 74 24 28 4c 8b 7c 24
30 48 83 c4 38 c3 66 90 55 53 89 f3 48 83 ec 18 48 8b b7 a0 00 00 00 48 8b 46 20
Mar 22 14:02:38 jupiter kernel: [58577.996805] RIP [<ffffffff81468753>] sock_ioctl+0x13/0x2b0
Mar 22 14:02:38 jupiter kernel: [58577.996808] RSP <ffff88021e7f9ec8>
Mar 22 14:02:38 jupiter kernel: [58577.996810] CR2: 0000000000000030
Mar 22 14:02:38 jupiter kernel: [58577.996873] ---[ end trace 527c00f5943cff75 ]---
Mar 22 14:02:38 jupiter kernel: [58577.997006] block drbd0: helper command: /sbin/drbdadm
after-resync-target minor-0 exit code 0 (0x9)
Mar 22 14:02:39 jupiter kernel: [58578.484500] block drbd0: bitmap WRITE of 7243 pages took 488
jiffies
Mar 22 14:02:39 jupiter kernel: [58578.526086] block drbd0: 0 KB (0 bits) marked out-of-sync by on
disk bit-map.
This is my DRBD related configuration on Pacemaker:
primitive resDLM ocf:pacemaker:controld \
op monitor interval="120s"
primitive resDRBD ocf:linbit:drbd \
params drbd_resource="export"
primitive resFS ocf:heartbeat:Filesystem \
params device="/dev/drbd/by-res/export" directory="/export" fstype="ocfs2"
options="rw,noatime" \
op monitor interval="120s"
primitive resO2CB ocf:ocfs2:o2cb \
op monitor interval="120s"
ms msDRBD resDRBD \
meta resource-stickines="100" notify="true" master-max="2" clone-max="2" interleave="true"
target-role="Started"
clone cloneDLM resDLM \
meta globally-unique="false" interleave="true" target-role="Stopped"
clone cloneFS resFS \
meta interleave="true" ordered="true" target-role="Stopped"
clone cloneO2CB resO2CB \
meta globally-unique="false" interleave="true" target-role="Stopped"
colocation colDLMDRBD inf: cloneDLM msDRBD:Master
colocation colFSO2CB inf: cloneFS cloneO2CB
colocation colO2CBDLM inf: cloneO2CB cloneDLM
colocation mysql-with-vip inf: mysql ip_mysql
colocation vip-with-mount inf: ip_mysql cloneFS
order mysql-after-vip 0: ip_mysql mysql
order ordDLMO2CB 0: cloneDLM cloneO2CB
order ordDRBDDLM 0: msDRBD:promote cloneDLM:start
order ordO2CBFS 0: cloneO2CB cloneFS
Any help will be welcome.