Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello, I have tried to make a dual primary setup, but can't get it work. I'm using two Celisus 550 Workstations with two sata drives on each machine building a softraid 0. There are dedicated 1GBit network cards for use with drbd directly connected with a cross connect calbe. Creating a drbd device on top of /dev/md3 (on top of /dev/sda3 /dev/sdb3) works without problems. The initial sync reports no problems. It is possible to make one machine primary and the other secondary and vice versa. Everything works so far. But when trying to get both machines primary the system is respondig very slow or is not respondig anymore. In /var/log/messages I found a message which looks like a timing problem. "PingAck did not arrive in time" Then I tried drbd version 8.4.1 and all problems are gone. Then I tried 8.4.2rc3. Even no problems. Dual primary is working as expected. Then 8.4.2 again. Only single primay works. Then I reverted to 8.4.2rc3 and everything is fine again. So I gess that there is a bug created with 8.4.2 final :( Here are some logs starting after reboot and console output of things I have done until the machine stops. Hope someone can find why 8.4.2 stops working ... Henning content of /etc/drbd/drbd-sr1.res: resource drbd-sr1 { volume 0 { device /dev/drbd0; disk /dev/md3; meta-disk internal; } on XS-211 { address 10.1.1.211:7788; } on XS-212 { address 10.1.1.212:7788; } } contend of /etc/drbd/global_common.conf global { usage-count yes; } common { syncer { rate 33M; } handlers { split-brain "/usr/lib/drbd/notify-split-brain.sh root"; } startup { become-primary-on both; } disk { #disk-barrier no; #USV #disk-flushes no; #USV } protocol C; net { sndbuf-size 0; allow-two-primaries yes; ping-timeout 100; shared-secret "85ht9hHruiH"; after-sb-0pri discard-zero-changes; after-sb-1pri consensus; after-sb-2pri disconnect; } } [root at XS-211 ~]# cat /var/log/messages | grep drbd Nov 11 17:03:00 XS-211 kernel: [ 32.209940] events: mcg drbd: 5 Nov 11 17:03:00 XS-211 kernel: [ 32.212162] drbd: initialized. Version: 8.4.2 (api:1/proto:86-101) Nov 11 17:03:00 XS-211 kernel: [ 32.212165] drbd: GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root at localhost.localdomain, 2012-10-29 13:22:58 Nov 11 17:03:00 XS-211 kernel: [ 32.212167] drbd: registered as block device major 147 Nov 11 17:03:00 XS-211 kernel: [ 32.336528] d-con drbd-sr1: Starting worker thread (from drbdsetup [6984]) Nov 11 17:03:00 XS-211 kernel: [ 32.336676] block drbd0: disk( Diskless -> Attaching ) Nov 11 17:03:00 XS-211 kernel: [ 32.337015] d-con drbd-sr1: Method to ensure write ordering: flush Nov 11 17:03:00 XS-211 kernel: [ 32.337019] block drbd0: max BIO size = 65536 Nov 11 17:03:00 XS-211 kernel: [ 32.337025] block drbd0: drbd_bm_resize called with capacity == 1677670072 Nov 11 17:03:00 XS-211 kernel: [ 32.347804] block drbd0: resync bitmap: bits=209708759 words=6553400 pages=6400 Nov 11 17:03:00 XS-211 kernel: [ 32.347819] block drbd0: size = 800 GB (838835036 KB) Nov 11 17:03:01 XS-211 kernel: [ 32.466241] block drbd0: bitmap READ of 6400 pages took 12 jiffies Nov 11 17:03:01 XS-211 kernel: [ 32.502238] block drbd0: recounting of set bits took additional 3 jiffies Nov 11 17:03:01 XS-211 kernel: [ 32.502243] block drbd0: 0 KB (0 bits) marked out-of-sync by on disk bit-map. Nov 11 17:03:01 XS-211 kernel: [ 32.502253] block drbd0: disk( Attaching -> UpToDate ) Nov 11 17:03:01 XS-211 kernel: [ 32.502257] block drbd0: attached to UUIDs FAED0840B50DDA5E:0000000000000000:E91787DC09996421:E91687DC09996421 Nov 11 17:03:01 XS-211 kernel: [ 32.502287] block drbd0: Barriers not supported on meta data device - disabling Nov 11 17:03:01 XS-211 kernel: [ 32.504162] d-con drbd-sr1: conn( StandAlone -> Unconnected ) Nov 11 17:03:01 XS-211 kernel: [ 32.504181] d-con drbd-sr1: Starting receiver thread (from drbd_w_drbd-sr1 [6985]) Nov 11 17:03:01 XS-211 kernel: [ 32.504266] d-con drbd-sr1: receiver (re)started Nov 11 17:03:01 XS-211 kernel: [ 32.504284] d-con drbd-sr1: conn( Unconnected -> WFConnection ) Nov 11 17:03:01 XS-211 kernel: [ 32.514238] block drbd0: role( Secondary -> Primary ) Nov 11 17:03:01 XS-211 kernel: [ 32.514573] block drbd0: new current UUID 0B230DF7BDF229AF:FAED0840B50DDA5E:E91787DC09996421:E91687DC09996421 Nov 11 17:06:39 XS-211 kernel: [ 251.196646] d-con drbd-sr1: Handshake successful: Agreed network protocol version 101 Nov 11 17:06:39 XS-211 kernel: [ 251.196691] d-con drbd-sr1: conn( WFConnection -> WFReportParams ) Nov 11 17:06:39 XS-211 kernel: [ 251.196695] d-con drbd-sr1: Starting asender thread (from drbd_r_drbd-sr1 [6993]) Nov 11 17:06:39 XS-211 kernel: [ 251.284292] d-con drbd-sr1: meta connection shut down by peer. Nov 11 17:06:39 XS-211 kernel: [ 251.284317] d-con drbd-sr1: conn( WFReportParams -> NetworkFailure ) Nov 11 17:06:39 XS-211 kernel: [ 251.284320] d-con drbd-sr1: asender terminated Nov 11 17:06:39 XS-211 kernel: [ 251.284323] d-con drbd-sr1: Terminating asender thread Nov 11 17:06:39 XS-211 kernel: [ 251.286548] d-con drbd-sr1: Connection closed Nov 11 17:06:39 XS-211 kernel: [ 251.286633] d-con drbd-sr1: conn( NetworkFailure -> Unconnected ) Nov 11 17:06:39 XS-211 kernel: [ 251.286636] d-con drbd-sr1: receiver terminated Nov 11 17:06:39 XS-211 kernel: [ 251.286638] d-con drbd-sr1: Restarting receiver thread Nov 11 17:06:39 XS-211 kernel: [ 251.286640] d-con drbd-sr1: receiver (re)started Nov 11 17:06:39 XS-211 kernel: [ 251.286647] d-con drbd-sr1: conn( Unconnected -> WFConnection ) [root at XS-212 ~]# cat /var/log/messages | grep drbd Nov 11 17:06:38 XS-212 kernel: [ 32.463248] events: mcg drbd: 5 Nov 11 17:06:38 XS-212 kernel: [ 32.465610] drbd: initialized. Version: 8.4.2 (api:1/proto:86-101) Nov 11 17:06:38 XS-212 kernel: [ 32.465613] drbd: GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root at localhost.localdomain, 2012-10-29 13:22:58 Nov 11 17:06:38 XS-212 kernel: [ 32.465615] drbd: registered as block device major 147 Nov 11 17:06:39 XS-212 kernel: [ 32.576723] d-con drbd-sr1: Starting worker thread (from drbdsetup [7000]) Nov 11 17:06:39 XS-212 kernel: [ 32.576844] block drbd0: disk( Diskless -> Attaching ) Nov 11 17:06:39 XS-212 kernel: [ 32.577189] d-con drbd-sr1: Method to ensure write ordering: flush Nov 11 17:06:39 XS-212 kernel: [ 32.577192] block drbd0: max BIO size = 65536 Nov 11 17:06:39 XS-212 kernel: [ 32.577198] block drbd0: drbd_bm_resize called with capacity == 1677670072 Nov 11 17:06:39 XS-212 kernel: [ 32.586947] block drbd0: resync bitmap: bits=209708759 words=6553400 pages=6400 Nov 11 17:06:39 XS-212 kernel: [ 32.586965] block drbd0: size = 800 GB (838835036 KB) Nov 11 17:06:39 XS-212 kernel: [ 32.707802] block drbd0: bitmap READ of 6400 pages took 12 jiffies Nov 11 17:06:39 XS-212 kernel: [ 32.747658] block drbd0: recounting of set bits took additional 4 jiffies Nov 11 17:06:39 XS-212 kernel: [ 32.747663] block drbd0: 0 KB (0 bits) marked out-of-sync by on disk bit-map. Nov 11 17:06:39 XS-212 kernel: [ 32.747674] block drbd0: disk( Attaching -> UpToDate ) Nov 11 17:06:39 XS-212 kernel: [ 32.747677] block drbd0: attached to UUIDs FAED0840B50DDA5E:0000000000000000:E91787DC09996420:E91687DC09996421 Nov 11 17:06:39 XS-212 kernel: [ 32.747710] block drbd0: Barriers not supported on meta data device - disabling Nov 11 17:06:39 XS-212 kernel: [ 32.749834] d-con drbd-sr1: conn( StandAlone -> Unconnected ) Nov 11 17:06:39 XS-212 kernel: [ 32.749857] d-con drbd-sr1: Starting receiver thread (from drbd_w_drbd-sr1 [7001]) Nov 11 17:06:39 XS-212 kernel: [ 32.749945] d-con drbd-sr1: receiver (re)started Nov 11 17:06:39 XS-212 kernel: [ 32.749966] d-con drbd-sr1: conn( Unconnected -> WFConnection ) Nov 11 17:06:39 XS-212 kernel: [ 32.760219] block drbd0: role( Secondary -> Primary ) Nov 11 17:06:39 XS-212 kernel: [ 32.760626] block drbd0: new current UUID EE531CBF563726C3:FAED0840B50DDA5E:E91787DC09996420:E91687DC09996421 Nov 11 17:06:39 XS-212 kernel: [ 33.246194] d-con drbd-sr1: Handshake successful: Agreed network protocol version 101 Nov 11 17:06:39 XS-212 kernel: [ 33.246272] d-con drbd-sr1: conn( WFConnection -> WFReportParams ) Nov 11 17:06:39 XS-212 kernel: [ 33.246276] d-con drbd-sr1: Starting asender thread (from drbd_r_drbd-sr1 [7006]) Nov 11 17:06:39 XS-212 kernel: [ 33.313875] block drbd0: drbd_sync_handshake: Nov 11 17:06:39 XS-212 kernel: [ 33.313890] block drbd0: self EE531CBF563726C3:FAED0840B50DDA5E:E91787DC09996420:E91687DC09996421 bits:0 flags:0 Nov 11 17:06:39 XS-212 kernel: [ 33.313895] block drbd0: peer 0B230DF7BDF229AF:FAED0840B50DDA5E:E91787DC09996421:E91687DC09996421 bits:0 flags:0 Nov 11 17:06:39 XS-212 kernel: [ 33.313900] block drbd0: uuid_compare()=100 by rule 90 Nov 11 17:06:39 XS-212 kernel: [ 33.313905] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 Nov 11 17:06:39 XS-212 kernel: [ 33.315465] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 exit code 0 (0x0) Nov 11 17:06:39 XS-212 kernel: [ 33.315477] block drbd0: Split-Brain detected but unresolved, dropping connection! Nov 11 17:06:39 XS-212 kernel: [ 33.315484] block drbd0: helper command: /sbin/drbdadm split-brain minor-0 Nov 11 17:06:39 XS-212 notify-split-brain.sh[7098]: invoked for drbd-sr1/0 (drbd0) Nov 11 17:06:39 XS-212 notify-split-brain.sh[7098]: /usr/lib/drbd/notify- split-brain.sh: line 110: mail: command not found Nov 11 17:06:39 XS-212 kernel: [ 33.333392] block drbd0: helper command: /sbin/drbdadm split-brain minor-0 exit code 127 (0x7f00) Nov 11 17:06:39 XS-212 kernel: [ 33.333423] d-con drbd-sr1: conn( WFReportParams -> Disconnecting ) Nov 11 17:06:39 XS-212 kernel: [ 33.333428] d-con drbd-sr1: error receiving ReportState, e: -5 l: 0! Nov 11 17:06:39 XS-212 kernel: [ 33.333442] d-con drbd-sr1: asender terminated Nov 11 17:06:39 XS-212 kernel: [ 33.333446] d-con drbd-sr1: Terminating asender thread Nov 11 17:06:39 XS-212 kernel: [ 33.333576] d-con drbd-sr1: Connection closed Nov 11 17:06:39 XS-212 kernel: [ 33.333641] d-con drbd-sr1: conn( Disconnecting -> StandAlone ) Nov 11 17:06:39 XS-212 kernel: [ 33.333644] d-con drbd-sr1: receiver terminated Nov 11 17:06:39 XS-212 kernel: [ 33.333647] d-con drbd-sr1: Terminating receiver thread [root at XS-211 ~]# drbd-overview 0:drbd-sr1/0 WFConnection Primary/Unknown UpToDate/DUnknown C r----- [root at XS-212 ~]# drbd-overview 0:drbd-sr1/0 StandAlone Primary/Unknown UpToDate/DUnknown r----- [root at XS-212 ~]# drbdadm secondary drbd-sr1 [root at XS-212 ~]# drbd-overview 0:drbd-sr1/0 StandAlone Secondary/Unknown UpToDate/DUnknown r----- [root at XS-212 ~]# drbdadm -- --discard-my-data connect drbd-sr1 drbd-sr1: Failure: (102) Local address(port) already in use. Command 'drbdsetup connect drbd-sr1 ipv4:10.1.1.212:7788 ipv4:10.1.1.211:7788 --after-sb-2pri=disconnect --after-sb-1pri=consensus --after-sb-0pri=discard- zero-changes --shared-secret=85ht9hHruiH --allow-two-primaries=yes --sndbuf- size=0 --protocol=C --discard-my-data' terminated with exit code 10 [root at XS-212 ~]# drbd-overview 0:drbd-sr1/0 Connected Secondary/Primary UpToDate/UpToDate C r----- [root at XS-211 ~]# cat /var/log/messages | grep drbd Nov 11 17:17:19 XS-211 kernel: [ 891.313566] d-con drbd-sr1: Handshake successful: Agreed network protocol version 101 Nov 11 17:17:19 XS-211 kernel: [ 891.313612] d-con drbd-sr1: conn( WFConnection -> WFReportParams ) Nov 11 17:17:19 XS-211 kernel: [ 891.313616] d-con drbd-sr1: Starting asender thread (from drbd_r_drbd-sr1 [6993]) Nov 11 17:17:20 XS-211 kernel: [ 891.483560] block drbd0: drbd_sync_handshake: Nov 11 17:17:20 XS-211 kernel: [ 891.483565] block drbd0: self 0B230DF7BDF229AF:FAED0840B50DDA5E:E91787DC09996421:E91687DC09996421 bits:0 flags:0 Nov 11 17:17:20 XS-211 kernel: [ 891.483570] block drbd0: peer EE531CBF563726C2:FAED0840B50DDA5E:E91787DC09996420:E91687DC09996421 bits:0 flags:1 Nov 11 17:17:20 XS-211 kernel: [ 891.483575] block drbd0: uuid_compare()=100 by rule 90 Nov 11 17:17:20 XS-211 kernel: [ 891.483591] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 Nov 11 17:17:20 XS-211 kernel: [ 891.485400] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 exit code 0 (0x0) Nov 11 17:17:20 XS-211 kernel: [ 891.485413] block drbd0: Split-Brain detected, 1 primaries, automatically solved. Sync from this node Nov 11 17:17:20 XS-211 kernel: [ 891.485419] block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> Consistent ) Nov 11 17:17:20 XS-211 kernel: [ 891.496537] block drbd0: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% Nov 11 17:17:20 XS-211 kernel: [ 891.657337] block drbd0: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% Nov 11 17:17:20 XS-211 kernel: [ 891.657344] block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0 Nov 11 17:17:20 XS-211 kernel: [ 891.659185] block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0 exit code 0 (0x0) Nov 11 17:17:20 XS-211 kernel: [ 891.659200] block drbd0: conn( WFBitMapS -> SyncSource ) pdsk( Consistent -> Inconsistent ) Nov 11 17:17:20 XS-211 kernel: [ 891.659208] block drbd0: Began resync as SyncSource (will sync 0 KB [0 bits set]). Nov 11 17:17:20 XS-211 kernel: [ 891.659269] block drbd0: updated sync UUID 0B230DF7BDF229AF:FAEE0840B50DDA5E:FAED0840B50DDA5E:E91787DC09996421 Nov 11 17:17:20 XS-211 kernel: [ 891.676704] block drbd0: Resync done (total 1 sec; paused 0 sec; 0 K/sec) Nov 11 17:17:20 XS-211 kernel: [ 891.676709] block drbd0: updated UUIDs 0B230DF7BDF229AF:0000000000000000:FAEE0840B50DDA5E:FAED0840B50DDA5E Nov 11 17:17:20 XS-211 kernel: [ 891.676714] block drbd0: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate ) [root at XS-212 ~]# cat /var/log/messages | grep drbd Nov 11 17:17:03 XS-212 kernel: [ 656.815632] block drbd0: role( Primary -> Secondary ) Nov 11 17:17:03 XS-212 kernel: [ 656.815744] block drbd0: bitmap WRITE of 0 pages took 0 jiffies Nov 11 17:17:03 XS-212 kernel: [ 656.815747] block drbd0: 0 KB (0 bits) marked out-of-sync by on disk bit-map. Nov 11 17:17:19 XS-212 kernel: [ 672.875771] d-con drbd-sr1: conn( StandAlone -> Unconnected ) Nov 11 17:17:19 XS-212 kernel: [ 672.875799] d-con drbd-sr1: Starting receiver thread (from drbd_w_drbd-sr1 [7001]) Nov 11 17:17:19 XS-212 kernel: [ 672.875871] d-con drbd-sr1: receiver (re)started Nov 11 17:17:19 XS-212 kernel: [ 672.875895] d-con drbd-sr1: conn( Unconnected -> WFConnection ) Nov 11 17:17:19 XS-212 kernel: [ 673.374075] d-con drbd-sr1: Handshake successful: Agreed network protocol version 101 Nov 11 17:17:19 XS-212 kernel: [ 673.374130] d-con drbd-sr1: conn( WFConnection -> WFReportParams ) Nov 11 17:17:19 XS-212 kernel: [ 673.374134] d-con drbd-sr1: Starting asender thread (from drbd_r_drbd-sr1 [9645]) Nov 11 17:17:20 XS-212 kernel: [ 673.704473] block drbd0: drbd_sync_handshake: Nov 11 17:17:20 XS-212 kernel: [ 673.704478] block drbd0: self EE531CBF563726C2:FAED0840B50DDA5E:E91787DC09996420:E91687DC09996421 bits:0 flags:0 Nov 11 17:17:20 XS-212 kernel: [ 673.704482] block drbd0: peer 0B230DF7BDF229AF:FAED0840B50DDA5E:E91787DC09996421:E91687DC09996421 bits:0 flags:0 Nov 11 17:17:20 XS-212 kernel: [ 673.704487] block drbd0: uuid_compare()=100 by rule 90 Nov 11 17:17:20 XS-212 kernel: [ 673.704493] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 Nov 11 17:17:20 XS-212 kernel: [ 673.706113] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 exit code 0 (0x0) Nov 11 17:17:20 XS-212 kernel: [ 673.706125] block drbd0: Split-Brain detected, 1 primaries, automatically solved. Sync from peer node Nov 11 17:17:20 XS-212 kernel: [ 673.706132] block drbd0: peer( Unknown -> Primary ) conn( WFReportParams -> WFBitMapT ) disk( UpToDate -> Outdated ) pdsk( DUnknown -> UpToDate ) Nov 11 17:17:20 XS-212 kernel: [ 673.706431] block drbd0: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% Nov 11 17:17:20 XS-212 kernel: [ 673.717558] block drbd0: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% Nov 11 17:17:20 XS-212 kernel: [ 673.717570] block drbd0: conn( WFBitMapT -> WFSyncUUID ) Nov 11 17:17:20 XS-212 kernel: [ 673.720121] block drbd0: updated sync uuid FAEE0840B50DDA5E:0000000000000000:E91787DC09996420:E91687DC09996421 Nov 11 17:17:20 XS-212 kernel: [ 673.720408] block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 Nov 11 17:17:20 XS-212 kernel: [ 673.721889] block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 exit code 0 (0x0) Nov 11 17:17:20 XS-212 kernel: [ 673.721903] block drbd0: conn( WFSyncUUID -> SyncTarget ) disk( Outdated -> Inconsistent ) Nov 11 17:17:20 XS-212 kernel: [ 673.721915] block drbd0: Began resync as SyncTarget (will sync 0 KB [0 bits set]). Nov 11 17:17:20 XS-212 kernel: [ 673.734978] block drbd0: Resync done (total 1 sec; paused 0 sec; 0 K/sec) Nov 11 17:17:20 XS-212 kernel: [ 673.734986] block drbd0: updated UUIDs 0B230DF7BDF229AE:0000000000000000:FAEE0840B50DDA5E:FAED0840B50DDA5E Nov 11 17:17:20 XS-212 kernel: [ 673.734994] block drbd0: conn( SyncTarget -> Connected ) disk( Inconsistent -> UpToDate ) Nov 11 17:17:20 XS-212 kernel: [ 673.735264] block drbd0: helper command: /sbin/drbdadm after-resync-target minor-0 Nov 11 17:17:20 XS-212 kernel: [ 673.736639] block drbd0: helper command: /sbin/drbdadm after-resync-target minor-0 exit code 0 (0x0) [root at XS-212 ~]# drbdadm primary drbd-sr1 [root at XS-212 ~]# drbd-overview 0:drbd-sr1/0 StandAlone Primary/Unknown UpToDate/DUnknown r----- [root at XS-212 ~]# cat /var/log/messages | grep drbd Nov 11 17:26:00 XS-212 kernel: [ 1193.804220] d-con drbd-sr1: PingAck did not arrive in time. Nov 11 17:26:00 XS-212 kernel: [ 1193.804254] d-con drbd-sr1: peer( Primary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown ) Nov 11 17:26:00 XS-212 kernel: [ 1193.804333] block drbd0: role( Secondary -> Primary ) Nov 11 17:26:00 XS-212 kernel: [ 1193.804804] d-con drbd-sr1: asender terminated Nov 11 17:26:00 XS-212 kernel: [ 1193.804808] d-con drbd-sr1: Terminating asender thread Nov 11 17:26:00 XS-212 kernel: [ 1193.805411] d-con drbd-sr1: Connection closed Nov 11 17:26:00 XS-212 kernel: [ 1193.805443] block drbd0: new current UUID 7A82AC13D6E2A441:0B230DF7BDF229AE:FAEE0840B50DDA5E:FAED0840B50DDA5E Nov 11 17:26:00 XS-212 kernel: [ 1193.805489] d-con drbd-sr1: conn( NetworkFailure -> Unconnected ) Nov 11 17:26:00 XS-212 kernel: [ 1193.805493] d-con drbd-sr1: receiver terminated Nov 11 17:26:00 XS-212 kernel: [ 1193.805496] d-con drbd-sr1: Restarting receiver thread Nov 11 17:26:00 XS-212 kernel: [ 1193.805498] d-con drbd-sr1: receiver (re)started Nov 11 17:26:00 XS-212 kernel: [ 1193.805504] d-con drbd-sr1: conn( Unconnected -> WFConnection ) Nov 11 17:26:00 XS-212 kernel: [ 1194.304287] d-con drbd-sr1: Handshake successful: Agreed network protocol version 101 Nov 11 17:26:00 XS-212 kernel: [ 1194.304341] d-con drbd-sr1: conn( WFConnection -> WFReportParams ) Nov 11 17:26:00 XS-212 kernel: [ 1194.304345] d-con drbd-sr1: Starting asender thread (from drbd_r_drbd-sr1 [9645]) Nov 11 17:26:01 XS-212 notify-split-brain.sh[9990]: invoked for drbd-sr1/0 (drbd0) Nov 11 17:26:01 XS-212 kernel: [ 1194.484292] block drbd0: drbd_sync_handshake: Nov 11 17:26:01 XS-212 kernel: [ 1194.484297] block drbd0: self 7A82AC13D6E2A441:0B230DF7BDF229AE:FAEE0840B50DDA5E:FAED0840B50DDA5E bits:0 flags:0 Nov 11 17:26:01 XS-212 kernel: [ 1194.484302] block drbd0: peer 3718448BC15DB8CD:0B230DF7BDF229AF:FAEE0840B50DDA5E:FAED0840B50DDA5E bits:0 flags:0 Nov 11 17:26:01 XS-212 kernel: [ 1194.484307] block drbd0: uuid_compare()=100 by rule 90 Nov 11 17:26:01 XS-212 kernel: [ 1194.484312] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 Nov 11 17:26:01 XS-212 kernel: [ 1194.485791] block drbd0: helper command: /sbin/drbdadm initial-split-brain minor-0 exit code 0 (0x0) Nov 11 17:26:01 XS-212 kernel: [ 1194.485802] block drbd0: Split-Brain detected but unresolved, dropping connection! Nov 11 17:26:01 XS-212 kernel: [ 1194.485811] block drbd0: helper command: /sbin/drbdadm split-brain minor-0 Nov 11 17:26:01 XS-212 notify-split-brain.sh[9990]: /usr/lib/drbd/notify- split-brain.sh: line 110: mail: command not found Nov 11 17:26:01 XS-212 kernel: [ 1194.496559] block drbd0: helper command: /sbin/drbdadm split-brain minor-0 exit code 127 (0x7f00) Nov 11 17:26:01 XS-212 kernel: [ 1194.496585] d-con drbd-sr1: conn( WFReportParams -> Disconnecting ) Nov 11 17:26:01 XS-212 kernel: [ 1194.496589] d-con drbd-sr1: error receiving ReportState, e: -5 l: 0! Nov 11 17:26:01 XS-212 kernel: [ 1194.496611] d-con drbd-sr1: asender terminated Nov 11 17:26:01 XS-212 kernel: [ 1194.496615] d-con drbd-sr1: Terminating asender thread Nov 11 17:26:01 XS-212 kernel: [ 1194.496742] d-con drbd-sr1: Connection closed Nov 11 17:26:01 XS-212 kernel: [ 1194.496820] d-con drbd-sr1: conn( Disconnecting -> StandAlone ) Nov 11 17:26:01 XS-212 kernel: [ 1194.496823] d-con drbd-sr1: receiver terminated Nov 11 17:26:01 XS-212 kernel: [ 1194.496825] d-con drbd-sr1: Terminating receiver thread