[DRBD-user] Is it a Split Brain situation or something else?
Paul O'Rorke
paul at tracker-software.com
Tue Apr 10 22:16:56 CEST 2018
Hi list,
I have a number of stacked resources replicating from 2 hosts to a
common third host. There are three resources 'stuck' waiting for a
connection with a state "Inconsistent". I have tried discarding the
data on this third node and reconnecting with *drbdadm connect
--discard-my-data* but I'm not having any luck.
I even (lv)removed the backing device, (lv)created a new volume, said
yes to remove the existing drbd meta-data:
root at trk-kvm-03:/etc/drbd.d# lvcreate -L 300G -n winserv-data trk-kvm-03-vg
WARNING: drbd signature detected on /dev/trk-kvm-03-vg/winserv-data at
offset 322122543164. Wipe it? [y/n]: yes
Wiping drbd signature on /dev/trk-kvm-03-vg/winserv-data.
Logical volume "winserv-data" created.
and ran drbdadm create-md to create new meta-data.
root at trk-kvm-03:/etc/drbd.d# drbdadm create-md fileserver-data-U
md_offset 322122543104
al_offset 322122510336
bm_offset 322112679936
Found some data
==> This might destroy existing data! <==
Do you want to proceed?
[need to type 'yes' to confirm] yes
initializing activity log
NOT initializing bitmap
Writing meta data...
New drbd meta data block successfully created.
success
I have 12 resources being replicated to this host and I am seeing this
on three of them.
root at trk-kvm-03:~# drbdadm connect --discard-my-data fileserver-data-U
root at trk-kvm-03:/etc/drbd.d# drbd-overview
20:fileserver-data-U/0 WFConnection Secondary/Unknown
Inconsistent/DUnknown
30:svn-U/0 Connected Secondary/Primary
UpToDate/UpToDate
40:MySQL-Slave-R-U/0 Connected Secondary/Primary
UpToDate/UpToDate
60:rt44-U/0 Connected Secondary/Primary
UpToDate/UpToDate
70:sharepoint_server-U/0 Connected Secondary/Primary
UpToDate/UpToDate
80:techwiki-U/0 Connected Secondary/Primary
UpToDate/UpToDate
90:sdkHelp-U/0 WFConnection Secondary/Unknown
UpToDate/DUnknown
100:sql-srv-2014-U/0 Connected Secondary/Primary UpToDate/UpToDate
101:fileserver-U/0 Connected Secondary/Primary UpToDate/UpToDate
110:informer-U/0 Connected Secondary/Primary UpToDate/UpToDate
120:convirt-U/0 Connected Secondary/Primary UpToDate/UpToDate
130:user_uploads-U/0 WFConnection Secondary/Unknown UpToDate/DUnknown
This third node, connected using protocol A, is intended for disaster
recovery only so none of these resources should ever have been the
primary so I am leaning away from a split-brain however the symptoms
appear to me to be like a split brain. I'm not seeing anything related
to split-brain in the logs.
The resources are UpToDate on the primaries:
root at trk-kvm-01:/etc/drbd.d# drbd-overview
1:fileserver/0 Connected Primary/Secondary
UpToDate/UpToDate *fileserver vda virtio
2:fileserver-data/0 Connected Primary/Secondary
UpToDate/UpToDate *fileserver vdb virtio
3:svn/0 Connected Secondary/Primary
UpToDate/UpToDate _svn vda virtio
4:MySQL-Slave-R/0 Connected Secondary/Primary
UpToDate/UpToDate _MySQL-Slave-R vda virtio
5:prtg-srv/0 Connected Secondary/Primary
UpToDate/UpToDate _prtg-srv vdb virtio
6:rt44/0 Connected Primary/Secondary
UpToDate/UpToDate *RT44 vda virtio
7:sharepoint_server/0 Connected Primary/Secondary
UpToDate/UpToDate *sharepoint_server hda ide
8:techwiki/0 Connected Secondary/Primary
UpToDate/UpToDate _techwiki vda virtio
9:sdkHelp/0 Connected Secondary/Primary
UpToDate/UpToDate _sdkHelp vda virtio
10:sql-srv-2014/0 Connected Primary/Secondary
UpToDate/UpToDate *sql-srv-2014 vda virtio
11:informer/0 Connected Secondary/Primary
UpToDate/UpToDate _informer vda virtio
12:convirt/0 Connected Primary/Secondary
UpToDate/UpToDate _convirt vda virtio
13:user_uploads/0 Connected Secondary/Primary
UpToDate/UpToDate _user_uploads vda virtio
20:fileserver-data-U/0 ^^2 WFConnection Primary/Unknown
UpToDate/DUnknown
60:rt44-U/0 ^^6 Connected Primary/Secondary
UpToDate/UpToDate
70:sharepoint_server-U/0 ^^7 Connected Primary/Secondary
UpToDate/UpToDate
100:sql-srv-2014-U/0 ^^10 Connected Primary/Secondary
UpToDate/UpToDate
101:fileserver-U/0 ^^1 Connected Primary/Secondary
UpToDate/UpToDate
120:convirt-U/0 ^^12 Connected Primary/Secondary
UpToDate/UpToDate
root at trk-kvm-02:~/qemu# drbd-overview
1:fileserver/0 Connected Secondary/Primary
UpToDate/UpToDate _fileserver vda virtio
2:fileserver-data/0 Connected Secondary/Primary
UpToDate/UpToDate _fileserver vdb virtio
3:svn/0 Connected Primary/Secondary
UpToDate/UpToDate *svn vda virtio
4:MySQL-Slave-R/0 Connected Primary/Secondary
UpToDate/UpToDate *MySQL-Slave-R vda virtio
5:prtg-srv/0 Connected Primary/Secondary
UpToDate/UpToDate
6:rt44/0 Connected Secondary/Primary
UpToDate/UpToDate _RT44 vda virtio
7:sharepoint_server/0 Connected Secondary/Primary
UpToDate/UpToDate _sharepoint_server hda ide
8:techwiki/0 Connected Primary/Secondary
UpToDate/UpToDate *techwiki vda virtio
9:sdkHelp/0 Connected Primary/Secondary
UpToDate/UpToDate *sdkHelp vda virtio
10:sql-srv-2014/0 Connected Secondary/Primary
UpToDate/UpToDate _sql-srv-2014 vda virtio
11:informer/0 Connected Primary/Secondary
UpToDate/UpToDate *informer vda virtio
12:convirt/0 Connected Secondary/Primary
UpToDate/UpToDate _convirt vda virtio
13:user_uploads/0 Connected Primary/Secondary
UpToDate/UpToDate *user_uploads vda virtio
30:svn-U/0 ^^3 Connected Primary/Secondary
UpToDate/UpToDate
40:MySQL-Slave-R-U/0 ^^4 Connected Primary/Secondary
UpToDate/UpToDate
80:techwiki-U/0 ^^8 Connected Primary/Secondary
UpToDate/UpToDate
90:sdkHelp-U/0 ^^9 WFConnection Primary/Unknown
UpToDate/DUnknown
110:informer-U/0 ^^11 Connected Primary/Secondary
UpToDate/UpToDate
130:user_uploads-U/0 ^^13 WFConnection Primary/Unknown UpToDate/DUnknown
I am keen to get these three resources replicating. Any suggestions
what I'm missing?
regards
--
*Paul O'Rorke*
*Tracker Software Products (Canada) Limited *
www.tracker-software.com <http://www.tracker-software.com/>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623
<http://www.tracker-software.com/>
Support:
http://www.tracker-software.com/support
Download latest Releases
http://www.tracker-software.com/downloads/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20180410/3e605575/attachment.htm>
More information about the drbd-user
mailing list