[DRBD-user] Could not add a new node

大川敬臣 yoshiomi-okawa at kccs.co.jp
Mon Jul 24 10:25:40 CEST 2017

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Please let me ask you questions about DRBD.
I'm testing DRBD 9.0.8 with RHEL 7.3 on AWS environment.

【Configurations】
OS:RHEL 7.3
Kernel version:3.10.0-514.el7.x86_64
DRBD:drbd-9.0.8-1
     drbd-utils-9.0.0
     drbdmanage-0.99.5
Host Name:drbd-01、drbd-02
Disk for DRBD: /dev/xvdb

・DRBD volume is going to be used  as MySQL data volume.
・AWS env is only test environment. PROD env is VM ESXi.

【Installing steps】
・Installing required packages
# yum -y install kernel-devel.x86_64
# yum -y install gcc.x86_64
# yum -y install flex.x86_64

・Installing DRBD
# cd /usr/local/src/
# tar zxf drbd-9.0.8-1.tar.gz
# cd drbd-9.0.8-1
# make KDIR=/usr/src/kernels/$(uname -r)
# make install
# modprobe drbd
# cat /proc/drbd

・Installing drbd-util
# cd /usr/local/src/
# tar zxf drbd-utils-9.0.0.tar.gz
# cd drbd-utils-9.0.0
# ./configure
# make
# make install

・Installing DRBD Manage
# cd /usr/local/src/
# tar zxf drbdmanage-0.99.5.tar.gz
# cd drbdmanage-0.99.5
# ./setup.py build
# ./setup.py install

【Initialize DRBD】
- Creating pool for DRBD
# vgcreate drbdpool /dev/xvdb1
  Physical volume "/dev/xvdb1" successfully created.
  Volume group "drbdpool" successfully created
#

- Execute on drbd-01
# drbdmanage init 172.31.1.155

You are going to initialize a new drbdmanage cluster.
CAUTION! Note that:
  * Any previous drbdmanage cluster information may be removed
  * Any remaining resources managed by a previous drbdmanage installation
    that still exist on this system will no longer be managed by drbdmanage

Confirm:

  yes/no: yes
Empty drbdmanage control volume initialized on '/dev/drbd0'.
Empty drbdmanage control volume initialized on '/dev/drbd1'.
Waiting for server: .
Operation completed successfully
#
# drbdadm status
.drbdctrl role:Primary
  volume:0 disk:UpToDate
  volume:1 disk:UpToDate

#
#
# lvdisplay -c
  /dev/drbdpool/.drbdctrl_0:drbdpool:3:1:-1:2:8192:1:-1:0:-1:253:0
  /dev/drbdpool/.drbdctrl_1:drbdpool:3:1:-1:2:8192:1:-1:0:-1:253:1
#
# drbdmanage list-nodes
+------------------------------------------------------------------------------------------------------------+
| Name    | Pool Size | Pool Free |
                         | State |
|------------------------------------------------------------------------------------------------------------|
| drbd-01 |     10236 |     10228 |
                         |    ok |
+------------------------------------------------------------------------------------------------------------+
#
# drbdmanage new-node drbd-02 172.31.8.103
Operation completed successfully
Operation completed successfully

Executing join command using ssh.
IMPORTANT: The output you see comes from drbd-02
IMPORTANT: Your input is executed on drbd-02
You are going to join an existing drbdmanage cluster.
CAUTION! Note that:
  * Any previous drbdmanage cluster information may be removed
  * Any remaining resources managed by a previous drbdmanage installation
    that still exist on this system will no longer be managed by drbdmanage

Confirm:

  yes/no: yes
Waiting for server to start up (can take up to 1 min)
Waiting for server: ......
Operation completed successfully
Give leader time to contact the new node
Operation completed successfully
Operation completed successfully
#
#
# drbdmanage howto-join drbd-02
IMPORTANT: Execute the following command only on node drbd-02!
drbdmanage join -p 6999 172.31.8.103 1 drbd-01 172.31.1.155 0
aez1qL969FHRDYJH4qYD
Operation completed successfully
#
#

- Execute on drbd-02
# drbdmanage join -p 6999 172.31.8.103 1 drbd-01 172.31.1.155 0
aez1qL969FHRDYJH4qYD
You are going to join an existing drbdmanage cluster.
CAUTION! Note that:
  * Any previous drbdmanage cluster information may be removed
  * Any remaining resources managed by a previous drbdmanage installation
    that still exist on this system will no longer be managed by drbdmanage

Confirm:

  yes/no: yes
Waiting for server to start up (can take up to 1 min)
Operation completed successfully
#
#

- Execute on drbd-01
# drbdmanage list-nodes
+------------------------------------------------------------------------------------------+
| Name    | Pool Size | Pool Free |                          |
          State |
|------------------------------------------------------------------------------------------|
| drbd-01 |     10236 |     10228 |                          |
 online/quorum vote ignored |
| drbd-02 |     10236 |     10228 |                          |
offline/quorum vote ignored |
+------------------------------------------------------------------------------------------+
[root at drbd-01 drbd.d]#

- Execute on drbd-02
# drbdmanage list-nodes
+------------------------------------------------------------------------------------------+
| Name    | Pool Size | Pool Free |                          |
          State |
|------------------------------------------------------------------------------------------|
| drbd-01 |     10236 |     10228 |                          |
offline/quorum vote ignored |
| drbd-02 |     10236 |     10228 |                          |
 online/quorum vote ignored |
+------------------------------------------------------------------------------------------+
#
#

When I checked syslog on drbd-01, the messages blow were writen.
----------
Jul 24 03:27:58 drbd-01 dbus-daemon: .drbdctrl role:Primary
Jul 24 03:27:58 drbd-01 dbus-daemon: volume:0 disk:UpToDate
Jul 24 03:27:58 drbd-01 dbus-daemon: volume:1 disk:UpToDate
Jul 24 03:29:55 drbd-01 drbdmanaged[2221]: INFO       DrbdAdm: Running
external command: drbdadm -vvv adjust .drbdctrl
Jul 24 03:29:55 drbd-01 drbdmanaged[2221]: ERROR      DrbdAdm: External
command 'drbdadm': Exit code 1
Jul 24 03:29:55 drbd-01 dbus-daemon: .drbdctrl role:Primary
----------

on drbd-02
----------
Jul 24 03:29:59 drbd-02 drbdmanaged[2184]: INFO       DrbdAdm: Running
external command: drbdadm -vvv adjust .drbdctrl
Jul 24 03:29:59 drbd-02 drbdmanaged[2184]: ERROR      DrbdAdm: External
command 'drbdadm': Exit code 1
Jul 24 03:29:59 drbd-02 drbdmanaged[2184]: INFO       DRBDManage starting
as potential leader node
Jul 24 03:29:59 drbd-02 dbus-daemon: .drbdctrl role:Secondary
Jul 24 03:29:59 drbd-02 dbus-daemon: volume:0 disk:Inconsistent
Jul 24 03:29:59 drbd-02 dbus-daemon: volume:1 disk:Inconsistent
Jul 24 03:29:59 drbd-02 dbus-daemon: drbd-01 connection:Connecting
Jul 24 03:30:41 drbd-02 drbdmanaged[2184]: ERROR      drbdsetup events
tracing has failed, restarting
Jul 24 03:30:41 drbd-02 drbdmanaged[2184]: INFO       drbdsetup events
tracing reestablished
Jul 24 03:31:21 drbd-02 drbdmanaged[2184]: INFO       Reading initial DRBD
control volume status
Jul 24 03:31:21 drbd-02 drbdmanaged[2184]: INFO       Finished reading
initial DRBD control volume status
Jul 24 03:31:21 drbd-02 drbdmanaged[2184]: INFO       Leader election by
wait for connections
Jul 24 03:31:21 drbd-02 drbdmanaged[2184]: INFO       DrbdAdm: Running
external command: drbdsetup wait-connect-resource --wait-after-sb=yes
--wfc-timeout=2 .drbdctrl
Jul 24 03:31:23 drbd-02 drbdmanaged[2184]: ERROR      DrbdAdm: External
command 'drbdsetup': Exit code 5
Jul 24 03:31:23 drbd-02 drbdmanaged[2184]: ERROR      drbdsetup/stderr:
degr-wfc-timeout has to be shorter than wfc-timeout
Jul 24 03:31:23 drbd-02 drbdmanaged[2184]: ERROR      drbdsetup/stderr:
degr-wfc-timeout implicitly set to wfc-timeout (2s)
Jul 24 03:31:23 drbd-02 drbdmanaged[2184]: ERROR      drbdsetup/stderr:
outdated-wfc-timeout has to be shorter than degr-wfc-timeout
Jul 24 03:31:23 drbd-02 drbdmanaged[2184]: ERROR      drbdsetup/stderr:
outdated-wfc-timeout implicitly set to degr-wfc-timeout (2s)
Jul 24 03:31:23 drbd-02 drbdmanaged[2184]: WARNING    Resource '.drbdctrl':
wait-connect-resource not finished within 2 seconds
----------

Please let me know what these errors mean and how I can resolve this issue.

Your help would be appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20170724/e05aeb13/attachment.htm>


More information about the drbd-user mailing list