[DRBD-user] Could not add a new node

大川敬臣 yoshiomi-okawa at kccs.co.jp
Thu Jul 27 02:21:29 CEST 2017

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hi, Roland

I'm sorry to late reply and thank you for your advice.
I didn't understand how DRBD starts.
As you mentioned, res file wasn't read.
I included the path of res file in drbd.conf and two machines have
connected successfully.
Even if I still have a problem with this environment but it is related to
AWS environment so I will ask to AWS support.

Thank you for your help.
I appreciate it.


2017-07-25 15:50 GMT+09:00 Roland Kammerer <roland.kammerer at linbit.com>:

> On Mon, Jul 24, 2017 at 05:25:40PM +0900, 大川敬臣 wrote:
> > Please let me ask you questions about DRBD.
> > I'm testing DRBD 9.0.8 with RHEL 7.3 on AWS environment.
> >
> > 【Configurations】
> > OS:RHEL 7.3
> > Kernel version:3.10.0-514.el7.x86_64
> > DRBD:drbd-9.0.8-1
> >      drbd-utils-9.0.0
> >      drbdmanage-0.99.5
> > Host Name:drbd-01、drbd-02
> > Disk for DRBD: /dev/xvdb
> >
> > ・DRBD volume is going to be used  as MySQL data volume.
> > ・AWS env is only test environment. PROD env is VM ESXi.
> >
> > 【Installing steps】
> > ・Installing required packages
> > # yum -y install kernel-devel.x86_64
> > # yum -y install gcc.x86_64
> > # yum -y install flex.x86_64
> >
> > ・Installing DRBD
> > # cd /usr/local/src/
> > # tar zxf drbd-9.0.8-1.tar.gz
> > # cd drbd-9.0.8-1
> > # make KDIR=/usr/src/kernels/$(uname -r)
> > # make install
> > # modprobe drbd
> > # cat /proc/drbd
> >
> > ・Installing drbd-util
> > # cd /usr/local/src/
> > # tar zxf drbd-utils-9.0.0.tar.gz
> > # cd drbd-utils-9.0.0
> > # ./configure
> > # make
> > # make install
> >
> > ・Installing DRBD Manage
> > # cd /usr/local/src/
> > # tar zxf drbdmanage-0.99.5.tar.gz
> > # cd drbdmanage-0.99.5
> > # ./setup.py build
> > # ./setup.py install
> >
> > 【Initialize DRBD】
> > - Creating pool for DRBD
> > # vgcreate drbdpool /dev/xvdb1
> >   Physical volume "/dev/xvdb1" successfully created.
> >   Volume group "drbdpool" successfully created
> > #
> >
> > - Execute on drbd-01
> > # drbdmanage init 172.31.1.155
> >
> > You are going to initialize a new drbdmanage cluster.
> > CAUTION! Note that:
> >   * Any previous drbdmanage cluster information may be removed
> >   * Any remaining resources managed by a previous drbdmanage installation
> >     that still exist on this system will no longer be managed by
> drbdmanage
> >
> > Confirm:
> >
> >   yes/no: yes
> > Empty drbdmanage control volume initialized on '/dev/drbd0'.
> > Empty drbdmanage control volume initialized on '/dev/drbd1'.
> > Waiting for server: .
> > Operation completed successfully
> > #
> > # drbdadm status
> > .drbdctrl role:Primary
> >   volume:0 disk:UpToDate
> >   volume:1 disk:UpToDate
> >
> > #
> > #
> > # lvdisplay -c
> >   /dev/drbdpool/.drbdctrl_0:drbdpool:3:1:-1:2:8192:1:-1:0:-1:253:0
> >   /dev/drbdpool/.drbdctrl_1:drbdpool:3:1:-1:2:8192:1:-1:0:-1:253:1
> > #
> > # drbdmanage list-nodes
> > +-----------------------------------------------------------
> -------------------------------------------------+
> > | Name    | Pool Size | Pool Free |
> >                          | State |
> > |-----------------------------------------------------------
> -------------------------------------------------|
> > | drbd-01 |     10236 |     10228 |
> >                          |    ok |
> > +-----------------------------------------------------------
> -------------------------------------------------+
> > #
> > # drbdmanage new-node drbd-02 172.31.8.103
> > Operation completed successfully
> > Operation completed successfully
> >
>
> Please read the following lines again:
>
> > Executing join command using ssh.
> > IMPORTANT: The output you see comes from drbd-02
> > IMPORTANT: Your input is executed on drbd-02
>
> Reread the lines above^^ ;-)
>
> > You are going to join an existing drbdmanage cluster.
> > CAUTION! Note that:
> >   * Any previous drbdmanage cluster information may be removed
> >   * Any remaining resources managed by a previous drbdmanage installation
> >     that still exist on this system will no longer be managed by
> drbdmanage
> >
> > Confirm:
> >
> >   yes/no: yes
> > Waiting for server to start up (can take up to 1 min)
> > Waiting for server: ......
> > Operation completed successfully
> > Give leader time to contact the new node
> > Operation completed successfully
> > Operation completed successfully
>
> I guess at that point your second node was successfully joined.
>
> > #
> > #
> > # drbdmanage howto-join drbd-02
> > IMPORTANT: Execute the following command only on node drbd-02!
> > drbdmanage join -p 6999 172.31.8.103 1 drbd-01 172.31.1.155 0
> > aez1qL969FHRDYJH4qYD
> > Operation completed successfully
> > #
> > #
> >
> > - Execute on drbd-02
> > # drbdmanage join -p 6999 172.31.8.103 1 drbd-01 172.31.1.155 0
> > aez1qL969FHRDYJH4qYD
> > You are going to join an existing drbdmanage cluster.
> > CAUTION! Note that:
> >   * Any previous drbdmanage cluster information may be removed
> >   * Any remaining resources managed by a previous drbdmanage installation
> >     that still exist on this system will no longer be managed by
> drbdmanage
> >
> > Confirm:
> >
> >   yes/no: yes
> > Waiting for server to start up (can take up to 1 min)
> > Operation completed successfully
> > #
> > #
>
> Why did you rejoin it? It was already joined.
>
> >
> > - Execute on drbd-01
> > # drbdmanage list-nodes
> > +-----------------------------------------------------------
> -------------------------------+
> > | Name    | Pool Size | Pool Free |                          |
> >           State |
> > |-----------------------------------------------------------
> -------------------------------|
> > | drbd-01 |     10236 |     10228 |                          |
> >  online/quorum vote ignored |
> > | drbd-02 |     10236 |     10228 |                          |
> > offline/quorum vote ignored |
> > +-----------------------------------------------------------
> -------------------------------+
> > [root at drbd-01 drbd.d]#
> >
> > - Execute on drbd-02
> > # drbdmanage list-nodes
> > +-----------------------------------------------------------
> -------------------------------+
> > | Name    | Pool Size | Pool Free |                          |
> >           State |
> > |-----------------------------------------------------------
> -------------------------------|
> > | drbd-01 |     10236 |     10228 |                          |
> > offline/quorum vote ignored |
> > | drbd-02 |     10236 |     10228 |                          |
> >  online/quorum vote ignored |
> > +-----------------------------------------------------------
> -------------------------------+
> > #
> > #
> >
> > When I checked syslog on drbd-01, the messages blow were writen.
> > ----------
> > Jul 24 03:27:58 drbd-01 dbus-daemon: .drbdctrl role:Primary
> > Jul 24 03:27:58 drbd-01 dbus-daemon: volume:0 disk:UpToDate
> > Jul 24 03:27:58 drbd-01 dbus-daemon: volume:1 disk:UpToDate
> > Jul 24 03:29:55 drbd-01 drbdmanaged[2221]: INFO       DrbdAdm: Running
> > external command: drbdadm -vvv adjust .drbdctrl
> > Jul 24 03:29:55 drbd-01 drbdmanaged[2221]: ERROR      DrbdAdm: External
> > command 'drbdadm': Exit code 1
> > Jul 24 03:29:55 drbd-01 dbus-daemon: .drbdctrl role:Primary
> > ----------
> >
> > on drbd-02
> > ----------
> > Jul 24 03:29:59 drbd-02 drbdmanaged[2184]: INFO       DrbdAdm: Running
> > external command: drbdadm -vvv adjust .drbdctrl
> > Jul 24 03:29:59 drbd-02 drbdmanaged[2184]: ERROR      DrbdAdm: External
> > command 'drbdadm': Exit code 1
> > Jul 24 03:29:59 drbd-02 drbdmanaged[2184]: INFO       DRBDManage starting
> > as potential leader node
> > Jul 24 03:29:59 drbd-02 dbus-daemon: .drbdctrl role:Secondary
> > Jul 24 03:29:59 drbd-02 dbus-daemon: volume:0 disk:Inconsistent
> > Jul 24 03:29:59 drbd-02 dbus-daemon: volume:1 disk:Inconsistent
> > Jul 24 03:29:59 drbd-02 dbus-daemon: drbd-01 connection:Connecting
>
> Looks like the two machines never connected successfully on DRBD level.
> I saw that recently on AWS and it is usually a problem with the network.
> All ports that you need are open? For the join you used the node name
> shown in "uname -n"?
>
> Review the res file of the control volume and try to connect the
> .drbdctrl on DRBD level. As long as the nodes can not connect on DRBD
> level, the next higher level - drbdmanage - will fail.
>
> Regards, rck
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20170727/b2f65b89/attachment.htm>


More information about the drbd-user mailing list