Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi all,
I am trying to setup a 3-node cluster with DRBD9. The nodes are VMs on
KVM with CentOS 7.2. I followed manual from
http://docs.linbit.com/docs/users-guide-9.0/#ch-admin-drbdmanage to
initialize the cluster and add nodes to it. I don't have password-less
SSH authentication between the nodes so I first add the slave nodes on
the master node and call the join command returned by `drbdmanage
add-node` on the slave nodes. Storage is LVM with non-default pool name
configured in /etc/drbdmanaged.cfg.
RPMs with DRBD are built from latest tags (drbd: 9.0.9, drbd-utils:
9.1.0, drbdmanage: 0.99.10).
To setup the cluster automatically I use ansible. The roles for master
and slave are very simple, the commands they run are pretty much these:
master
------
drbdmanage init --quiet [IP_ADDRESS]
slaves
------
drbdmanage add-node -j [HOSTNAME] [IP_ADDRESS] (executed on master)
[JOIN COMMAND PRINTED BY "ADD NODE"]
drbdadm wait-connect .drbdctrl
drbdadm wait-sync .drbdctrl
I use `wait-connect`/`wait-sync` because I need to be sure that when
ansible is done the cluster is fully operational. The problem is that
quite often the commands block indefinitely (sometimes `wait-connect`,
sometimes `wait-sync`). What helps, most of the time, is to run `drbdadm
adjust all` on master node. After that the cluster synchronizes and
calls to `wait-connect`/`wait-sync` return. However, I assume that this
step should not be needed - it is not mentioned in the manual at least.
But since DRBD is a new thing for me it is highly likely that I am
missing something, some options in the configuration I should set(?)
some commands I should additionally execute (or maybe `adjust all` is
needed after all?). I would appreciate any help with this, thanks.
Anything related to DRBD found in syslog:
- master:
https://raw.githubusercontent.com/localghost/issues/master/drbd/wait_connect_hangs/10.9.4.216/syslog_drbd.log
- slaves:
*
https://raw.githubusercontent.com/localghost/issues/master/drbd/wait_connect_hangs/10.9.4.166/syslog_drbd.log
*
https://raw.githubusercontent.com/localghost/issues/master/drbd/wait_connect_hangs/10.9.4.231/syslog_drbd.log
I have also collected DRBD configuration, output from `drbd-overview`,
`drbdadm status`, `drbdsetup show` on
https://github.com/localghost/issues/tree/master/drbd/wait_connect_hangs.
If any other logs could be helpful I can re-produce the issue and upload
more logs anytime.
Regards,
Zbigniew Kostrzewa