<div dir="ltr"><div>Two things:</div><div><br></div>- I would use drbd8 instead of drbd9 for a 2 node setup.<div>- I would first test with 1 nic instead of 2.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 23, 2018 at 11:01 AM, Dirk Bonenkamp - ProActive <span dir="ltr"><<a href="mailto:dirk@proactive.nl" target="_blank">dirk@proactive.nl</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi List,<br>
<br>
I'm struggling with a new DRBD9 setup. It's a simple Master/Slave setup.<br>
I'm running Ubuntu 16.04 LTS with the DRBD9 packages from the Launchpad PPA.<br>
<br>
I'm running some DRBD8 systems in production for quite some years, so I<br>
have some experience. This setup is very similar, the only major<br>
difference is that this is DRBD9 and I use LUKS encrypted partitions as<br>
backend.<br>
<br>
I keep running into this 'PingAck did not arrive in time.' error, which<br>
points to network issues if I am correct (see complete log snippet<br>
below). This error occurs when I try to reattach the secondary node<br>
after a reboot. Initial sync works fine.<br>
<br>
The servers are interconnected with 2 10Gb NICs. I had bonding & jumbo<br>
frames configured, but deactivated all this, to no avail. I've also<br>
stripped the DRBD configuration to the bare minimum (see below).<br>
<br>
I've tested the connection with iperf and some other tools and it seems<br>
just fine.<br>
<br>
Could somebody point me in the right direction?<br>
<br>
Thank you in advance, regards,<br>
<br>
Dirk Bonenkamp<br>
<br>
syslog messages:<br>
<br>
May 23 11:31:56 data2 kernel: [ 704.111755] drbd: loading out-of-tree<br>
module taints kernel.<br>
May 23 11:31:56 data2 kernel: [ 704.112290] drbd: module verification<br>
failed: signature and/or required key missing - tainting kernel<br>
May 23 11:31:56 data2 kernel: [ 704.127677] drbd: initialized. Version:<br>
9.0.14-1 (api:2/proto:86-113)<br>
May 23 11:31:56 data2 kernel: [ 704.127680] drbd: GIT-hash:<br>
62f906cf44ef02a30ce0c148fec223<wbr>b40c51c533 build by root@data2, 2018-05-23<br>
09:19:54<br>
May 23 11:31:56 data2 kernel: [ 704.127683] drbd: registered as block<br>
device major 147<br>
May 23 11:31:56 data2 kernel: [ 704.153565] drbd r0: Starting worker<br>
thread (from drbdsetup [4495])<br>
May 23 11:31:56 data2 kernel: [ 704.183031] drbd r0/0 drbd0: disk(<br>
Diskless -> Attaching )<br>
May 23 11:31:56 data2 kernel: [ 704.183066] drbd r0/0 drbd0: Maximum<br>
number of peer devices = 1<br>
May 23 11:31:56 data2 kernel: [ 704.183293] drbd r0: Method to ensure<br>
write ordering: flush<br>
May 23 11:31:56 data2 kernel: [ 704.183308] drbd r0/0 drbd0:<br>
drbd_bm_resize called with capacity == 273437203064<br>
May 23 11:31:58 data2 kernel: [ 706.508228] drbd r0/0 drbd0: resync<br>
bitmap: bits=34179650383 words=534057038 pages=1043081<br>
May 23 11:31:58 data2 kernel: [ 706.508234] drbd r0/0 drbd0: size = 127<br>
TB (136718601532 KB)<br>
May 23 11:31:58 data2 kernel: [ 706.508236] drbd r0/0 drbd0: size = 127<br>
TB (136718601532 KB)<br>
May 23 11:32:10 data2 kernel: [ 717.890420] drbd r0/0 drbd0: recounting<br>
of set bits took additional 1256ms<br>
May 23 11:32:10 data2 kernel: [ 717.890435] drbd r0/0 drbd0: disk(<br>
Attaching -> Outdated )<br>
May 23 11:32:10 data2 kernel: [ 717.890439] drbd r0/0 drbd0: attached<br>
to current UUID: 244DD61D2781DF44<br>
May 23 11:32:10 data2 kernel: [ 717.918473] drbd r0 data1: Starting<br>
sender thread (from drbdsetup [4544])<br>
May 23 11:32:10 data2 kernel: [ 717.922534] drbd r0 data1: conn(<br>
StandAlone -> Unconnected )<br>
May 23 11:32:10 data2 kernel: [ 717.922820] drbd r0 data1: Starting<br>
receiver thread (from drbd_w_r0 [4498])<br>
May 23 11:32:10 data2 kernel: [ 717.922973] drbd r0 data1: conn(<br>
Unconnected -> Connecting )<br>
May 23 11:32:10 data2 kernel: [ 718.421219] drbd r0 data1: Handshake to<br>
peer 1 successful: Agreed network protocol version 113<br>
May 23 11:32:10 data2 kernel: [ 718.421229] drbd r0 data1: Feature<br>
flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME<br>
WRITE_ZEROES.<br>
May 23 11:32:10 data2 kernel: [ 718.421259] drbd r0 data1: Starting<br>
ack_recv thread (from drbd_r_r0 [4550])<br>
May 23 11:32:10 data2 kernel: [ 718.424095] drbd r0: Preparing<br>
cluster-wide state change 1205605755 (0->1 499/146)<br>
May 23 11:32:10 data2 kernel: [ 718.437172] drbd r0: State change<br>
1205605755: primary_nodes=2, weak_nodes=FFFFFFFFFFFFFFFC<br>
May 23 11:32:10 data2 kernel: [ 718.437185] drbd r0: Aborting<br>
cluster-wide state change 1205605755 (12ms) rv = -22<br>
May 23 11:32:12 data2 kernel: [ 719.896223] drbd r0: Preparing<br>
cluster-wide state change 445952355 (0->1 499/146)<br>
May 23 11:32:12 data2 kernel: [ 719.896498] drbd r0: State change<br>
445952355: primary_nodes=2, weak_nodes=FFFFFFFFFFFFFFFC<br>
May 23 11:32:12 data2 kernel: [ 719.896508] drbd r0: Committing<br>
cluster-wide state change 445952355 (0ms)<br>
May 23 11:32:12 data2 kernel: [ 719.896541] drbd r0 data1: conn(<br>
Connecting -> Connected ) peer( Unknown -> Primary )<br>
May 23 11:32:12 data2 kernel: [ 719.912186] drbd r0/0 drbd0 data1:<br>
drbd_sync_handshake:<br>
May 23 11:32:12 data2 kernel: [ 719.912198] drbd r0/0 drbd0 data1: self<br>
244DD61D2781DF44:<wbr>0000000000000000:<wbr>0000000000000000:<wbr>0000000000000000<br>
bits:52035 flags:20<br>
May 23 11:32:12 data2 kernel: [ 719.912207] drbd r0/0 drbd0 data1: peer<br>
E38BE51FE782EAE0:<wbr>244DD61D2781DF44:<wbr>934CAB8662DF0410:<wbr>E555BDC58E528356<br>
bits:53162 flags:20<br>
May 23 11:32:12 data2 kernel: [ 719.912214] drbd r0/0 drbd0 data1:<br>
uuid_compare()=-2 by rule 50<br>
May 23 11:32:12 data2 kernel: [ 719.912248] drbd r0/0 drbd0 data1:<br>
pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )<br>
May 23 11:32:32 data2 kernel: [ 740.397026] drbd r0 data1: PingAck did<br>
not arrive in time.<br>
May 23 11:32:32 data2 kernel: [ 740.397121] drbd r0 data1: conn(<br>
Connected -> NetworkFailure ) peer( Primary -> Unknown )<br>
May 23 11:32:32 data2 kernel: [ 740.397131] drbd r0/0 drbd0 data1:<br>
pdsk( UpToDate -> DUnknown ) repl( WFBitMapT -> Off )<br>
May 23 11:32:32 data2 kernel: [ 740.397176] drbd r0 data1: ack_receiver<br>
terminated<br>
May 23 11:32:32 data2 kernel: [ 740.397182] drbd r0 data1: Terminating<br>
ack_recv thread<br>
May 23 11:32:32 data2 kernel: [ 740.458608] drbd r0 data1: Connection<br>
closed<br>
May 23 11:32:32 data2 kernel: [ 740.458650] drbd r0 data1: conn(<br>
NetworkFailure -> Unconnected )<br>
May 23 11:32:32 data2 kernel: [ 740.458688] drbd r0 data1: Restarting<br>
receiver thread<br>
May 23 11:32:32 data2 kernel: [ 740.458723] drbd r0 data1: conn(<br>
Unconnected -> Connecting )<br>
<br>
resources:<br>
<br>
resource r0 {<br>
on data1 {<br>
device /dev/drbd0;<br>
disk /dev/mapper/mapper_secure;<br>
address <a href="http://172.16.11.21:7789" rel="noreferrer" target="_blank">172.16.11.21:7789</a>;<br>
meta-disk internal;<br>
}<br>
on data2 {<br>
device /dev/drbd0;<br>
disk /dev/mapper/mapper_secure;<br>
address <a href="http://172.16.11.22:7789" rel="noreferrer" target="_blank">172.16.11.22:7789</a>;<br>
meta-disk internal;<br>
}<br>
}<br>
<br>
drbd configuration:<br>
<br>
global {<br>
usage-count yes;<br>
}<br>
<br>
common {<br>
#handlers {<br>
# fence-peer "/usr/lib/drbd/<a href="http://crm-fence-peer.9.sh" rel="noreferrer" target="_blank">crm-fence-peer.<wbr>9.sh</a>";<br>
# after-resync-target "/usr/lib/drbd/<a href="http://crm-unfence-peer.9.sh" rel="noreferrer" target="_blank">crm-unfence-<wbr>peer.9.sh</a>";<br>
#}<br>
#disk {<br>
# on-io-error detach;<br>
# disk-barrier no;<br>
# disk-flushes no;<br>
# al-extents 3833;<br>
# c-plan-ahead 7;<br>
# c-fill-target 2M;<br>
# c-min-rate 80M;<br>
# c-max-rate 720M;<br>
#}<br>
net {<br>
protocol C;<br>
#fencing resource-only;<br>
#cram-hmac-alg sha1;<br>
#verify-alg sha1;<br>
#shared-secret 1e69dc721fd2e65368ae3ba1e59299<wbr>79;<br>
#after-sb-0pri disconnect;<br>
#after-sb-1pri disconnect;<br>
#after-sb-2pri disconnect;<br>
#max-buffers 8000;<br>
#max-epoch-size 8000;<br>
#sndbuf-size 0;<br>
#rcvbuf-size 2048k;<br>
}<br>
}<br>
<br>
<br>
<br>
______________________________<wbr>_________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" rel="noreferrer" target="_blank">http://lists.linbit.com/<wbr>mailman/listinfo/drbd-user</a><br>
</blockquote></div><br></div>