<div dir="ltr">Thank you Phil. Yes, I replied earlier this morning to this thread confirming that it is indeed zfs using the wrong flag (FMODE_EXCL) in newer kernels (post 2.6.32) when opening the drbd resource (or any block device for that matter). We didn't know this until late Friday evening, so I wasn't aware it was a zfs issue before starting this thread. We are going to pull in the fix from zfs 0.8.x to make this work with auto-promote. Thanks for your time and replies.<div><br></div><div>Thanks,</div><div>-Doug</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 18, 2019 at 4:27 PM Philipp Reisner <<a href="mailto:philipp.reisner@linbit.com">philipp.reisner@linbit.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
from top of my head ZFS is special in that sense that it opens the backing<br>
device only for a short amount of time. <br>
<br>
I mean it does the eqivalent of an open(2) call on the backend block device<br>
and than very soon after that does the equivalent of a close(2).<br>
<br>
All other linux file systems open(2) it during mount and keep it<br>
open until unmount. Which seems pretty logical.<br>
<br>
DRBD's auto-promote relies on open/close calls. In other words, if you<br>
put ZFS on top of DRBD, do not use auto-promote. Use manual promote.<br>
<br>
It is nothing we can fix on DRBD's side. This should be fixed in ZFS.<br>
<br>
best regards,<br>
Phil<br>
<br>
<br>
Am Mittwoch, 13. November 2019, 14:08:37 CST schrieb Doug Cahill:<br>
> I'm configuring a two node setup with drbd 9.0.20-1 on CentOS 7<br>
> (3.10.0-957.1.3.el7.x86_64) with a single resource backed by an SSDs. I've<br>
> explicitly enabled auto-promote in my resource configuration to use this<br>
> feature.<br>
> <br>
> The drbd device is being used in a single-primary configuration as a zpool<br>
> SLOG device. The zpool is only ever imported on one node at a time and the<br>
> import is successful during cluster failover events between nodes. I<br>
> confirmed through zdb that the zpool includes the configured drbd device<br>
> path.<br>
> <br>
> My concern is that the drbdadm status output shows the Role of the drbd<br>
> resource as "Secondary" on both sides. The documentations reads that the<br>
> drbd resource will be auto promoted to primary when it is opened for<br>
> writing.<br>
> <br>
> drbdadm status<br>
> r0 role:Secondary<br>
> disk:UpToDate<br>
> dccdx0 role:Secondary<br>
> peer-disk:UpToDate<br>
> <br>
> My device should be opened for writing when the zpool is imported. I've<br>
> even tested writing to the pool with "dd oflag=sync" to force sync writes<br>
> to the SLOG which is the drbd resource. The drbd device never changes the<br>
> reported state but it appears to be writable.<br>
> <br>
> Have I misconfigured my drbd resource for an auto-promote configuration<br>
> and/or is my use case to obscure for auto-promote to detect the device is<br>
> being written to when used in a zpool?<br>
> <br>
> =========== /etc/drbd.d/r0.res<br>
> resource r0 {<br>
> disk "/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:2:0-part1";<br>
> meta-disk internal;<br>
> device minor 0;<br>
> <br>
> on dccdx0 {<br>
> address <a href="http://192.0.2.10:7000" rel="noreferrer" target="_blank">192.0.2.10:7000</a>;<br>
> }<br>
> <br>
> on dccdx1 {<br>
> address <a href="http://192.0.2.20:7000" rel="noreferrer" target="_blank">192.0.2.20:7000</a>;<br>
> }<br>
> <br>
> disk {<br>
> read-balancing when-congested-remote;<br>
> no-disk-flushes;<br>
> no-md-flushes;<br>
> # al-updates no; # turn off al-tracking; requires full sync on crash<br>
> c-fill-target 4M;<br>
> resync-rate 500M;<br>
> c-max-rate 1000M;<br>
> c-min-rate 700M;<br>
> }<br>
> <br>
> net {<br>
> sndbuf-size 10M;<br>
> rcvbuf-size 10M;<br>
> max-buffers 20000;<br>
> fencing resource-and-stonith;<br>
> }<br>
> }<br>
> <br>
> -Doug<br>
<br>
<br>
<br>
<br>
</blockquote></div>