<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Hi Joel,<br>
<br>
The additional debug setting gives one additional line:<br>
2019-06-12T09:58:27.977088+02:00 barentsz kern warning kernel - - -
[525370.955129] EXT4-fs (dm-16): DAX enabled. Warning: EXPERIMENTAL,
use at your own risk<br>
2019-06-12T09:58:27.977121+02:00 barentsz kern debug kernel - - -
[525370.955135] dm-16: error: dax access failed (-95)<br>
2019-06-12T09:58:27.977122+02:00 barentsz kern err kernel - - -
[525370.955136] EXT4-fs (dm-16): DAX unsupported by block device.
Turning off DAX.<br>
2019-06-12T09:58:27.977124+02:00 barentsz kern info kernel - - -
[525370.957006] EXT4-fs (dm-16): mounted filesystem with ordered
data mode. Opts: dax<br>
<br>
Regards,<br>
Rob<br>
<br>
<br>
<div class="moz-cite-prefix">On 6/12/19 9:50 AM, Joel Colledge
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAGNP_+VeFYOor4JpfLtJCPRYwEaCs1f-OLveLNzP1H13TNAazw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Hi Rob,
<div><br>
</div>
<div>This is strange, since the filesystem DAX access uses
essentially the same checks as DRBD does. You can get more
detail about the failure by doing the mount test again after
enabling the relevant debug logging as follows:</div>
<div>echo "file drivers/dax/super.c +p" >
/sys/kernel/debug/dynamic_debug/control<br>
</div>
<div><br>
</div>
<div>Perhaps there is a subtle difference in the checks that
DRBD does which is relevant for your device on this kernel.</div>
<div><br>
</div>
<div>Best regards,</div>
<div>Joel</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Jun 12, 2019 at 8:04
AM Rob van der Wal <<a
href="mailto:rob.vanderwal@surfsara.nl"
moz-do-not-send="true">rob.vanderwal@surfsara.nl</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Hi Joel,<br>
<br>
Yes, I can mount with "-o dax":<br>
<br>
lvcreate -L 40G -n test-lv vg_ssd<br>
scan_dev_close /dev/sdb2 no DEV_IN_BCACHE set<br>
Logical volume "test-lv" created.<br>
mkfs.ext4 -b 4K /dev/vg_ssd/test-lv<br>
mount -o dax /dev/vg_ssd/test-lv /mnt<br>
df /mnt<br>
Filesystem 1K-blocks Used Available Use%
Mounted on<br>
/dev/mapper/vg_ssd-test--lv 41022688 49176 38859976 1%
/mnt<br>
<br>
But it shows a failure in syslog:<br>
[518270.833954] EXT4-fs (dm-16): DAX enabled. Warning:
EXPERIMENTAL, use at your own risk<br>
[518270.833959] EXT4-fs (dm-16): DAX unsupported by block
device. Turning off DAX.<br>
[518270.835599] EXT4-fs (dm-16): mounted filesystem with
ordered data mode. Opts: dax<br>
<br>
Regards,<br>
Rob<br>
<br>
<br>
<div class="gmail-m_3095418601436962624moz-cite-prefix">On
6/11/19 3:57 PM, Joel Colledge wrote:<br>
</div>
<blockquote type="cite">
<pre class="gmail-m_3095418601436962624moz-quote-pre">Hi Rob,
This is strange. It seems that your LV is reporting that it supports
PMEM/DAX. I suggest that you check that this issue also occurs without
DRBD. For example, create a filesystem and try to mount it with DAX
enabled:
mkfs.ext4 -b 4K /dev/<your-lv>
mount -o dax /dev/<your-lv> <somewhere>
The check the syslog to see whether DAX was successfully enabled. You
can see the expected success and failure output here:
<a class="gmail-m_3095418601436962624moz-txt-link-freetext" href="https://nvdimm.wiki.kernel.org/#filesystems" target="_blank" moz-do-not-send="true">https://nvdimm.wiki.kernel.org/#filesystems</a>
Best regards,
Joel
PS Please "reply all" to keep the mailing list in CC
On Tue, Jun 11, 2019 at 2:53 PM Rob van der Wal
<a class="gmail-m_3095418601436962624moz-txt-link-rfc2396E" href="mailto:rob.vanderwal@surfsara.nl" target="_blank" moz-do-not-send="true"><rob.vanderwal@surfsara.nl></a> wrote:
</pre>
<blockquote type="cite">
<pre class="gmail-m_3095418601436962624moz-quote-pre">Hi Joel,
I've build the module from source (downloaded from <a class="gmail-m_3095418601436962624moz-txt-link-freetext" href="https://www.linbit.com/en/drbd-community/drbd-download/" target="_blank" moz-do-not-send="true">https://www.linbit.com/en/drbd-community/drbd-download/</a>).
I think we don't use external metadata and we are using a ssd disk for storing. This is a part of our drbd.conf:
drbd.conf:
on barentsz {
node-id 1;
device /dev/drbd0 minor 0;
disk /dev/vg_ssd/vm_prace-ldap;
meta-disk internal;
address ipv4 <a href="http://10.0.0.30:7788" target="_blank" moz-do-not-send="true">10.0.0.30:7788</a>;
}
LVM info:
PV /dev/sdb2 VG vg_ssd lvm2 [446.43 GiB / 102.43 GiB free]
ACTIVE '/dev/vg_ssd/vm_prace-ldap' [38.00 GiB] inherit
The requested line in the syslog is:
[1037978.736262] drbd r0/0 drbd0: meta-data IO uses: dax-pmem
But I don't think we are using a PMEM device so this looks not correct.
Regards,
Rob van der Wal
</pre>
</blockquote>
</blockquote>
<br>
<div class="gmail-m_3095418601436962624moz-signature">-- <br>
<div><font face="Arial">Rob van der Wal</font></div>
<div><font face="Arial"><span style="font-size:13px">Supercomputing
| SURFsara | Science Park 140 | 1098 XG Amsterdam |
T +31 20 800 1300 | <a
class="gmail-m_3095418601436962624moz-txt-link-abbreviated"
href="mailto:Rob.vanderWal@surfsara.nl"
target="_blank" moz-do-not-send="true">Rob.vanderWal@surfsara.nl</a>
| <a
class="gmail-m_3095418601436962624moz-txt-link-abbreviated"
href="http://www.surfsara.nl" target="_blank"
moz-do-not-send="true">www.surfsara.nl</a> |</span></font></div>
<div><font face="Arial"><span style="font-size:13px"></span></font></div>
<br>
<div style="display:inline-block;vertical-align:top"> <img
src="cid:part8.C33F0539.D2805799@surfsara.nl"
type="image/jpg" class=""></div>
<div style="display:inline-block">
<div><span style="font-size:13px;color:rgb(0,85,165)"> We
are ISO 27001 certified and meet the high
requirements for information security. </span><span
style="box-sizing:inherit;color:rgb(0,0,205)"><img
src="cid:part9.4410BBD8.AF5894AE@surfsara.nl"
type="image/png" class=""></span></div>
</div>
</div>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr"><font size="2" face="arial, helvetica,
sans-serif"><span style="color:rgb(0,0,0)">Joel
Colledge</span><span style="color:rgb(0,0,0)"> - </span><span
style="color:rgb(0,0,0)">Software Developer</span><br
style="color:rgb(0,0,0)">
<a href="tel:+4318178292" style="color:rgb(0,0,0)"
target="_blank" moz-do-not-send="true">+43-1-817-82-92
x<span>53</span></a><br style="color:rgb(0,0,0)">
<a href="mailto:joel.colledge@linbit.com"
style="color:rgb(0,0,0)" target="_blank"
moz-do-not-send="true"><span>joel.colledge</span>@linbit.com</a><br
style="color:rgb(0,0,0)">
<br style="color:rgb(0,0,0)">
<a href="http://www.linbit.com/en/"
style="color:rgb(250,133,34)" target="_blank"
moz-do-not-send="true">LIN</a><a
href="http://www.linbit.com/en/"
style="color:rgb(0,0,0)" target="_blank"
moz-do-not-send="true">BIT</a><span
style="color:rgb(0,0,0)"> | Keeping the Digital
World Running</span><br style="color:rgb(0,0,0)">
<span style="color:rgb(0,0,0)">DRBD HA - Disaster
Recovery - Software-defined Storage</span><br
style="color:rgb(0,0,0)">
<a href="https://twitter.com/linbit"
title="Twitter" style="color:rgb(0,172,237)"
target="_blank" moz-do-not-send="true">t</a><span
style="color:rgb(0,0,0)"> / </span><a
href="https://www.facebook.com/pg/linbitdrbd/posts/"
title="Facebook" style="color:rgb(59,89,152)"
target="_blank" moz-do-not-send="true">f</a><span
style="color:rgb(0,0,0)"> / </span><a
href="https://www.linkedin.com/company/linbit"
title="LinkedIn" style="color:rgb(0,123,182)"
target="_blank" moz-do-not-send="true">in</a><span
style="color:rgb(0,0,0)"> / </span><a
href="https://www.youtube.com/user/linbit"
title="YouTube" style="color:rgb(255,0,0)"
target="_blank" moz-do-not-send="true">y</a><span
style="color:rgb(0,0,0)"> / </span><a
href="https://plus.google.com/+Linbit/about"
title="Google+" style="color:rgb(221,75,57)"
target="_blank" moz-do-not-send="true">g+</a><br
style="color:rgb(0,0,0)">
<br style="color:rgb(0,0,0)">
<span style="color:rgb(0,0,0)">DRBD® and LINBIT®
are registered trademarks of LINBIT, Austria.</span></font><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<div><font face="Arial">Rob van der Wal</font></div>
<div><font face="Arial"><span style="font-size: 13px;">Supercomputing
| SURFsara | Science Park 140 | 1098 XG Amsterdam | T +31 20
800 1300 | <a class="moz-txt-link-abbreviated" href="mailto:Rob.vanderWal@surfsara.nl">Rob.vanderWal@surfsara.nl</a> | <a class="moz-txt-link-abbreviated" href="http://www.surfsara.nl">www.surfsara.nl</a> |</span></font></div>
<div><font face="Arial"><span style="font-size: 13px;"></span></font></div>
<br>
<div style="display:inline-block;vertical-align:top;">
<img src="file:/home/rob/images/SURFsara" type="image/jpg"></div>
<div style="display:inline-block;">
<div><span style="font-size: 13px; color: #0055a5;"> We
are ISO 27001 certified and meet the high requirements for
information security. </span><span style="box-sizing:
inherit; color: #0000cd;"><img
src="file:/home/rob/images/ISO27001" type="image/png"></span></div>
</div>
</div>
</body>
</html>