<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    Hi Joel,<br>
    <br>
    The additional debug setting gives one additional line:<br>
    2019-06-12T09:58:27.977088+02:00 barentsz kern warning kernel - - -
    [525370.955129] EXT4-fs (dm-16): DAX enabled. Warning: EXPERIMENTAL,
    use at your own risk<br>
    2019-06-12T09:58:27.977121+02:00 barentsz kern debug kernel - - -
    [525370.955135] dm-16: error: dax access failed (-95)<br>
    2019-06-12T09:58:27.977122+02:00 barentsz kern err kernel - - -
    [525370.955136] EXT4-fs (dm-16): DAX unsupported by block device.
    Turning off DAX.<br>
    2019-06-12T09:58:27.977124+02:00 barentsz kern info kernel - - -
    [525370.957006] EXT4-fs (dm-16): mounted filesystem with ordered
    data mode. Opts: dax<br>
    <br>
    Regards,<br>
    Rob<br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 6/12/19 9:50 AM, Joel Colledge
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAGNP_+VeFYOor4JpfLtJCPRYwEaCs1f-OLveLNzP1H13TNAazw@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">Hi Rob,
        <div><br>
        </div>
        <div>This is strange, since the filesystem DAX access uses
          essentially the same checks as DRBD does. You can get more
          detail about the failure by doing the mount test again after
          enabling the relevant debug logging as follows:</div>
        <div>echo "file drivers/dax/super.c +p" &gt;
          /sys/kernel/debug/dynamic_debug/control<br>
        </div>
        <div><br>
        </div>
        <div>Perhaps there is a subtle difference in the checks that
          DRBD does which is relevant for your device on this kernel.</div>
        <div><br>
        </div>
        <div>Best regards,</div>
        <div>Joel</div>
        <div><br>
        </div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Wed, Jun 12, 2019 at 8:04
          AM Rob van der Wal &lt;<a
            href="mailto:rob.vanderwal@surfsara.nl"
            moz-do-not-send="true">rob.vanderwal@surfsara.nl</a>&gt;
          wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0px 0px 0px
          0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div> Hi Joel,<br>
            <br>
            Yes, I can mount with "-o dax":<br>
            <br>
            lvcreate -L 40G -n test-lv  vg_ssd<br>
              scan_dev_close /dev/sdb2 no DEV_IN_BCACHE set<br>
              Logical volume "test-lv" created.<br>
            mkfs.ext4 -b 4K /dev/vg_ssd/test-lv<br>
            mount -o dax /dev/vg_ssd/test-lv /mnt<br>
            df /mnt<br>
            Filesystem                  1K-blocks  Used Available Use%
            Mounted on<br>
            /dev/mapper/vg_ssd-test--lv  41022688 49176  38859976   1%
            /mnt<br>
            <br>
            But it shows a failure in syslog:<br>
            [518270.833954] EXT4-fs (dm-16): DAX enabled. Warning:
            EXPERIMENTAL, use at your own risk<br>
            [518270.833959] EXT4-fs (dm-16): DAX unsupported by block
            device. Turning off DAX.<br>
            [518270.835599] EXT4-fs (dm-16): mounted filesystem with
            ordered data mode. Opts: dax<br>
            <br>
            Regards,<br>
            Rob<br>
            <br>
            <br>
            <div class="gmail-m_3095418601436962624moz-cite-prefix">On
              6/11/19 3:57 PM, Joel Colledge wrote:<br>
            </div>
            <blockquote type="cite">
              <pre class="gmail-m_3095418601436962624moz-quote-pre">Hi Rob,

This is strange. It seems that your LV is reporting that it supports
PMEM/DAX. I suggest that you check that this issue also occurs without
DRBD. For example, create a filesystem and try to mount it with DAX
enabled:
mkfs.ext4 -b 4K /dev/&lt;your-lv&gt;
mount -o dax /dev/&lt;your-lv&gt; &lt;somewhere&gt;

The check the syslog to see whether DAX was successfully enabled. You
can see the expected success and failure output here:
<a class="gmail-m_3095418601436962624moz-txt-link-freetext" href="https://nvdimm.wiki.kernel.org/#filesystems" target="_blank" moz-do-not-send="true">https://nvdimm.wiki.kernel.org/#filesystems</a>

Best regards,
Joel

PS Please "reply all" to keep the mailing list in CC


On Tue, Jun 11, 2019 at 2:53 PM Rob van der Wal
<a class="gmail-m_3095418601436962624moz-txt-link-rfc2396E" href="mailto:rob.vanderwal@surfsara.nl" target="_blank" moz-do-not-send="true">&lt;rob.vanderwal@surfsara.nl&gt;</a> wrote:
</pre>
              <blockquote type="cite">
                <pre class="gmail-m_3095418601436962624moz-quote-pre">Hi Joel,

I've build the module from source (downloaded from <a class="gmail-m_3095418601436962624moz-txt-link-freetext" href="https://www.linbit.com/en/drbd-community/drbd-download/" target="_blank" moz-do-not-send="true">https://www.linbit.com/en/drbd-community/drbd-download/</a>).

I think we don't use external metadata and we are using a ssd disk for storing. This is a part of our drbd.conf:
drbd.conf:
    on barentsz {
        node-id 1;
        device           /dev/drbd0 minor 0;
        disk             /dev/vg_ssd/vm_prace-ldap;
        meta-disk        internal;
        address          ipv4 <a href="http://10.0.0.30:7788" target="_blank" moz-do-not-send="true">10.0.0.30:7788</a>;
    }


LVM info:
  PV /dev/sdb2   VG vg_ssd          lvm2 [446.43 GiB / 102.43 GiB free]
  ACTIVE            '/dev/vg_ssd/vm_prace-ldap' [38.00 GiB] inherit

The requested line in the syslog is:
[1037978.736262] drbd r0/0 drbd0: meta-data IO uses: dax-pmem

But I don't think we are using a PMEM device so this looks not correct.

Regards,
Rob van der Wal

</pre>
              </blockquote>
            </blockquote>
            <br>
            <div class="gmail-m_3095418601436962624moz-signature">-- <br>
              <div><font face="Arial">Rob van der Wal</font></div>
              <div><font face="Arial"><span style="font-size:13px">Supercomputing
                    | SURFsara | Science Park 140 | 1098 XG Amsterdam |
                    T +31 20 800  1300 | <a
                      class="gmail-m_3095418601436962624moz-txt-link-abbreviated"
                      href="mailto:Rob.vanderWal@surfsara.nl"
                      target="_blank" moz-do-not-send="true">Rob.vanderWal@surfsara.nl</a>
                    | <a
                      class="gmail-m_3095418601436962624moz-txt-link-abbreviated"
                      href="http://www.surfsara.nl" target="_blank"
                      moz-do-not-send="true">www.surfsara.nl</a> |</span></font></div>
              <div><font face="Arial"><span style="font-size:13px"></span></font></div>
              <br>
              <div style="display:inline-block;vertical-align:top"> <img
                  src="cid:part8.C33F0539.D2805799@surfsara.nl"
                  type="image/jpg" class=""></div>
              <div style="display:inline-block">
                <div><span style="font-size:13px;color:rgb(0,85,165)">            We
                    are ISO 27001 certified and meet the high
                    requirements for information security. </span><span
                    style="box-sizing:inherit;color:rgb(0,0,205)"><img
                      src="cid:part9.4410BBD8.AF5894AE@surfsara.nl"
                      type="image/png" class=""></span></div>
              </div>
            </div>
          </div>
        </blockquote>
      </div>
      <br clear="all">
      <div><br>
      </div>
      -- <br>
      <div dir="ltr" class="gmail_signature">
        <div dir="ltr">
          <div>
            <div dir="ltr">
              <div>
                <div dir="ltr">
                  <div dir="ltr"><font size="2" face="arial, helvetica,
                      sans-serif"><span style="color:rgb(0,0,0)">Joel
                        Colledge</span><span style="color:rgb(0,0,0)"> - </span><span
                        style="color:rgb(0,0,0)">Software Developer</span><br
                        style="color:rgb(0,0,0)">
                      <a href="tel:+4318178292" style="color:rgb(0,0,0)"
                        target="_blank" moz-do-not-send="true">+43-1-817-82-92
                        x<span>53</span></a><br style="color:rgb(0,0,0)">
                      <a href="mailto:joel.colledge@linbit.com"
                        style="color:rgb(0,0,0)" target="_blank"
                        moz-do-not-send="true"><span>joel.colledge</span>@linbit.com</a><br
                        style="color:rgb(0,0,0)">
                      <br style="color:rgb(0,0,0)">
                      <a href="http://www.linbit.com/en/"
                        style="color:rgb(250,133,34)" target="_blank"
                        moz-do-not-send="true">LIN</a><a
                        href="http://www.linbit.com/en/"
                        style="color:rgb(0,0,0)" target="_blank"
                        moz-do-not-send="true">BIT</a><span
                        style="color:rgb(0,0,0)"> | Keeping the Digital
                        World Running</span><br style="color:rgb(0,0,0)">
                      <span style="color:rgb(0,0,0)">DRBD HA - Disaster
                        Recovery - Software-defined Storage</span><br
                        style="color:rgb(0,0,0)">
                      <a href="https://twitter.com/linbit"
                        title="Twitter" style="color:rgb(0,172,237)"
                        target="_blank" moz-do-not-send="true">t</a><span
                        style="color:rgb(0,0,0)"> / </span><a
                        href="https://www.facebook.com/pg/linbitdrbd/posts/"
                        title="Facebook" style="color:rgb(59,89,152)"
                        target="_blank" moz-do-not-send="true">f</a><span
                        style="color:rgb(0,0,0)"> / </span><a
                        href="https://www.linkedin.com/company/linbit"
                        title="LinkedIn" style="color:rgb(0,123,182)"
                        target="_blank" moz-do-not-send="true">in</a><span
                        style="color:rgb(0,0,0)"> / </span><a
                        href="https://www.youtube.com/user/linbit"
                        title="YouTube" style="color:rgb(255,0,0)"
                        target="_blank" moz-do-not-send="true">y</a><span
                        style="color:rgb(0,0,0)"> / </span><a
                        href="https://plus.google.com/+Linbit/about"
                        title="Google+" style="color:rgb(221,75,57)"
                        target="_blank" moz-do-not-send="true">g+</a><br
                        style="color:rgb(0,0,0)">
                      <br style="color:rgb(0,0,0)">
                      <span style="color:rgb(0,0,0)">DRBD® and LINBIT®
                        are registered trademarks of LINBIT, Austria.</span></font><br>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
    <div class="moz-signature">-- <br>
      <div><font face="Arial">Rob van der Wal</font></div>
      <div><font face="Arial"><span style="font-size: 13px;">Supercomputing
            | SURFsara | Science Park 140 | 1098 XG Amsterdam | T +31 20
            800  1300 | <a class="moz-txt-link-abbreviated" href="mailto:Rob.vanderWal@surfsara.nl">Rob.vanderWal@surfsara.nl</a> | <a class="moz-txt-link-abbreviated" href="http://www.surfsara.nl">www.surfsara.nl</a> |</span></font></div>
      <div><font face="Arial"><span style="font-size: 13px;"></span></font></div>
      <br>
      <div style="display:inline-block;vertical-align:top;">
        <img src="file:/home/rob/images/SURFsara" type="image/jpg"></div>
      <div style="display:inline-block;">
        <div><span style="font-size: 13px; color: #0055a5;">            We
            are ISO 27001 certified and meet the high requirements for
            information security. </span><span style="box-sizing:
            inherit; color: #0000cd;"><img
              src="file:/home/rob/images/ISO27001" type="image/png"></span></div>
      </div>
    </div>
  </body>
</html>