[DRBD-user] Write barriers with ext3 (-o barrier=1)

[Lars Volker]

Hallo everybody,

i'm currently trying to get barriers to work on ext3. So far I have been 
successful, but when it comes to DRBD, barriers don't seem to be 
supported. Let me describe the setup.

2 boxes used for the test: dual 4core opteron, 8G ram, adaptec sas raid6 
supporting tcq. OS is CentOS 5.2.

DRBD uses /dev/sda3, so the overall setup looks like this:

/dev/sda3 <-> DRBD <-> lvm <-> ext3

I already know of the issues lvm and especially the device mapper has 
with barriers. CentOS comes with barriers enabled for linear 
dm-personalities which I used here.

If I mount a volume with "mount -o barrier=1" in the above setup and 
attempt to write, i get

"JBD: barrier-based sync failed on dm-7 - disabling barriers"

In the syslog. By looking into the jbd-code it's clear, that this error 
should be caused by drbd not supporting barriers. If I remove DRBD from 
the setup and use sda3 as physical volume, this error does not occur.

I tried and verified this for WFConnection, Standalone and Connected state.

Here is the drbd.conf file:

==== drbd.conf ====

resource r0 {

   protocol C;

   handlers {
     pri-on-incon-degr "echo o > /proc/sysrq-trigger ; halt -f";
     pri-lost-after-sb "echo o > /proc/sysrq-trigger ; halt -f";
     local-io-error "echo o > /proc/sysrq-trigger ; halt -f";
     #outdate-peer "/usr/lib/heartbeat/drbd-peer-outdater -t 5";
   }

   startup {
     wfc-timeout 0;
     degr-wfc-timeout 120;    # 2 minutes.
   }

   disk {
     on-io-error   detach;
     no-disk-flushes;
     no-md-flushes;
   }

   net {
     max-buffers     4096;
     # unplug-watermark   128;
     after-sb-0pri disconnect;
     after-sb-1pri disconnect;
     after-sb-2pri disconnect;
     rr-conflict disconnect;
   }


   on bert {
     device     /dev/drbd0;
     disk       /dev/sda3;
     address    192.168.1.1:9901;
     flexible-meta-disk  internal;
   }

   on ernie {
     device    /dev/drbd0;
     disk      /dev/sda3;
     address   192.168.1.2:9901;
     flexible-meta-disk internal;
   }
}

==== end ====

The kernel running on the machines is the current openvz kernel released 
for rhel5. However i recompiled it without the builtin drbd stuff and 
compiled drbd 8.2.6 as a module afterwards.

I also verified this with the kernel shipped with centos, so it is not 
caused by my self-packaged kernel.

The question is: Could someone please comment on the current status of 
write barriers with drbd? Is it supported to mount ext3 with "-o 
barrier=1"? If so, what am i doing wrong?

Please CC replies to me, as i'm note subscribed to the list.


Kind regards,

   Lars Volker