[DRBD-user] DRBD fsync() seems to return before writing to disk

[Phil Frost]

On 06/20/2012 10:43 AM, Shaun Thomas wrote:
> I think the problem here is that you didn't provide your DRBD config. 
> For all we know, you did this:
>
> disk {
>   no-disk-barrier;
>   no-disk-flushes;
>   no-disk-drain;
>   no-md-flushes;
> }
> [...]
> "Unfortunately device mapper (LVM) might not support barriers."

Ah sorry, I forgot to include that with the original post. Full 
configuration copied below, but I haven't disabled any of the methods 
you mention.

I have read that part of the manual, but I think in my case, LVM does 
support barriers. It looks to me like they began to be supported in 
2.6.29 [1] and all the kinks were ironed out by 2.6.31-rc1 [2]. I'm 
running a 3.2 kernel (though even Debian squeeze runs 2.6.32) so these 
issues should be history.

Further, the fsync() latency I observed running ext4 on MD and LVM (but 
not DRBD devices) empirically suggest that barriers are working for 
DRBD's underlying block device, at least given my understanding that 
working barriers are required for a fsync() that really does flush to 
non-volatile storage.

[1] http://lwn.net/Articles/326597/
[2] https://bugzilla.kernel.org/show_bug.cgi?id=9554

DRBD config follows:

global { usage-count no; }
common {
     protocol C;
     handlers { }
     startup { }
     disk { on-io-error detach; }
     net { cram-hmac-alg sha1; }
     syncer {
         verify-alg crc32c;
         rate 5M;
     }
}
resource nfsexports {
     device minor 0;
     net { shared-secret secret; }
     on storage01 {
         disk /dev/storage01/nfsexports;
         flexible-meta-disk /dev/storage01/nfsexports-drbdmd;
         address 10.0.0.7:7789;
     }
     on storage02 {
         disk /dev/storage02/nfsexports;
         flexible-meta-disk /dev/storage02/nfsexports-drbdmd;
         address 10.0.0.9:7789;
     }
}