[DRBD-user] DRBD on zfs vol

Vlad vovan at vovan.nl
Wed May 1 06:49:31 CEST 2013

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


I'm currently testing setup where DRBD is used on top of ZFS (zvol).
Even worse, I'm using btrfs on top of DRBD device. I have this setup
working without problems until I'm doing heavy load tests. When I'm
trying to do bulk write random files to file system (btrfs on DRBD) I
see sometimes messages similar to the one below:

Apr 30 11:31:29 srv2 kernel: d-con diskless-nfs: [drbd_w_diskless/31594] sock_sendmsg time expired, ko = 6

I know that this most likely indicates physical network issues, but:
- I don't have any problems under heavy loads until I use ZFS instead LVM
- "ip -s link dev..." doesn't show any errors or dropped packages
- while doing tests I'm running ping from another node and can't see any
delays or dropped packets

Machines have two Gb network interfaces one is dedicated to DRBD. I'm
running gkrellm monitor form my workstation to see the "the whole
picture" and the strange thing is that while I see errors above the
gkrellm monitor is frozen for a few seconds, but then resumes and works
fine again.

I know this setup is a kind of... strange, but still. I'd appreciate any
help with this. Maybe it's necessary to tune some DRBD parameters when
using on top of ZFS, particularly disk related.

Currently I'm running vanilla kernel 3.8.8 with drbd-8.4.2. The DRBD
device setup is below:

resource diskless-nfs {
    options {
    net {
        protocol                A;
    _remote_host {
        address                 ipv4;
    _this_host {
        address                 ipv4;
        volume 0 {
            device                      minor 30;
            disk                        "/dev/pool/diskless-nfs";
            meta-disk                   internal;
            disk {
                fencing                 resource-only;
                resync-rate             15360k; # bytes/second
} On the primary node I'm using SSD as L2ARC ZFS read cache and with
DRBD protocol A it performs very well. Hopefully somebody else would be
interested in this setup as well.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130501/74380e0d/attachment.htm>

More information about the drbd-user mailing list