[DRBD-user] Performance issues with drbd and nfs

Stephano-Shachter, Dylan dstathis at seas.harvard.edu
Mon Jun 6 22:17:02 CEST 2016

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hello all,

I am building an HA NFS server using drbd and pacemaker. Everything is
working well except I am getting lower write speeds than I would expect. I
have been doing all of my benchmarking with bonnie++. I always get read
speeds of about 112 MB/s which is just about saturating the network. When I
perform a write, however, I get about 89 MB/s which is significantly slower.

The weird thing is that if I run the test locally, on the server (not using
nfs), I get 112 MB/s read. Also, if I run the tests over nfs but with the
secondary downed via "drbdadm down name", then I also get 112 MB/s. I can't
understand what is causing the bottleneck if it is not drbd replication or
nfs.

If anyone could help me to figure out what is slowing down the write
performance if would be very helpful. My configs are


--------------------drbd-config-----------------------------


# /etc/drbd.conf
global {
    usage-count yes;
    cmd-timeout-medium 600;
    cmd-timeout-long 0;
}

common {
    net {
        protocol           C;
        after-sb-0pri    discard-zero-changes;
        after-sb-1pri    discard-secondary;
        after-sb-2pri    disconnect;
        max-buffers      8000;
        max-epoch-size   8000;
    }
    disk {
        resync-rate      1024M;
    }
    handlers {
        pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;
reboot -f";
        pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;
reboot -f";
        local-io-error   "/usr/lib/drbd/notify-io-error.sh;
/usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ;
halt -f";
        split-brain      "/usr/lib/drbd/notify-split-brain.sh root";
    }
}

# resource <res_name> on <host1>: not ignored, not stacked
# defined at /etc/drbd.d/<res_name>.res:1
resource <res_name> {
    on <host2> {
        device           /dev/drbd1 minor 1;
        disk             /dev/sdb1;
        meta-disk        internal;
        address          ipv4 55.555.55.55:7789;
    }
    on <host1> {
        device           /dev/drbd1 minor 1;
        disk             /dev/sdb1;
        meta-disk        internal;
        address          ipv4 55.555.55.55:7789;
    }
    net {
        allow-two-primaries  no;
        after-sb-0pri    discard-zero-changes;
        after-sb-1pri    discard-secondary;
        after-sb-2pri    disconnect;
    }
}



-----------------------nfs.conf-----------------------------



MOUNTD_NFS_V3="yes"
RPCNFSDARGS="-N 2"
LOCKD_TCPPORT=32803
LOCKD_UDPPORT=32769
MOUNTD_PORT=892
RPCNFSDCOUNT=48
#RQUOTAD_PORT=875
#STATD_PORT=662
#STATD_OUTGOING_PORT=2020
STATDARG="--no-notify"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20160606/17a7c632/attachment.htm>


More information about the drbd-user mailing list