<div dir="ltr">Bump.<div><br></div><div>Could really use some help if anyone has any ideas about how I might improve performance.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 6, 2016 at 4:17 PM, Stephano-Shachter, Dylan <span dir="ltr"><<a href="mailto:dstathis@seas.harvard.edu" target="_blank">dstathis@seas.harvard.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello all,<div><br></div><div>I am building an HA NFS server using drbd and pacemaker. Everything is working well except I am getting lower write speeds than I would expect. I have been doing all of my benchmarking with bonnie++. I always get read speeds of about 112 MB/s which is just about saturating the network. When I perform a write, however, I get about 89 MB/s which is significantly slower.</div><div><br></div><div>The weird thing is that if I run the test locally, on the server (not using nfs), I get 112 MB/s read. Also, if I run the tests over nfs but with the secondary downed via "drbdadm down name", then I also get 112 MB/s. I can't understand what is causing the bottleneck if it is not drbd replication or nfs. </div><div><br></div><div>If anyone could help me to figure out what is slowing down the write performance if would be very helpful. My configs are</div><div><br></div><div><br></div><div>--------------------drbd-config-----------------------------</div><div><br></div><div><br></div><div><div># /etc/drbd.conf</div><div>global {</div><div> usage-count yes;</div><div> cmd-timeout-medium 600;</div><div> cmd-timeout-long 0;</div><div>}</div><div><br></div><div>common {</div><div> net {</div><div> protocol C;</div><div> after-sb-0pri discard-zero-changes;</div><div> after-sb-1pri discard-secondary;</div><div> after-sb-2pri disconnect;</div><div> max-buffers 8000;</div><div> max-epoch-size 8000;</div><div> }</div><div> disk {</div><div> resync-rate 1024M;</div><div> }</div><div> handlers {</div><div> pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";</div><div> pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";</div><div> local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";</div><div> split-brain "/usr/lib/drbd/notify-split-brain.sh root";</div><div> }</div><div>}</div><div><br></div><div># resource <res_name> on <host1>: not ignored, not stacked</div><div># defined at /etc/drbd.d/<res_name>.res:1</div><div>resource <res_name> {</div><div> on <host2> {</div><div> device /dev/drbd1 minor 1;</div><div> disk /dev/sdb1;</div><div> meta-disk internal;</div><div> address ipv4 55.555.55.55:7789;</div><div> }</div><div> on <host1> {</div><div> device /dev/drbd1 minor 1;</div><div> disk /dev/sdb1;</div><div> meta-disk internal;</div><div> address ipv4 55.555.55.55:7789;</div><div> }</div><div> net {</div><div> allow-two-primaries no;</div><div> after-sb-0pri discard-zero-changes;</div><div> after-sb-1pri discard-secondary;</div><div> after-sb-2pri disconnect;</div><div> }</div><div>}</div></div><div><br></div><div><br></div><div><br></div><div>-----------------------nfs.conf-----------------------------</div><div><br></div><div><br></div><div><br></div><div><div>MOUNTD_NFS_V3="yes"</div><div>RPCNFSDARGS="-N 2"</div><div>LOCKD_TCPPORT=32803</div><div>LOCKD_UDPPORT=32769</div><div>MOUNTD_PORT=892</div><div>RPCNFSDCOUNT=48</div><div>#RQUOTAD_PORT=875</div><div>#STATD_PORT=662</div><div>#STATD_OUTGOING_PORT=2020</div><div>STATDARG="--no-notify"</div></div></div>
</blockquote></div><br></div>