[DRBD-user] very slow io with drbd and nfs

Matt Graham danceswithcrows at usa.net
Mon Jan 25 20:36:09 CET 2010


From: Fatih <fatih at kamilkoc.com.tr>
> NFSCLIENT------------(NFSSERVER)  drbd node1 active
>                      (NFSSERVER)  drbd node2 passive
> If I copy a 63M file (which is on nfs server) from an nfs client, it takes 
> approximately 55 seconds which is 1.2 MB per second, very slow.

> "drbdadm disconnect resource_name" on node 2. After that, if I copy a 63M
> file (which is on nfs server) from nfs client, it takes approximately 5
> seconds which is 12 MB per second which is normal.

> As you can see I am very confused and have no idea where the problem is.
> Both nfs server node1 and node2 have centos 5.4 32 bit drbd version 8.0.16 
> which is in default yum repository.

> All networks are gbit.  Nfs network and drbd NETWORKS ARE SEPARATED between

> each other with separated gbit nics. There is no router or any other active

> network device, they are connected with a crossover cable.

> resource repdata {
>   protocol A;
>   startup {
>   wfc-timeout 0; degr-wfc-timeout     120;
>   }
>   disk {
>   on-io-error detach;
>   }

When you're using DRBD 8, if your disk controllers have battery backup, you
can get performance gains by putting the options

no-disk-flushes;
no-md-flushes;

...in your disk{} section.  Then run "drbdadm adjust repdata" on both
nodes.  If you disk controllers don't have battery backup, and you do this,
you could have problems if the power goes out.  But a 1.2M/sec read speed
is pretty problematic in and of itself.  Make sure you Read The Fine
Manual about what those options do before putting them in place.

What's the FS that the NFS server is using?  Has it been mounted with
noatime?  What mount options is the NFS client using?  Those things may
be useful to know.  HTH anyway,

-- 
Matt G / Dances With Crows
The Crow202 Blog:  http://crow202.org/wordpress/
There is no Darkness in Eternity/But only Light too dim for us to see





More information about the drbd-user mailing list