[DRBD-user] Slow NFS behaviour (0.7.5)

Vasiliy Boulytchev VBoulytchev at coinfotech.com
Mon Nov 22 23:15:30 CET 2004


If you set DRBD in mode B, do you get the same issue?  How about A?

-----Original Message-----
From: drbd-user-bounces at lists.linbit.com
[mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Jaroslaw
Zachwieja
Sent: Monday, November 22, 2004 2:33 PM
To: drbd-user at linbit.com
Subject: [DRBD-user] Slow NFS behaviour (0.7.5)



Hello All,

I realise this topic has been discussed many times on the mailing list,
but 
after 4 days of reading the archives I have not found the answer or even

suggestion to what may be the problem.

First, let me describe the setup we are running here.

There are two NFS servers, primary (let's call it "primary" and
"secondary" 
for the sake of clarity). They are connected to the outside world by 
gigabit interfaces. Additionally there is a dedicated (crossover cable) 
interface (1Gbps) for drbd. It has been checked with iperf and the 
real-world troughput is 883Mbps average.

The heartbeat failover works without problems and is based on 100Mbps 
crossover network connection and serial ttyS0<->ttyS0 cable.

All the connections are tested and work at the expected rates.

Below is the mapping of the interfaces:

primary-----secondary-----bandwidth-----purpose

eth6--------eth1----------100Mbps-------heartbeat
eth2--------eth2----------1Gbps---------world
eth4--------eth3----------1Gbps---------drdb
ttyS0-------ttyS0---------19200bps------heartbeat

The storage on the primary is a 1.4TB direct attached (via SCSI U160) 
IDE-raid5. It's SB-3160S Westekuk.com unit. The average read speed form 
the array is 1356.21 MB/sec cached and 88.93 MB/sec buffered. Write
times 
are about 20% less in buffered mode. The storage is connected via
Adaptec 
AIC-7892A U160/m in PCI-X slot. The module loaded is aic7xxx.

Primary is Dual Xeon 2.8GHz, 4GB memory.

The storage on secondary is based on 3ware 7000-series ATA-RAID
controler. 
The module loaded is 3w-xxxx. The disks are configured as RAID5
(hardware 
on card) and have 743.74 MB/sec cached and 76.20 MB/sec buffered 
transfers.

Secondary is a single CPU, Pentium4 2.66GHz, 2GB of memory.

Now, here is the problem.

As soon as I start NFS demons,  Loadavg skyrockets to about 8.5 and
stays 
there. There is no need to actually have any activity, it's enough just
to 
have it there. (I'm using protocol C; max-buffers 16384; max-epoch-size

2048; rate 60M; al-extents 257; in the relevant sections)

I've tested (using iptraf) the amount of data pushed around and it
averages 
around 35 megabytes per second. That itself wouldn't be a problem, 
but /proc/drbd reports around 6 megabytes per second syncing most of the

time (average 9-10 megabytes) and the Primary is unusable as NFS server 
during that operation.

Any attempts to even mount off the nfs server during synch fail with 
timeout.

Without NFS running, drbd seems to work absolutely fine. Now I'm
probably 
going to be sent to nfs-users mailinglist :)

Did anyone experience similar behaviour?

Are there any recommended nice values to minimise the impact of 
synchronisation on the overall performance in "C" mode?

I will appreciate your comments on this matter.

Best regards,
-- 
Jaroslaw Zachwieja
Centre for Scientific Computing
University of Warwick, UK
_______________________________________________
drbd-user mailing list
drbd-user at lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user



More information about the drbd-user mailing list