[DRBD-user] write performance for DRBD devices seems to be slow

[Warren Beldad]

Hi all!

I have are a simple results of the performance of my drbd test machines.
Using the dd command, first I get the performance value of my
harddisk. That would be formatting the /dev/sda with XFS and then
mount it and then write a 1GB file (which is twice the size of my RAM)
into that disk with dd command on different block sizes and then read
back that file into the black hole.
time dd if=/dev/zero of=testfile.txt bs=512k count=2000
time dd if=testfile.txt of=/dev/null bs=512k
Second create a volume group->logical volume and then create the XFS filesystem
Third is XFS + LVM2 + DRBD

The ff is my  test bed:
CPU = P4 2.66MHz
Memory = 512MB
160GB on RAID 5
2.6.12 kernel
DRBD 0.7.11

below is my results:
 
mkfs.xfs -f /dev/sda
	isize = 256
	sectsz = 512
	bsize = 4096
1. XFS (mount /dev/sda /mnt/temp)
write
	bs=1024k	bs=512k 	bs=64k	bs=16k
	29.385s	28.732s	29.179s	31.723s
read
	bs=1024k	bs=512k	bs=64k	bs=16k
	21.747s	21.74s	21.738s	21.734s

2. XFS + LVM2 ( mount /dev/VG01200/share1 /mnt/temp)
                       ( cd /mnt/temp )
	PE size = 64MB
write
	bs=1024k	bs=512k	bs=64k	bs=16k
	29.651s	28.769s	30.548s	28.78s
read
	bs=1024k	bs=512k	bs=64k	bs=16k
	22.185s	21.923s	21.919s	21.971s

3. XFS + LVM2 + DRBD ( mount /dev/drbd4 /mnt/temp)
                                    ( cd /mnt/temp ) 
	protocol C, drbd.conf attached below
write
	bs=1024k	 bs=512k	   bs=64k	     bs=16k
	1m23.555s 1m25.741s  1m23.700s  1m23.106s
read
	bs=1024k	bs=512k	bs=64k	bs=16k
	22.550s	22.884s	22.558s	22.630s

As you've noticed, the write performance for XFS only and the XFS over
LVM2 are quite similar however on XFS + LVM2 + DRBD things get slower
and that would be around 200% slower. The read performance however
seems does not change from the XFS only up to the LVM with DRBD.
Could anyone please help me out on how to improve this one or maybe I
have some mistakes that need to be point out especially with my
drbd.conf.
Any questions regarding my methods are greatly appreciated.

thanks,
warren

drbd.conf
resource share1{
protocol C;
incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep 60 ; halt -f";
startup {
wfc-timeout 0;
degr-wfc-timeout 120;
}
disk {
on-io-error detach;
}
net {
sndbuf-size 512k;
timeout 60;
connect-int 10;
ping-int 10;
max-buffers 2048;
max-epoch-size 2048;
ko-count 4;
on-disconnect stand_alone;
}
syncer {
rate 30M;
group 1;
al-extents 257;
}
on server1{
device /dev/drbd4;
disk /dev/VG01200/share1;
address 192.168.99.163:7793;
meta-disk internal;
}
on server2{
device /dev/drbd4;
disk /dev/VG01200/share1;
address 192.168.99.164:7793;
meta-disk internal;
}
} #share1