Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello All
I'm facing a strange problem after the reinstallation of our storage cluster. We upgraded the raid controller and the interconnect to 10GbE. The Storage is very fast now:
dd if=/dev/zero of=/dev/sdb1 bs=512M count=12 oflag=direct
12+0 Datensätze ein
12+0 Datensätze aus
6442450944 Bytes (6,4 GB) kopiert, 10,5372 s, 611 MB/s
As also the 10GbE interconnect:
netperf -H 10.0.2.1,ipv4
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.2.1 (10.0.2.1) port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 65536 65536 10.04 9335.75
Only the drbd is very slow. (Single node, just to test write perfomance on local block device.
dd if=/dev/zero of=/dev/drbd0 bs=512M count=8
4+0 Datensätze ein
4+0 Datensätze aus
2147483648 Bytes (2,1 GB) kopiert, 63,9839 s, 33,6 MB/s
Cancelled with CTRL+C
After long googleing and testing around with io scheduler nr_request etc. blockdev, I cannot see where the bottlenec lies. Had anybody mad similar experiences and can give me a hint. OS is Scientific Linux-6.4 x64 (CentOS) with drbd 8.4.1.
Regards
Roland
Here my drbd.conf
global {
usage-count no;
}
common {
protocol B;
}
resource target1 {
meta-disk internal;
device /dev/drbd0;
syncer {
verify-alg sha1;
cpu-mask 0;
rate 850M;
al-extents 6433;
}
net {
max-buffers 16384;
max-epoch-size 4096;
cram-hmac-alg sha1;
unplug-watermark 8192;
no-tcp-cork;
sndbuf-size 4096k;
rcvbuf-size 4096k;
}
startup {
wfc-timeout 10;
degr-wfc-timeout 8;
}
disk {
resync-rate 4000M;
disk-flushes no;
disk-barrier no;
md-flushes no;
read-balancing 64K-striping;
}
on storage-node1.ziilrz.ch {
disk /dev/sdb1;
address 10.0.1.1:7789;
}
on storage-node2.ziilrz.ch {
disk /dev/sdb1;
address 10.0.1.2:7789;
}
}
resource target2 {
meta-disk internal;
device /dev/drbd1;
syncer {
verify-alg sha1;
cpu-mask 0;
rate 850M;
al-extents 6433;
}
net {
max-buffers 16384;
max-epoch-size 4096;
cram-hmac-alg sha1;
unplug-watermark 8192;
no-tcp-cork;
sndbuf-size 4096k;
rcvbuf-size 4096k;
}
startup {
wfc-timeout 10;
degr-wfc-timeout 8;
}
disk {
disk-flushes no;
disk-barrier no;
md-flushes no;
read-balancing 64K-striping;
resync-rate 4000M;
}
on storage-node1.ziilrz.ch {
disk /dev/sdb2;
address 10.0.2.1:7789;
}
on storage-node2.ziilrz.ch {
disk /dev/sdb2;
address 10.0.2.2:7789;
}
}
--
Ziil Informatiklösungen GmbH
Roland Käser
Leiter Technik
Romanshornerstr. 134,
CH-8280 Kreuzlingen
Tel: +41 71 671 27 78
Fax: +41 71 671 27 79
www.ziil.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20120926/38187f2a/attachment.htm>