Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi,
I have 2 Sun xFire 4140 servers, each with 4x 74GB SAS HDD (don't know
the manufacturer by heart). I used the hardware raid controller to
create 2x Raid 1 Partition out of those 4 disks. One of those partitions
is for the OS only and one is for DRBD only.
The Sun Servers are connected over 2x 1 Gigabit Ethernet Cross-over
Cable (bonding interface) so it should be fast enough.
Now I'm seeing huge performance issues with DRBD. I ran some tests
according to this ml.
This is writing to the normal partition, as it should be:
[root at nagios2 drbd]# dd if=/dev/zero of=/var/tmp/4GBtestfile bs=512M
count=8 oflag=direct
8+0 records in
8+0 records out
4294967296 bytes (4.3 GB) copied, 60.6458 seconds, 70.8 MB/s
[root at nagios2 drbd]# dd if=/dev/zero of=/var/tmp/testfile bs=512
count=100000 oflag=direct
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 11.4845 seconds, 4.5 MB/s
Now these values are ok, but if I do the same on the DRBD partition:
[root at nagios2 drbd]# dd if=/dev/zero of=/drbd/4GBtestfile bs=512M
count=8 oflag=direct
8+0 records in
8+0 records out
4294967296 bytes (4.3 GB) copied, 963.691 seconds, 4.5 MB/s
[root at nagios2 drbd]# dd if=/dev/zero of=/drbd/testfile bs=512
count=100000 oflag=direct
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 6441.26 seconds, 7.9 kB/s
If I shut down the 2nd node the io test is much faster! But the write of
the 4GB File is still about the same, if not even slower.
[root at nagios2 drbd]# dd if=/dev/zero of=/drbd/4GBtestfile bs=512M
count=8 oflag=direct
8+0 records in
8+0 records out
4294967296 bytes (4.3 GB) copied, 1164.49 seconds, 3.7 MB/s
[root at nagios2 drbd]# dd if=/dev/zero of=/drbd/testfile bs=512
count=100000 oflag=direct
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 14.4444 seconds, 3.5 MB/s
So the problem seems to be the connection, but as I stated ealier it's
connected via 2x 1 Gigabit Ethernet.
If I copy data over the network using scp for example, it copies at
roughly 40 MB/s!
I honestly have no idea what else I can do.
I'm using Version 8.0.13 since it's the only package I found for RHEL5.
The kernel module is wrong though, so each time I reboot I get the
'kernel tainted' message but according to this list I can safely ignore
this.
If I look at /proc/drbd I have a lot of misses - this can't be good now
can it?
version: 8.0.14 (api:86/proto:86)
GIT-hash: bb447522fc9a87d0069b7e14f0234911ebdab0f7 build by
bachbuilder@, 2008-12-19 22:49:51
0: cs:Connected st:Primary/Secondary ds:UpToDate/UpToDate C r---
ns:50575770 nr:0 dw:49852842 dr:1363161 al:53943 bm:435 lo:0 pe:2
ua:0 ap:1
resync: used:0/61 hits:80290 misses:151 starving:0 dirty:0 changed:151
act_log: used:2/127 hits:11534772 misses:58875 starving:87 dirty:4905
changed:53943
I'm really at a loss here and I'm grateful for any input.
Thanks in advance,
Tobi