[DRBD-user] Poor DRBD performance, HELP!

[Noah Mehl]

On Jun 18, 2011, at 2:27 PM, Florian Haas wrote:

> On 06/17/2011 05:04 PM, Noah Mehl wrote:
>> Below is the script I ran to do the performance testing.  I basically took the script from the user guide and removed the oflag=direct,
> 
> ... which means that dd wrote to your page cache (read: RAM). At this
> point, you started kidding yourself about your performance.

I do have a question here:  the total size of the dd write was 64GB, twice the amount of system RAM, does this still apply?

> 
>> because when it was in there, it brought the performance down to 26MB/s (not really my focus here, but maybe related?).
> 
> "Related" doesn't begin to describe it.
> 
> Rerun the tests with oflag=direct and then repost them.

Florian,

I apologize for posting again without seeing your reply.  I took the script directly from the user guide:

#!/bin/bash
TEST_RESOURCE=r0
TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE)
TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE)
drbdadm primary $TEST_RESOURCE
for i in $(seq 5); do
  dd if=/dev/zero of=$TEST_DEVICE bs=512M count=1 oflag=direct
done
drbdadm down $TEST_RESOURCE
for i in $(seq 5); do
  dd if=/dev/zero of=$TEST_LL_DEVICE bs=512M count=1 oflag=direct
done

Here are the results:

1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 0.911252 s, 589 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 1.04314 s, 515 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 1.0674 s, 503 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 1.12087 s, 479 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 1.11023 s, 484 MB/s
0: State change failed: (-12) Device is held open by someone
Command 'drbdsetup 0 down' terminated with exit code 11
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 0.627696 s, 855 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 0.631827 s, 850 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 0.602407 s, 891 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 0.688819 s, 779 MB/s
1+0 records in
1+0 records out
536870912 bytes (537 MB) copied, 0.681835 s, 787 MB/s

Please advise.  Thanks.

~Noah

Scanned for viruses and content by the Tranet Spam Sentinel service.