[DRBD-user] Poor DRBD performance, HELP!

Noah Mehl noah at tritonlimited.com
Mon Jun 20 07:16:49 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On Jun 20, 2011, at 12:39 AM, Noah Mehl wrote:

> On Jun 18, 2011, at 2:27 PM, Florian Haas wrote:
> 
>> On 06/17/2011 05:04 PM, Noah Mehl wrote:
>>> Below is the script I ran to do the performance testing.  I basically took the script from the user guide and removed the oflag=direct,
>> 
>> ... which means that dd wrote to your page cache (read: RAM). At this
>> point, you started kidding yourself about your performance.
> 
> I do have a question here:  the total size of the dd write was 64GB, twice the amount of system RAM, does this still apply?
> 
>> 
>>> because when it was in there, it brought the performance down to 26MB/s (not really my focus here, but maybe related?).
>> 
>> "Related" doesn't begin to describe it.
>> 
>> Rerun the tests with oflag=direct and then repost them.
> 
> Florian,
> 
> I apologize for posting again without seeing your reply.  I took the script directly from the user guide:
> 
> #!/bin/bash
> TEST_RESOURCE=r0
> TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE)
> TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE)
> drbdadm primary $TEST_RESOURCE
> for i in $(seq 5); do
>  dd if=/dev/zero of=$TEST_DEVICE bs=512M count=1 oflag=direct
> done
> drbdadm down $TEST_RESOURCE
> for i in $(seq 5); do
>  dd if=/dev/zero of=$TEST_LL_DEVICE bs=512M count=1 oflag=direct
> done
> 
> Here are the results:
> 
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 0.911252 s, 589 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 1.04314 s, 515 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 1.0674 s, 503 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 1.12087 s, 479 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 1.11023 s, 484 MB/s
> 0: State change failed: (-12) Device is held open by someone
> Command 'drbdsetup 0 down' terminated with exit code 11
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 0.627696 s, 855 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 0.631827 s, 850 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 0.602407 s, 891 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 0.688819 s, 779 MB/s
> 1+0 records in
> 1+0 records out
> 536870912 bytes (537 MB) copied, 0.681835 s, 787 MB/s
> 
> Please advise.  Thanks.
> 
> ~Noah
> 
> Scanned for viruses and content by the Tranet Spam Sentinel service.
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user


I changed the script to:

#!/bin/bash
TEST_RESOURCE=r0
TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE)
TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE)
drbdadm primary $TEST_RESOURCE
for i in $(seq 5); do
  dd if=/dev/zero of=$TEST_DEVICE bs=1G count=64 oflag=direct
done
drbdadm down $TEST_RESOURCE
for i in $(seq 5); do
  dd if=/dev/zero of=$TEST_LL_DEVICE bs=1G count=64 oflag=direct
done

and re-ran:

64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 118.145 s, 582 MB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 118.202 s, 581 MB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 119.25 s, 576 MB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 115.24 s, 596 MB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 115.72 s, 594 MB/s
0: State change failed: (-12) Device is held open by someone
Command 'drbdsetup 0 down' terminated with exit code 11
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 52.7087 s, 1.3 GB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 52.3662 s, 1.3 GB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 54.5977 s, 1.3 GB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 53.1176 s, 1.3 GB/s
64+0 records in
64+0 records out
68719476736 bytes (69 GB) copied, 56.9376 s, 1.2 GB/s

Just to make sure that the writes were more than the system ram.

~Noah


More information about the drbd-user mailing list