Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Jun 20, 2011, at 12:39 AM, Noah Mehl wrote: > On Jun 18, 2011, at 2:27 PM, Florian Haas wrote: > >> On 06/17/2011 05:04 PM, Noah Mehl wrote: >>> Below is the script I ran to do the performance testing. I basically took the script from the user guide and removed the oflag=direct, >> >> ... which means that dd wrote to your page cache (read: RAM). At this >> point, you started kidding yourself about your performance. > > I do have a question here: the total size of the dd write was 64GB, twice the amount of system RAM, does this still apply? > >> >>> because when it was in there, it brought the performance down to 26MB/s (not really my focus here, but maybe related?). >> >> "Related" doesn't begin to describe it. >> >> Rerun the tests with oflag=direct and then repost them. > > Florian, > > I apologize for posting again without seeing your reply. I took the script directly from the user guide: > > #!/bin/bash > TEST_RESOURCE=r0 > TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE) > TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE) > drbdadm primary $TEST_RESOURCE > for i in $(seq 5); do > dd if=/dev/zero of=$TEST_DEVICE bs=512M count=1 oflag=direct > done > drbdadm down $TEST_RESOURCE > for i in $(seq 5); do > dd if=/dev/zero of=$TEST_LL_DEVICE bs=512M count=1 oflag=direct > done > > Here are the results: > > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 0.911252 s, 589 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 1.04314 s, 515 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 1.0674 s, 503 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 1.12087 s, 479 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 1.11023 s, 484 MB/s > 0: State change failed: (-12) Device is held open by someone > Command 'drbdsetup 0 down' terminated with exit code 11 > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 0.627696 s, 855 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 0.631827 s, 850 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 0.602407 s, 891 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 0.688819 s, 779 MB/s > 1+0 records in > 1+0 records out > 536870912 bytes (537 MB) copied, 0.681835 s, 787 MB/s > > Please advise. Thanks. > > ~Noah > > Scanned for viruses and content by the Tranet Spam Sentinel service. > _______________________________________________ > drbd-user mailing list > drbd-user at lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user I changed the script to: #!/bin/bash TEST_RESOURCE=r0 TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE) TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE) drbdadm primary $TEST_RESOURCE for i in $(seq 5); do dd if=/dev/zero of=$TEST_DEVICE bs=1G count=64 oflag=direct done drbdadm down $TEST_RESOURCE for i in $(seq 5); do dd if=/dev/zero of=$TEST_LL_DEVICE bs=1G count=64 oflag=direct done and re-ran: 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 118.145 s, 582 MB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 118.202 s, 581 MB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 119.25 s, 576 MB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 115.24 s, 596 MB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 115.72 s, 594 MB/s 0: State change failed: (-12) Device is held open by someone Command 'drbdsetup 0 down' terminated with exit code 11 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 52.7087 s, 1.3 GB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 52.3662 s, 1.3 GB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 54.5977 s, 1.3 GB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 53.1176 s, 1.3 GB/s 64+0 records in 64+0 records out 68719476736 bytes (69 GB) copied, 56.9376 s, 1.2 GB/s Just to make sure that the writes were more than the system ram. ~Noah