Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On 06/10/2010 11:39 AM, Andreas Kurz wrote:
> On Thursday 10 June 2010 17:19:53 Michael Joyner wrote:
>
>> better, but still way below box2box bandwidth.... is fs type or fact
>> using lvm a factor? Here are my test results, will try w/o LVM next.
>>
> I can really recommend "Part V. Optimizing DRBD performance" in the DRBD
> Users Guide ... or invest in a "DRBD Healthcheck" offered by Linbit ;-)
>
> Regards,
> Andreas
>
I split it up into 3 volumes, to prevent split brain issue with LVM. :)
I retest with EXT4, same horrible results.
I then tested with XFS, much, much better results:
Migrated VMs over to it. They all seem to be running much better with
XFS and not EXT4.
As soon as I can get back into the COLO, I will reroute cabling to setup
a RR bond between the two boxes directly to see if that helps.
Thanks for all the help.
=== *** === XFS Timing Test:
Large files test:
rsync box2box /var/tmp:
sent 6.96G bytes received 164 bytes 47.16M bytes/sec
rsync local2/data w/ peer:
sent 6.96G bytes received 164 bytes 41.53M bytes/sec
rsync local2/data no peer:
sent 6.96G bytes received 164 bytes 114.99M bytes/sec
Iozone test (no peer):
real 0m4.983s
real 0m4.922s
real 0m4.920s
real 0m4.915s
real 0m4.917s
Iozone test (w/ peer);
real 0m29.644s
real 0m28.056s
real 0m27.634s
real 0m28.203s
real 0m28.209s
>
>> On 06/10/2010 04:05 AM, Andreas Kurz wrote:
>>
>>> .... LVM on your system supports barriers --> drbd supports barriers -->
>>> barriers are used to enforce write-after-write dependencies per default
>>> ... use drbd config-param 'no-disk-barrier'
>>>
>> === new layout ==============================
>>
>> using /dev/sdb as /dev/drbd0, non-spanned vg.
>>
>> === new drbd.conf ============================
>>
>> global {
>> usage-count yes;
>> }
>> common {
>> protocol C;
>> syncer { rate 700000K; }
>> net {
>> sndbuf-size 0;
>> after-sb-0pri discard-least-changes;
>> after-sb-1pri discard-secondary;
>> }
>> startup {
>> degr-wfc-timeout 30000;
>> }
>> disk {
>> no-disk-flushes;
>> no-md-flushes;
>> no-disk-barrier;
>> }
>>
>> }
>> resource sdb {
>> on san-node-1.ewc.edu {
>> device /dev/drbd0;
>> disk /dev/sdb;
>> address 192.168.75.201:7789;
>> meta-disk internal;
>> }
>> on san-node-2.ewc.edu {
>> device /dev/drbd0;
>> disk /dev/sdb;
>> address 192.168.75.202:7789;
>> meta-disk internal;
>> }
>> }
>>
>> *********
>> === *** mount /dev/vgdrbd0/nfs1 /data *** ===
>>
>> === iozone -a -e -g 4096 with peer down:
>> real 0m1.757s, real 0m1.738s, real 0m1.753s, real
>> 0m1.762s, real 0m1.745s
>>
>> === with peer up:
>> real 0m6.739s, real 0m6.758s, real 0m6.619s, real 0m6.653s,
>> real 0m6.636s
>>
>> === big files test =============================
>>
>> === DRBD === rm -rfv /data/ISOS; rsync -a --verbose --human-readable
>> --progress ~mjoyner/ISOS /data/ISOS
>>
>> building file list ...
>> 8 files to consider
>> created directory /data/ISOS
>> ISOS/
>> ISOS/ISOS/
>> ISOS/ISOS/.lck-5a00b81000000000
>> 84 100% 0.00kB/s 0:00:00 (xfer#1, to-check=5/8)
>> ISOS/ISOS/.lck-6500b81000000000
>> 84 100% 82.03kB/s 0:00:00 (xfer#2, to-check=4/8)
>> ISOS/ISOS/.lck-7700b81000000000
>> 84 100% 82.03kB/s 0:00:00 (xfer#3, to-check=3/8)
>> ISOS/ISOS/SW_DVD5_NTRL_SQL_Svr_2008_SP1_English_X15-51857.ISO
>> 943.07M 100% 189.42MB/s 0:00:04 (xfer#4, to-check=2/8)
>> ISOS/ISOS/SW_DVD5_SQL_Svr_Enterprise_Edtn_2008_English_MLF_X14-89207.ISO
>> 3.26G 100% 189.81MB/s 0:00:16 (xfer#5, to-check=1/8)
>> ISOS/ISOS/SW_DVD5_Windows_Svr_2008w_SP2_English__x64_DC_EE_SE_X15-41371.ISO
>> 2.76G 100% 162.03MB/s 0:00:16 (xfer#6, to-check=0/8)
>>
>> #1) sent 6.96G bytes received 164 bytes 190.60M bytes/sec
>> #2) sent 6.96G bytes received 164 bytes 244.10M bytes/sec
>> #3) sent 6.96G bytes received 164 bytes 244.10M bytes/sec
>>
>> === LOCAL FS === rm -rfv /var/tmp/ISOS; rsync -a --verbose
>> --human-readable --progress ~mjoyner/ISOS /var/tmp/ISOS
>>
>> #1) sent 6.96G bytes received 164 bytes 167.63M bytes/sec
>> #2) sent 6.96G bytes received 164 bytes 185.51M bytes/sec
>> #3) sent 6.96G bytes received 164 bytes 220.85M bytes/sec
>>
>> === time umount /data
>> real 2m11.090s
>> user 0m0.000s
>> sys 1m49.566s
>>
>>
>> *********
>> === *** mount -o sync,data=journal /dev/vgdrbd0/nfs1 /data *** ===
>>
>> === iozone test, no peer
>>
>> real 0m9.483s real 0m9.583s real 0m9.583s real 0m9.559s
>> real 0m9.491s
>>
>> === izone test, w/ peer up.
>>
>> real 0m41.558s real 0m41.386s real 0m41.456s real 0m41.448s
>>
>>
>> === BIG FILES TEST
>> no peer) sent 6.96G bytes received 164 bytes 74.40M bytes/sec
>> w/ peer) sent 6.96G bytes received 164 bytes 29.79M bytes/sec
>> rsync/var/tmp2/var/tmp) sent 6.96G bytes received 164 bytes 47.49M
>> bytes/sec
>>
>> === time umount /data
>>
>> real 0m1.078s
>> user 0m0.000s
>> sys 0m1.075s
>>
>> _______________________________________________
>> drbd-user mailing list
>> drbd-user at lists.linbit.com
>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>
>>
>
--
Michael Joyner ᏩᏯ
System Administrator/Edward Waters College
1658 Kings Road, Jacksonville, FL 32209
904-470-8170 (V) / 904-470-8170 (F)
ᏩᏙ
========================================================================
HOW TO SUBMIT A WORK REQUEST TO INFORMATION TECHNOLOGY
------------------------------------------------------------------------
Via the web: http://otrs.ewc.edu/otrs/customer.pl
Via email: support at ewc.edu
* It is imperative that you submit each problem as a new item.
* Please do NOT combine multiple problem reports in a single workorder.
* PLEASE DO NOT EMAIL SUPPORT STAFF REQUESTS DIRECTLY.
* PLEASE USE THE JOB TRACKING SYSTEM.
========================================================================