[DRBD-user] Sgisdela! Help! Horrible DRBD performance on CentOS 5. Where do I start looking?

Andreas Kurz andreas.kurz at linbit.com
Thu Jun 10 17:39:44 CEST 2010

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On Thursday 10 June 2010 17:19:53 Michael Joyner wrote:
> better, but still way below box2box bandwidth.... is fs type or fact
> using lvm a factor? Here are my test results, will try w/o LVM next.

I can really recommend  "Part V. Optimizing DRBD performance" in the DRBD 
Users Guide ... or invest in a "DRBD Healthcheck" offered by Linbit ;-)

Regards,
Andreas 

> 
> On 06/10/2010 04:05 AM, Andreas Kurz wrote:
> > .... LVM on your system supports barriers --> drbd supports barriers -->
> > barriers are used to enforce write-after-write dependencies per default
> > ... use drbd config-param 'no-disk-barrier'
> 
> === new layout ==============================
> 
> using /dev/sdb as /dev/drbd0, non-spanned vg.
> 
> === new drbd.conf ============================
> 
> global {
>    usage-count yes;
> }
> common {
>      protocol C;
>      syncer { rate 700000K; }
>      net {
>          sndbuf-size 0;
>          after-sb-0pri discard-least-changes;
>          after-sb-1pri discard-secondary;
>      }
>      startup {
>          degr-wfc-timeout 30000;
>      }
>      disk {
>          no-disk-flushes;
>          no-md-flushes;
>          no-disk-barrier;
>      }
> 
> }
> resource sdb {
>    on san-node-1.ewc.edu {
>      device    /dev/drbd0;
>      disk      /dev/sdb;
>      address   192.168.75.201:7789;
>      meta-disk internal;
>    }
>    on san-node-2.ewc.edu {
>      device    /dev/drbd0;
>      disk      /dev/sdb;
>      address   192.168.75.202:7789;
>      meta-disk internal;
>    }
> }
> 
> *********
> === *** mount /dev/vgdrbd0/nfs1 /data *** ===
> 
> === iozone -a -e -g 4096 with peer down:
> real    0m1.757s,  real    0m1.738s, real    0m1.753s, real
> 0m1.762s,  real    0m1.745s
> 
> === with peer up:
> real    0m6.739s, real    0m6.758s, real    0m6.619s, real    0m6.653s,
> real    0m6.636s
> 
> === big files test =============================
> 
> ===  DRBD === rm -rfv /data/ISOS; rsync -a --verbose --human-readable
> --progress ~mjoyner/ISOS /data/ISOS
> 
> building file list ...
> 8 files to consider
> created directory /data/ISOS
> ISOS/
> ISOS/ISOS/
> ISOS/ISOS/.lck-5a00b81000000000
>            84 100%    0.00kB/s    0:00:00 (xfer#1, to-check=5/8)
> ISOS/ISOS/.lck-6500b81000000000
>            84 100%   82.03kB/s    0:00:00 (xfer#2, to-check=4/8)
> ISOS/ISOS/.lck-7700b81000000000
>            84 100%   82.03kB/s    0:00:00 (xfer#3, to-check=3/8)
> ISOS/ISOS/SW_DVD5_NTRL_SQL_Svr_2008_SP1_English_X15-51857.ISO
>       943.07M 100%  189.42MB/s    0:00:04 (xfer#4, to-check=2/8)
> ISOS/ISOS/SW_DVD5_SQL_Svr_Enterprise_Edtn_2008_English_MLF_X14-89207.ISO
>         3.26G 100%  189.81MB/s    0:00:16 (xfer#5, to-check=1/8)
> ISOS/ISOS/SW_DVD5_Windows_Svr_2008w_SP2_English__x64_DC_EE_SE_X15-41371.ISO
>         2.76G 100%  162.03MB/s    0:00:16 (xfer#6, to-check=0/8)
> 
> #1) sent 6.96G bytes  received 164 bytes  190.60M bytes/sec
> #2) sent 6.96G bytes  received 164 bytes  244.10M bytes/sec
> #3) sent 6.96G bytes  received 164 bytes  244.10M bytes/sec
> 
> === LOCAL FS ===  rm -rfv /var/tmp/ISOS; rsync -a --verbose
> --human-readable --progress ~mjoyner/ISOS /var/tmp/ISOS
> 
> #1) sent 6.96G bytes  received 164 bytes  167.63M bytes/sec
> #2) sent 6.96G bytes  received 164 bytes  185.51M bytes/sec
> #3) sent 6.96G bytes  received 164 bytes  220.85M bytes/sec
> 
> === time umount /data
> real    2m11.090s
> user    0m0.000s
> sys    1m49.566s
> 
> 
> *********
> === *** mount -o sync,data=journal /dev/vgdrbd0/nfs1 /data *** ===
> 
> === iozone test, no peer
> 
> real    0m9.483s real    0m9.583s real    0m9.583s real    0m9.559s
> real    0m9.491s
> 
> === izone test, w/ peer up.
> 
> real    0m41.558s real    0m41.386s real    0m41.456s real    0m41.448s
> 
> 
> === BIG FILES TEST
> no peer) sent 6.96G bytes  received 164 bytes  74.40M bytes/sec
> w/ peer) sent 6.96G bytes  received 164 bytes  29.79M bytes/sec
> rsync/var/tmp2/var/tmp) sent 6.96G bytes  received 164 bytes  47.49M
> bytes/sec
> 
> === time umount /data
> 
> real    0m1.078s
> user    0m0.000s
> sys    0m1.075s
> 
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
> 

-- 
: Andreas Kurz					 
: LINBIT | Your Way to High Availability
: Tel +43-1-8178292-64, Fax +43-1-8178292-82
:
: http://www.linbit.com

------------------------------------------------------------------------
LINBIT - We're the HA experts that other experts ask for help!
http://www.linbit.com/en/training/
------------------------------------------------------------------------

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.

This e-mail is solely for use by the intended recipient(s). Information
contained in this e-mail and its attachments may be confidential,
privileged or copyrighted. If you are not the intended recipient you are
hereby formally notified that any use, copying, disclosure or
distribution of the contents of this e-mail, in whole or in part, is
prohibited. Also please notify immediately the sender by return e-mail
and delete this e-mail from your system. Thank you for your co-operation.



More information about the drbd-user mailing list