[DRBD-user] Sgisdela! Help! Horrible DRBD performance on CentOS 5. Where do I start looking?

[Michael Joyner]

better, but still way below box2box bandwidth.... is fs type or fact 
using lvm a factor? Here are my test results, will try w/o LVM next.

On 06/10/2010 04:05 AM, Andreas Kurz wrote:
> .... LVM on your system supports barriers --> drbd supports barriers -->
> barriers are used to enforce write-after-write dependencies per default ...
> use drbd config-param 'no-disk-barrier'
>    
=== new layout ==============================

using /dev/sdb as /dev/drbd0, non-spanned vg.

=== new drbd.conf ============================

global {
   usage-count yes;
}
common {
     protocol C;
     syncer { rate 700000K; }
     net {
         sndbuf-size 0;
         after-sb-0pri discard-least-changes;
         after-sb-1pri discard-secondary;
     }
     startup {
         degr-wfc-timeout 30000;
     }
     disk {
         no-disk-flushes;
         no-md-flushes;
         no-disk-barrier;
     }

}
resource sdb {
   on san-node-1.ewc.edu {
     device    /dev/drbd0;
     disk      /dev/sdb;
     address   192.168.75.201:7789;
     meta-disk internal;
   }
   on san-node-2.ewc.edu {
     device    /dev/drbd0;
     disk      /dev/sdb;
     address   192.168.75.202:7789;
     meta-disk internal;
   }
}

*********
=== *** mount /dev/vgdrbd0/nfs1 /data *** ===

=== iozone -a -e -g 4096 with peer down:
real    0m1.757s,  real    0m1.738s, real    0m1.753s, real    
0m1.762s,  real    0m1.745s

=== with peer up:
real    0m6.739s, real    0m6.758s, real    0m6.619s, real    0m6.653s, 
real    0m6.636s

=== big files test =============================

===  DRBD === rm -rfv /data/ISOS; rsync -a --verbose --human-readable 
--progress ~mjoyner/ISOS /data/ISOS

building file list ...
8 files to consider
created directory /data/ISOS
ISOS/
ISOS/ISOS/
ISOS/ISOS/.lck-5a00b81000000000
           84 100%    0.00kB/s    0:00:00 (xfer#1, to-check=5/8)
ISOS/ISOS/.lck-6500b81000000000
           84 100%   82.03kB/s    0:00:00 (xfer#2, to-check=4/8)
ISOS/ISOS/.lck-7700b81000000000
           84 100%   82.03kB/s    0:00:00 (xfer#3, to-check=3/8)
ISOS/ISOS/SW_DVD5_NTRL_SQL_Svr_2008_SP1_English_X15-51857.ISO
      943.07M 100%  189.42MB/s    0:00:04 (xfer#4, to-check=2/8)
ISOS/ISOS/SW_DVD5_SQL_Svr_Enterprise_Edtn_2008_English_MLF_X14-89207.ISO
        3.26G 100%  189.81MB/s    0:00:16 (xfer#5, to-check=1/8)
ISOS/ISOS/SW_DVD5_Windows_Svr_2008w_SP2_English__x64_DC_EE_SE_X15-41371.ISO
        2.76G 100%  162.03MB/s    0:00:16 (xfer#6, to-check=0/8)

#1) sent 6.96G bytes  received 164 bytes  190.60M bytes/sec
#2) sent 6.96G bytes  received 164 bytes  244.10M bytes/sec
#3) sent 6.96G bytes  received 164 bytes  244.10M bytes/sec

=== LOCAL FS ===  rm -rfv /var/tmp/ISOS; rsync -a --verbose 
--human-readable --progress ~mjoyner/ISOS /var/tmp/ISOS

#1) sent 6.96G bytes  received 164 bytes  167.63M bytes/sec
#2) sent 6.96G bytes  received 164 bytes  185.51M bytes/sec
#3) sent 6.96G bytes  received 164 bytes  220.85M bytes/sec

=== time umount /data
real    2m11.090s
user    0m0.000s
sys    1m49.566s


*********
=== *** mount -o sync,data=journal /dev/vgdrbd0/nfs1 /data *** ===

=== iozone test, no peer

real    0m9.483s real    0m9.583s real    0m9.583s real    0m9.559s 
real    0m9.491s

=== izone test, w/ peer up.

real    0m41.558s real    0m41.386s real    0m41.456s real    0m41.448s


=== BIG FILES TEST
no peer) sent 6.96G bytes  received 164 bytes  74.40M bytes/sec
w/ peer) sent 6.96G bytes  received 164 bytes  29.79M bytes/sec
rsync/var/tmp2/var/tmp) sent 6.96G bytes  received 164 bytes  47.49M 
bytes/sec

=== time umount /data

real    0m1.078s
user    0m0.000s
sys    0m1.075s