Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello again,
On Thursday 10 June 2010 02:32:18 Michael Joyner wrote:
> Two node setup for serving out NFS to vSphere.
>
> timing test is : sync; time iozone -a -e -g 4096
>
> /data/nfs is on LVM striped across 3 DRBD devices. EXT4. data=journal.
> /var/tmp is local filesystem.
... don't do this! There is no guarantee all 3 DRBD devices are up-to-date at
the same time ....
Regards,
Andreas
>
> both are on same set of disk platters and controller. Raid 6. 64K
> blocksize.
>
> FYI, the initial sync (5 TB) used a full gigabit of bandwidth without
> hesitation.
>
> === DRBD TIMES (both nodes up) ==============================
> real 0m10.028s
> user 0m0.116s
> sys 0m1.538s
>
> real 0m10.045s
> user 0m0.085s
> sys 0m1.541s
>
> real 0m9.990s
> user 0m0.091s
> sys 0m1.578s
>
> real 0m9.970s
> user 0m0.099s
> sys 0m1.557s
>
> real 0m9.960s
> user 0m0.091s
> sys 0m1.499s
>
> === DRBD TIMES (2nd node down) ==============================
> real 0m3.754s
> user 0m0.093s
> sys 0m1.070s
>
> real 0m3.855s
> user 0m0.079s
> sys 0m1.064s
>
> real 0m3.938s
> user 0m0.094s
> sys 0m1.044s
>
> real 0m3.809s
> user 0m0.066s
> sys 0m1.067s
>
> real 0m3.863s
> user 0m0.069s
> sys 0m1.072s
>
> === LOCAL TIMES ==============================
> real 0m1.770s
> user 0m0.070s
> sys 0m0.977s
>
> real 0m1.809s
> user 0m0.085s
> sys 0m0.974s
>
> real 0m1.737s
> user 0m0.067s
> sys 0m0.942s
>
> real 0m2.007s
> user 0m0.058s
> sys 0m0.955s
>
> real 0m1.808s
> user 0m0.072s
> sys 0m0.956s
>
> === After re-enabling 2nd node ==============================
>
> root at san-node-2 ~]# cat /proc/drbd
> version: 8.3.2 (api:88/proto:86-90)
> GIT-hash: dd7985327f146f33b86d4bff5ca8c94234ce840e build by
> mockbuild at v20z-x86-64.home.local, 2009-08-29 14:07:55
> 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----
> ns:0 nr:5180 dw:5180 dr:0 al:0 bm:24 lo:1 pe:1389 ua:0 ap:0 ep:1 wo:b
> oos:43964
> [==>.................] sync'ed: 16.7% (43964/49144)K
> finish: 0:00:08 speed: 5,180 (5,180) K/sec
> 1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----
> ns:0 nr:49044 dw:49044 dr:0 al:0 bm:44 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b
> oos:0
> 2: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----
> ns:0 nr:6896 dw:6896 dr:0 al:0 bm:28 lo:1 pe:1346 ua:0 ap:0 ep:1 wo:b
> oos:42436
> [===>................] sync'ed: 23.1% (42436/49332)K
> finish: 0:00:05 speed: 6,896 (6,896) K/sec
>
> === ifconfig ==============================
>
> bond0:0 Link encap:Ethernet HWaddr 00:1B:21:26:B8:18
> inet addr:192.168.XXX.XX0 Bcast:192.168.XXX.255
> Mask:255.255.255.0
> UP BROADCAST RUNNING MASTER MULTICAST MTU:9000 Metric:1
>
>
> === rpm -qa|grep drbd ==============================
> kmod-drbd83-8.3.2-6.el5_3
> drbd83-8.3.2-6.el5_3
>
> === uname -a ==============================
> Linux san-node-1.ewc.edu 2.6.18-194.3.1.el5 #1 SMP Thu May 13 13:08:30 EDT
> 2010 x86_64 x86_64 x86_64 GNU/Linux
>
> === cat /etc/issue ==============================
> CentOS release 5.5 (Final)
> Kernel \r on an \m
>
> === lspci ==============================
> 01:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 1078
> (rev 04)
>
> === box to box copy test ==============================
> dd if=/dev/sda of=testfile bs=1M count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 1.94192 seconds, 553 MB/s
>
> scp testfile root at san-node-2:/var/tmp/testfile
> testfile 100% 1024MB
> 46.6MB/s 00:22
>
> === /etc/drbd.conf ==============================
>
> global {
> usage-count yes;
> }
> common {
> protocol C;
> syncer { rate 700000K; }
> net {
> sndbuf-size 0;
> after-sb-0pri discard-least-changes;
> after-sb-1pri discard-secondary;
> }
> startup {
> degr-wfc-timeout 30000;
> }
> disk {
> no-disk-flushes;
> no-md-flushes;
> }
>
> }
> resource sdb {
> on san-node-1.ewc.edu {
> device /dev/drbd0;
> disk /dev/sdb;
> address 192.168.XXX.XX1:7789;
> meta-disk internal;
> }
> on san-node-2.ewc.edu {
> device /dev/drbd0;
> disk /dev/sdb;
> address 192.168.XXX.XX2:7789;
> meta-disk internal;
> }
> }
> resource sdc {
> on san-node-1.ewc.edu {
> device /dev/drbd1;
> disk /dev/sdc;
> address 192.168.XXX.XX1:7790;
> meta-disk internal;
> }
> on san-node-2.ewc.edu {
> device /dev/drbd1;
> disk /dev/sdc;
> address 192.168.XXX.XX2:7790;
> meta-disk internal;
> }
> }
> resource sdd {
> on san-node-1.ewc.edu {
> device /dev/drbd2;
> disk /dev/sdd;
> address 192.168.XXX.XX1:7791;
> meta-disk internal;
> }
> on san-node-2.ewc.edu {
> device /dev/drbd2;
> disk /dev/sdd;
> address 192.168.XXX.XX2:7791;
> meta-disk internal;
> }
> }
>
--
: Andreas Kurz
: LINBIT | Your Way to High Availability
: Tel +43-1-8178292-64, Fax +43-1-8178292-82
:
: http://www.linbit.com
------------------------------------------------------------------------
DRBD - You create, we replicate.
Find out more at http://www.linbit.com/
------------------------------------------------------------------------
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
This e-mail is solely for use by the intended recipient(s). Information
contained in this e-mail and its attachments may be confidential,
privileged or copyrighted. If you are not the intended recipient you are
hereby formally notified that any use, copying, disclosure or
distribution of the contents of this e-mail, in whole or in part, is
prohibited. Also please notify immediately the sender by return e-mail
and delete this e-mail from your system. Thank you for your co-operation.