Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Two node setup for serving out NFS to vSphere.
timing test is : sync; time iozone -a -e -g 4096
/data/nfs is on LVM striped across 3 DRBD devices. EXT4. data=journal.
/var/tmp is local filesystem.
both are on same set of disk platters and controller. Raid 6. 64K blocksize.
FYI, the initial sync (5 TB) used a full gigabit of bandwidth without
hesitation.
=== DRBD TIMES (both nodes up) ==============================
real 0m10.028s
user 0m0.116s
sys 0m1.538s
real 0m10.045s
user 0m0.085s
sys 0m1.541s
real 0m9.990s
user 0m0.091s
sys 0m1.578s
real 0m9.970s
user 0m0.099s
sys 0m1.557s
real 0m9.960s
user 0m0.091s
sys 0m1.499s
=== DRBD TIMES (2nd node down) ==============================
real 0m3.754s
user 0m0.093s
sys 0m1.070s
real 0m3.855s
user 0m0.079s
sys 0m1.064s
real 0m3.938s
user 0m0.094s
sys 0m1.044s
real 0m3.809s
user 0m0.066s
sys 0m1.067s
real 0m3.863s
user 0m0.069s
sys 0m1.072s
=== LOCAL TIMES ==============================
real 0m1.770s
user 0m0.070s
sys 0m0.977s
real 0m1.809s
user 0m0.085s
sys 0m0.974s
real 0m1.737s
user 0m0.067s
sys 0m0.942s
real 0m2.007s
user 0m0.058s
sys 0m0.955s
real 0m1.808s
user 0m0.072s
sys 0m0.956s
=== After re-enabling 2nd node ==============================
root at san-node-2 ~]# cat /proc/drbd
version: 8.3.2 (api:88/proto:86-90)
GIT-hash: dd7985327f146f33b86d4bff5ca8c94234ce840e build by
mockbuild at v20z-x86-64.home.local, 2009-08-29 14:07:55
0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----
ns:0 nr:5180 dw:5180 dr:0 al:0 bm:24 lo:1 pe:1389 ua:0 ap:0 ep:1 wo:b
oos:43964
[==>.................] sync'ed: 16.7% (43964/49144)K
finish: 0:00:08 speed: 5,180 (5,180) K/sec
1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----
ns:0 nr:49044 dw:49044 dr:0 al:0 bm:44 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b
oos:0
2: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----
ns:0 nr:6896 dw:6896 dr:0 al:0 bm:28 lo:1 pe:1346 ua:0 ap:0 ep:1 wo:b
oos:42436
[===>................] sync'ed: 23.1% (42436/49332)K
finish: 0:00:05 speed: 6,896 (6,896) K/sec
=== ifconfig ==============================
bond0:0 Link encap:Ethernet HWaddr 00:1B:21:26:B8:18
inet addr:192.168.XXX.XX0 Bcast:192.168.XXX.255
Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:9000 Metric:1
=== rpm -qa|grep drbd ==============================
kmod-drbd83-8.3.2-6.el5_3
drbd83-8.3.2-6.el5_3
=== uname -a ==============================
Linux san-node-1.ewc.edu 2.6.18-194.3.1.el5 #1 SMP Thu May 13 13:08:30 EDT
2010 x86_64 x86_64 x86_64 GNU/Linux
=== cat /etc/issue ==============================
CentOS release 5.5 (Final)
Kernel \r on an \m
=== lspci ==============================
01:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 1078
(rev 04)
=== box to box copy test ==============================
dd if=/dev/sda of=testfile bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.94192 seconds, 553 MB/s
scp testfile root at san-node-2:/var/tmp/testfile
testfile 100% 1024MB
46.6MB/s 00:22
=== /etc/drbd.conf ==============================
global {
usage-count yes;
}
common {
protocol C;
syncer { rate 700000K; }
net {
sndbuf-size 0;
after-sb-0pri discard-least-changes;
after-sb-1pri discard-secondary;
}
startup {
degr-wfc-timeout 30000;
}
disk {
no-disk-flushes;
no-md-flushes;
}
}
resource sdb {
on san-node-1.ewc.edu {
device /dev/drbd0;
disk /dev/sdb;
address 192.168.XXX.XX1:7789;
meta-disk internal;
}
on san-node-2.ewc.edu {
device /dev/drbd0;
disk /dev/sdb;
address 192.168.XXX.XX2:7789;
meta-disk internal;
}
}
resource sdc {
on san-node-1.ewc.edu {
device /dev/drbd1;
disk /dev/sdc;
address 192.168.XXX.XX1:7790;
meta-disk internal;
}
on san-node-2.ewc.edu {
device /dev/drbd1;
disk /dev/sdc;
address 192.168.XXX.XX2:7790;
meta-disk internal;
}
}
resource sdd {
on san-node-1.ewc.edu {
device /dev/drbd2;
disk /dev/sdd;
address 192.168.XXX.XX1:7791;
meta-disk internal;
}
on san-node-2.ewc.edu {
device /dev/drbd2;
disk /dev/sdd;
address 192.168.XXX.XX2:7791;
meta-disk internal;
}
}
--
Michael Joyner/System Administrator
Edward Waters College
1658 Kings Road
Jacksonville, FL 32209
(V) 904-470-8170/(F) 904-470-8170
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20100609/3829fd51/attachment.htm>