<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=utf-8">
<STYLE>
BLOCKQUOTE {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px; MARGIN-LEFT: 2em
}
OL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
UL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
DIV.FoxDiv20120426093304547828 {
COLOR: #000000
}
P {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
BODY {
FONT-SIZE: 10.5pt; COLOR: #000080; LINE-HEIGHT: 1.5; FONT-FAMILY: 宋体
}
</STYLE>
<META content="MSHTML 6.00.2900.6197" name=GENERATOR></HEAD>
<BODY style="MARGIN: 10px" text=#000000 bgColor=#ffffff>
<DIV>yes, the network is ok. I used netperf to test the network.</DIV>
<DIV> </DIV>
<DIV>
<DIV># netperf -H 192.168.100.231 -L 30</DIV>
<DIV>TCP STREAM TEST from 30 (0.0.0.30) port 0 AF_INET to 192.168.100.231 (192.168.100.231) port 0 AF_INET</DIV>
<DIV>Recv Send Send</DIV>
<DIV>Socket Socket Message Elapsed</DIV>
<DIV>Size Size Size Time Throughput</DIV>
<DIV>bytes bytes bytes secs. 10^6bits/sec</DIV>
<DIV> </DIV>
<DIV> 87380 16384 16384 10.03 941.37</DIV></DIV>
<DIV> </DIV>
<DIV>the network is ok.</DIV>
<DIV> </DIV>
<DIV>then enable 'jumbo frames' on, set both machine MTU = 9000, but there
is no difference. </DIV>
<DIV>the dd result is still about 40 MBps.</DIV>
<DIV> </DIV>
<HR style="WIDTH: 210px; HEIGHT: 1px" align=left color=#b5c4df SIZE=1>
<DIV> </DIV>
<DIV
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0cm; BORDER-TOP: #b5c4df 1pt solid; PADDING-LEFT: 0cm; PADDING-BOTTOM: 0cm; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<DIV
style="PADDING-RIGHT: 8px; PADDING-LEFT: 8px; FONT-SIZE: 12px; BACKGROUND: #efefef; PADDING-BOTTOM: 8px; COLOR: #000000; PADDING-TOP: 8px">
<DIV><B>From:</B> <A href="mailto:themsley@voiceflex.com">Trevor
Hemsley</A></DIV>
<DIV><B>Date:</B> 2012-04-25 23:13</DIV>
<DIV><B>To:</B> </DIV>
<DIV><B>CC:</B> <A
href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</A></DIV>
<DIV><B>Subject:</B> Re: [DRBD-user] drbd write performance slow, per disk
40M/s by dd command</DIV></DIV></DIV>
<DIV>
<DIV class=FoxDiv20120426093304547828>Did you test your network connection to
make sure that it can transfer at a greater speed than that? Maybe it is the
bottleneck - jumbo frames on?<BR><BR>On 25/04/12 15:40, Chris Dickson wrote:
<BLOCKQUOTE
cite=mid:CANSMP-5VwiAOQhfg_0VhQ7dxuFgfQb5F8JkdSghtLfiqJqkGGA@mail.gmail.com
type="cite">
<DIV class=gmail_extra>Also use oflag=direct in both tests and perform them a
few times, sometimes high speeds are the result of caching.<BR><BR>
<DIV class=gmail_quote>On Wed, Apr 25, 2012 at 10:35 AM, Chris Dickson <SPAN
dir=ltr><<A href="mailto:chrisd1100@gmail.com" target=_blank
moz-do-not-send="true">chrisd1100@gmail.com</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV class=gmail_extra>Try turning off disk-barrier and disk-flushes and see
if that makes a difference.
<DIV>
<DIV class=h5><BR><BR>
<DIV class=gmail_quote>2012/4/25 feng zheng <SPAN dir=ltr><<A
href="mailto:zf5984599@gmail.com" target=_blank
moz-do-not-send="true">zf5984599@gmail.com</A>></SPAN><BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV>
<DIV>hi, dear all:<BR><BR>When I use drbd, I found the write performance
very slow, against<BR>testing without drbd module.<BR><BR>1. the
environment:<BR>-) CentOS 5.6<BR>-) 2.6.18 kernel<BR>-) drbd 8.4.1<BR>-)
drbd.conf：<BR>resource r0<BR>{<BR> protocol B;<BR><BR>
net<BR> {<BR>max-buffers 8000;<BR>max-epoch-size
8000;<BR>sndbuf-size 512K;<BR> }<BR><BR>
disk<BR> {<BR> al-extents 3389;<BR>
}<BR><BR><BR> on OSS211<BR> {<BR>
device /dev/drbd0;<BR> disk
/dev/sdb1;<BR> address <A
href="http://192.168.100.231:7788" target=_blank
moz-do-not-send="true"><FONT color=red><B>MailScanner warning: numerical
links are often malicious:</B></FONT> 192.168.100.231:7788</A>;<BR>
meta-disk internal;<BR> }<BR><BR>
on OSS213<BR> {<BR> device
/dev/drbd0;<BR> disk /dev/sde1;<BR>
address <A href="http://192.168.100.213:7788" target=_blank
moz-do-not-send="true"><FONT color=red><B>MailScanner warning: numerical
links are often malicious:</B></FONT> 192.168.100.213:7788</A>;<BR>
meta-disk internal;<BR>
}<BR><BR>}<BR><BR><BR>2. Test scenario:<BR>*) without drbd
module,<BR>dd to write 1G stream into one disk, which formatted to
ext3:<BR>[para]# !echo<BR>echo 3 > /proc/sys/vm/drop_caches<BR>[para]#
!dd<BR>dd if=/dev/zero of=test1 bs=1M count=1000 conv=fdatasync<BR>1000+0
records in<BR>1000+0 records out<BR>1048576000 bytes (1.0 GB) copied,
10.9905 seconds, 95.4 MB/s<BR><BR>*) with drbd module,<BR>dd 1G stream to
the disk, which is ext3 too:<BR>[para]# cat /proc/drbd<BR>version: 8.4.1
(api:1/proto:86-100)<BR>GIT-hash: 91b4c048c1a0e06777b5f65d312b38d47abaea80
build by<BR>root@OSS213, 2012-04-16 21:38:36<BR> 0: cs:Connected
ro:Primary/Secondary ds:UpToDate/UpToDate B r-----<BR>
ns:1260036 nr:0 dw:1260036 dr:297 al:330 bm:0 lo:0 pe:0 ua:0
ap:0<BR>ep:1 wo:b oos:0<BR>[para]#<BR>[para]# dd if=/dev/zero of=test1
bs=1M count=1000 conv=fdatasync<BR>1000+0 records in<BR>1000+0 records
out<BR>1048576000 bytes (1.0 GB) copied, 26.7392 seconds, 39.2
MB/s<BR>[para]# cat /proc/drbd<BR>version: 8.4.1
(api:1/proto:86-100)<BR>GIT-hash: 91b4c048c1a0e06777b5f65d312b38d47abaea80
build by<BR>root@OSS213, 2012-04-16 21:38:36<BR> 0: cs:Connected
ro:Primary/Secondary ds:UpToDate/UpToDate B r-----<BR><BR>All the upper
test writing disk are same. From the upper result, if i<BR>use DRBD to
test, the performance<BR>is 39 MB/s; while if i do not use, the
performance is about 95M/s.<BR><BR>3. My question is:<BR>-) this write
performance decays so large is normal or not?<BR>I had read the following
from the DRBD website:<BR>"15.1. Hardware considerations:<BR>.... A
single, reasonably recent, SCSI or SAS disk will<BR>typically allow
streaming writes of roughly 40MB/s to the single disk."<BR>But this is
very slow.<BR><BR>-）if this is not normal, how can i turn this? is the
config file<BR>something not correct?<BR><BR>thanks a
lot<BR>BRs,<BR>feng<BR></DIV></DIV>_______________________________________________<BR>drbd-user
mailing list<BR><A href="mailto:drbd-user@lists.linbit.com" target=_blank
moz-do-not-send="true">drbd-user@lists.linbit.com</A><BR><A
href="http://lists.linbit.com/mailman/listinfo/drbd-user" target=_blank
moz-do-not-send="true">http://lists.linbit.com/mailman/listinfo/drbd-user</A><BR></BLOCKQUOTE></DIV><BR></DIV></DIV></DIV></BLOCKQUOTE></DIV><BR></DIV></BLOCKQUOTE><BR></DIV></DIV></BODY></HTML>