Since either network nor hard-drive is the bottle-neck, I would assume that DRBD replication contribute most to the performance bottleneck, of course, with the premise that my configuration does not put any limitation on DRBD device.<br>
<br><br clear="all">Commit yourself to constant self-improvement<br>
<br><br><div class="gmail_quote">On Fri, Oct 29, 2010 at 12:18 PM, Bart Coninckx <span dir="ltr"><<a href="mailto:bart.coninckx@telenet.be">bart.coninckx@telenet.be</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div><div></div><div class="h5">On Friday 29 October 2010 18:09:29 wang xuchen wrote:<br>
> Hi all,<br>
><br>
> I have encountered a DRBD write performance bottleneck issue.<br>
><br>
> According to DRBD specification "DRBD then reduces that throughput maximum<br>
> by its additional throughput overhead, which can be expected to be less<br>
> than 3 percent."<br>
><br>
> My current test environment is:<br>
><br>
> (1) Hard-drive: 300G SSD with 8 partitions on it, each of which has a DRBD<br>
> device create on top it. I use dd utility to test its performance: 97 MB/s<br>
> with 4k block size.<br>
><br>
><br>
> (2) netowork: dedicated 10G ethernet card for data replication:<br>
> ethtool eth2<br>
> Settings for eth2:<br>
> ...<br>
> Speed: 10000Mb/s<br>
> ...<br>
><br>
> (3) DRBD configuration: (Here is one of them).<br>
><br>
> on Server1 {<br>
> device /dev/drbd3 minor 3;<br>
> disk /dev/fioa3;<br>
> address ipv4 <a href="http://192.168.202.107:7793" target="_blank">192.168.202.107:7793</a>;<br>
> meta-disk internal;<br>
> }<br>
> on NSS_108 {<br>
> device /dev/drbd3 minor 3;<br>
> disk /dev/fioa3;<br>
> address ipv4 <a href="http://192.168.202.108:7793" target="_blank">192.168.202.108:7793</a>;<br>
> meta-disk internal;<br>
> }<br>
> net {<br>
> allow-two-primaries;<br>
> after-sb-0pri discard-zero-changes;<br>
> after-sb-1pri consensus;<br>
> after-sb-2pri call-pri-lost-after-sb;<br>
> rr-conflict disconnect;<br>
> max-buffers 4000;<br>
> max-epoch-size 16000;<br>
> unplug-watermark 4000;<br>
> sndbuf-size 2M;<br>
> data-integrity-alg crc32c;<br>
> }<br>
> syncer {<br>
> rate 300M;<br>
> csums-alg md5;<br>
> verify-alg crc32c;<br>
> al-extents 3800;<br>
> cpu-mask 2;<br>
> }<br>
> }<br>
><br>
> (4) Test result:<br>
><br>
> I have a simple script which use multiple instance of dd to their<br>
> corresponding DRBD device<br>
><br>
> dd if=/dev/zero of=/dev/drbd1 bs=4k count=10000 oflag=direct &<br>
> ....<br>
><br>
> For one device, I got roughly 8M/s. As the test goes, I increase the number<br>
> of device to see if it helps the performance. Unfortunately, as the number<br>
> of device grows, performance seems to be distributed on each of the device<br>
> with the total add up to 10M/s.<br>
><br>
> Can somebody give me a hint on what was going wrong?<br>
><br>
> Many Thanks.<br>
> Ben<br>
><br>
><br>
> Commit yourself to constant self-improvement<br>
<br>
</div></div>What does a netperf speed test tell you?<br>
<font color="#888888"><br>
B.<br>
</font></blockquote></div><br>