Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Since either network nor hard-drive is the bottle-neck, I would assume that
DRBD replication contribute most to the performance bottleneck, of course,
with the premise that my configuration does not put any limitation on DRBD
device.
Commit yourself to constant self-improvement
On Fri, Oct 29, 2010 at 12:18 PM, Bart Coninckx <bart.coninckx at telenet.be>wrote:
> On Friday 29 October 2010 18:09:29 wang xuchen wrote:
> > Hi all,
> >
> > I have encountered a DRBD write performance bottleneck issue.
> >
> > According to DRBD specification "DRBD then reduces that throughput
> maximum
> > by its additional throughput overhead, which can be expected to be less
> > than 3 percent."
> >
> > My current test environment is:
> >
> > (1) Hard-drive: 300G SSD with 8 partitions on it, each of which has a
> DRBD
> > device create on top it. I use dd utility to test its performance: 97
> MB/s
> > with 4k block size.
> >
> >
> > (2) netowork: dedicated 10G ethernet card for data replication:
> > ethtool eth2
> > Settings for eth2:
> > ...
> > Speed: 10000Mb/s
> > ...
> >
> > (3) DRBD configuration: (Here is one of them).
> >
> > on Server1 {
> > device /dev/drbd3 minor 3;
> > disk /dev/fioa3;
> > address ipv4 192.168.202.107:7793;
> > meta-disk internal;
> > }
> > on NSS_108 {
> > device /dev/drbd3 minor 3;
> > disk /dev/fioa3;
> > address ipv4 192.168.202.108:7793;
> > meta-disk internal;
> > }
> > net {
> > allow-two-primaries;
> > after-sb-0pri discard-zero-changes;
> > after-sb-1pri consensus;
> > after-sb-2pri call-pri-lost-after-sb;
> > rr-conflict disconnect;
> > max-buffers 4000;
> > max-epoch-size 16000;
> > unplug-watermark 4000;
> > sndbuf-size 2M;
> > data-integrity-alg crc32c;
> > }
> > syncer {
> > rate 300M;
> > csums-alg md5;
> > verify-alg crc32c;
> > al-extents 3800;
> > cpu-mask 2;
> > }
> > }
> >
> > (4) Test result:
> >
> > I have a simple script which use multiple instance of dd to their
> > corresponding DRBD device
> >
> > dd if=/dev/zero of=/dev/drbd1 bs=4k count=10000 oflag=direct &
> > ....
> >
> > For one device, I got roughly 8M/s. As the test goes, I increase the
> number
> > of device to see if it helps the performance. Unfortunately, as the
> number
> > of device grows, performance seems to be distributed on each of the
> device
> > with the total add up to 10M/s.
> >
> > Can somebody give me a hint on what was going wrong?
> >
> > Many Thanks.
> > Ben
> >
> >
> > Commit yourself to constant self-improvement
>
> What does a netperf speed test tell you?
>
> B.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20101029/ea85d658/attachment.htm>