[DRBD-user] multiple DRBD + solid state drive + 10G Ethernet performance tuning. Help!!

wang xuchen ben.wxc at gmail.com
Fri Oct 29 18:23:14 CEST 2010


Since either network nor hard-drive is the bottle-neck, I would assume that
DRBD replication contribute most to the performance bottleneck, of course,
with the premise that my configuration does not put any limitation on DRBD
device.


Commit yourself to constant self-improvement


On Fri, Oct 29, 2010 at 12:18 PM, Bart Coninckx <bart.coninckx at telenet.be>wrote:

> On Friday 29 October 2010 18:09:29 wang xuchen wrote:
> > Hi all,
> >
> > I have encountered a DRBD write performance bottleneck issue.
> >
> > According to DRBD specification "DRBD then reduces that throughput
> maximum
> > by its additional throughput overhead, which can be expected to be less
> > than 3 percent."
> >
> > My current test environment is:
> >
> > (1) Hard-drive:  300G SSD with 8 partitions on it, each of which has a
> DRBD
> > device create on top it. I use dd utility to test its performance: 97
> MB/s
> > with 4k block size.
> >
> >
> > (2) netowork: dedicated 10G ethernet card for data replication:
> > ethtool eth2
> > Settings for eth2:
> > ...
> >         Speed: 10000Mb/s
> > ...
> >
> > (3) DRBD configuration: (Here is one of them).
> >
> >     on Server1 {
> >         device           /dev/drbd3 minor 3;
> >         disk             /dev/fioa3;
> >         address          ipv4 192.168.202.107:7793;
> >         meta-disk        internal;
> >     }
> >     on NSS_108 {
> >         device           /dev/drbd3 minor 3;
> >         disk             /dev/fioa3;
> >         address          ipv4 192.168.202.108:7793;
> >         meta-disk        internal;
> >     }
> >     net {
> >         allow-two-primaries;
> >         after-sb-0pri    discard-zero-changes;
> >         after-sb-1pri    consensus;
> >         after-sb-2pri    call-pri-lost-after-sb;
> >         rr-conflict      disconnect;
> >         max-buffers      4000;
> >         max-epoch-size   16000;
> >         unplug-watermark 4000;
> >         sndbuf-size       2M;
> >         data-integrity-alg crc32c;
> >     }
> >     syncer {
> >         rate             300M;
> >         csums-alg        md5;
> >         verify-alg       crc32c;
> >         al-extents       3800;
> >         cpu-mask           2;
> >     }
> > }
> >
> > (4) Test result:
> >
> > I have a simple script which use multiple instance of dd to their
> > corresponding DRBD device
> >
> > dd if=/dev/zero of=/dev/drbd1 bs=4k count=10000 oflag=direct &
> > ....
> >
> > For one device, I got roughly 8M/s. As the test goes, I increase the
> number
> > of device to see if it helps the performance. Unfortunately, as the
> number
> > of device grows, performance seems to be distributed on each of the
> device
> > with the total add up to 10M/s.
> >
> > Can somebody give me a hint on what was going wrong?
> >
> > Many Thanks.
> > Ben
> >
> >
> > Commit yourself to constant self-improvement
>
> What does a netperf speed test tell you?
>
> B.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20101029/ea85d658/attachment.htm>


More information about the drbd-user mailing list