Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Tue, Dec 16, 2008 at 08:23:39PM +0000, Rudolph Bott wrote: > Hi List, > > I was wondering if anyone might be able to share some performance > information about his/her DRBD setup. Ours comes along with the > following Hardware: > > Hardware: Xeon QuadCore CPU, 2GB RAM, Intel Mainboard with 2 Onboard > e1000 NICs and one additional plugged into a regular PCI slot, 3ware > 9650SE (PCI-Express) with 4 S-ATA Disks in a RAID-10 array > > Software: Ubuntu Hardy LTS with DRBD 8.0.11 (from the ubuntu repository), Kernel 2.6.24 > > one NIC acts as "management interface", one as the DRBD Link, one as > the heartbeat interface. On top of DRBD runs LVM to allow the creation > of volumes (which are in turn exported via iSCSI). Everything seems to > run smoothly - but I'm not quite satisfied with the write speed > available on the DRBD device (locally, I don't care about the iSCSI > part yet). > > All tests were done with dd (either copying from /dev/zero or to > /dev/null with 1, 2 or 4GB sized files). Reading gives me speeds at > around 390MB/sec which is way more than enough - but writing does not > exceed 39MB/sec. Direct writes to the raid controller (without DRBD) > are at around 95MB/sec which is still below the limit of Gig-Ethernet. > I spent the whole day tweaking various aspects (Block-Device tuning, > TCP-offload-settings, DRBD net-settings etc.) and managed to raise the > write speed from initially 25MB/sec to 39MB/sec that way. > > Any suggestions what happens to the missing ~60-50MB/sec that the > 3ware controller is able to handle? Do you think the PCI bus is > "overtasked"? Would it be enough to simply replace the onboard NICs > with an additional PCI-Express Card or do you think the limit is > elsewhere? (DRBD settings, Options set in the default Distro Kernel > etc.). drbdadm dump all drbdsetup /dev/drbd0 show what exactly does your micro benchmark look like? how do "StandAlone" and "Connected" drbd compare? what thoughput does the drbd resync achieve? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed