[DRBD-user] Problems with drbd performance with Xen

Lars Ellenberg lars.ellenberg at linbit.com
Mon Feb 16 19:46:28 CET 2009

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On Mon, Feb 16, 2009 at 10:51:45AM +0000, Igor Neves wrote:
> Hi,
> Some of you may know my problem since I have been discussing it with you
> in the IRC in the last few days, but I will explain it.
> I'm doing some tests with drbd with xen virtual machines, our solution
> was with vmware and drbd, but this solution it's with very low
> performance compared to xen, and i'm trying to improve our solution to xen.
> I started my tests with xen on both machines (node1 & node2). Both
> machines have 3 disks, sdb it's the drbd0.
> The disk can only deliver 102MB/sec writing, following seagate, and I
> get about 105MB/sec from each disk in direct access.
> I started my resouce in drbd and i'm able to sync at about 95MB/sec,
> which it's very good. Testing the disk with dd or with your benchmark
> tool called md, I get about 85MB/sec writing, which it's not bad.
> Now when I start the Xen Windows HVM Virtual Machine on node1 and I give
> "/dev/drbd0" as a block device, it works great, but inside windows VM, I
> get:
> - With drbd rpm package 8.2.6 from redhat and with this configuration
> "al-extents 1801" i get about 45MB/sec writing inside VM.
> - With drbd rpm 8.2.7 and 8.3.0 from linbit and with this configuration
> "al_extents 1801" i get about 4MB/sec writing inside VM.
> - With source 8.2.7 and 8.3.0 from linbit and with "al_extents 1801" i
> get again 4MB/sec writing inside VM.
> Now what I have found it's, all this tests were made with drbd in
> connected & updated state. In the middle of the test if I do in node2
> "drbdadm disconnect resource" I get the state WForconnection and Update
> on node1 and in that moment I start getting 85MB/sec.
> So inside Xen Windows HVM Virtual Machine in and with drbd under it as a
> block device i get:
> - drbd  8.2.6 from redhat and drbd connected = 45MB/sec, with drbd
> disconnected = 85MB/sec
> - drbd 8.2.7 from linbit and drbd connected = 4MB/sec, with drbd
> disconnected = 85MB/sec
> - drbd 8.3.0 from linbit and drbd connected = 4MB/sec, with drbd
> disconnected = 85MB/sec

in 8.2.7, we introduced a new method to ensure write ordering
guarantees, which now needs to be explicitly disabled
if you live on a "safe" device (battery backed write cache,
or no cache at all), where this hurts performance.

so where you had "no-disk-flushes" in drbd.conf before,
you now need an additional "no-disk-barrier".

if that does not help, or if you did not have "no-disk-flushes" before,
let me know, and I try to come up with something else.

> I know you can ask, it's not Xen block device driver?
> - Yes it could, so i have added the other disk  (exacly equal to the one
> in drbd resource and with the same throughput) with direct access and
> inside THE SAME windows virtual machine, I have 100MB/sec writing to the
> disk, so the Xen block device it's going good and it's working like a
> charm. Besides that Xen block device does not know "connected" and
> "disconnected" as I explained in the early example.
> I have done this tests all again in node2 as Brian suggested, but i get
> exactly the same values in the tests.
> So there is a problem here with drbd<->xen, something I have not managed
> to find and could be a bug or maybe some bad configuration from my side.
> Please, someone with more experience help out on this.
> My kernel version it's "2.6.18-92.1.22.el5xen" and I'm with Centos 5 and
> with all the latest updates available.
> Thanks,
> Igor Neves

: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
please don't Cc me, but send to list   --   I'm subscribed

More information about the drbd-user mailing list