[DRBD-user] degraded write performace

[Duane Cox]

> how much memory do you have?

1GIG ECC DDR

> did you try to watch some other monitoring tool during these tests?
> e.g. "vmstat 1"?

Wasn't sure what I was looking for when reading the output...

> did you check the write cache (hdparm)?

root at mailbox2:/src/hdparm-6.9# hdparm -i /dev/sdb
/dev/sdb:
 HDIO_GET_IDENTITY failed: Inappropriate ioctl for device

root at mailbox2:/src/hdparm-6.9# hdparm -I /dev/sdb
/dev/sdb:
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device

I don't think the RAID controller wants to give it (hdparm) access.


> does something change when you use
>  ./dm -a 0 -b 1M -s 3G -y -m -p -x -o /dev/sdb
>                                 ^^^
> (-x: use O_DIRECT...)

./dm -a 0 -b 1M -s 3G -y -m -p -o /dev/sdb ~ consistent 70 MB/s
./dm -a 0 -b 1M -s 3G -x -y -m -p -o /dev/sdb ~ consistent 50 MB/s

>
>
> ((you should not use /dev/sdb when that is configured as lower level
> device for drbd))

ok, as per your instructions, I tried this and here are the results...

running this "./dm -a 0 -b 1M -s 3G -y -m -p -x -o /dev/sdb" concurrently
produces the same repetitive results, and those are about a 70 MB/s
benchmark.

now I "modprobe drbd" and "drbdadm adjust all" and "drbdadm primary all" (I
only have one resource BTW and state is disconnected) and ...
running this "./dm -a 0 -b 1M -s 3G -y -m -p -o /dev/drbd0" concurrently
produces the same repetitive results, and those are about a 33 MB/s
benchmark.  Constantly 33MB/s nothing higher, nothing lower.

I am losing more that 50% just by going through the DRBD block device when
it's disconnected from the peer!

>
> == drbd disconnected ==
>
> when you write to drbd, and it is disconnected,
> drbd needs to update its activity log and dirty bitmap every now and
> then, which introduces additional writes, and seeks to the appropriate
> drbd meta data locations and back...

I'm using an internal meta disk, would that cause the more than 50%
reduction?