[DRBD-user] Performance Issue - HP cciss

Matteo Tescione matteo at rmnet.it
Wed Jan 30 15:02:24 CET 2008

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

> Lars Ellenberg wrote:
>> On Wed, Jan 30, 2008 at 11:12:54AM +0000, Ben Clewett wrote:
>>> Hi Lars,
>>> I am still having performance issues, ref DRBD locking.  You were
> kind enough to spend some time to worked out it was probably my
> underlying storage at fault.
>>> I am using HP cciss P400 hardware RAID controllers.  I have these in
> 12 HP Proliant DL G5 servers, only four of these are on DRBD.  These
> four are the ones which suffer the problem.
>>> I have recently changed the read-ahead-buffer from the default 4 MB
> to 512 KB.  Therefore when the disk reads 1 or more bytes, it used to
> read another 4 MB into cache, and now reads 512 KB.  I can't take these
> servers off line to rigorously test, but anecdotally this seems to have
> made a difference for the better.
>>> I note my meta-disks are covered by this read-ahead-buffer.
>>> Do you think this might effect DRBD in some scenario?
>>> If you had to suggest a read-ahead-buffer size for the meta-disk,
> can you suggest what might be optimum?
>> since DRBD meta-data access most of the time is sinlge sector
>> synchronous IO, read-ahead is completely irrelevant here.
> Oh well, back to the drawing board.
> I think changing the read-ahead value might have caused the disk
> controller to re-start, therefore unlocking what ever was stuck on the
> card.  In which case I could just change this value from 'cron' every
> hour and never see this problem again :)
>>> Thanks again for your excellent analysis,
>> we also do support and consulting, you know :)
> It may come to that, and I am happy to contract you if need be.  However
> I still have a few options, and I would like to fix it my self.  I have
> to maintain these objects of delight, and this is a good way of learning
> how they work, or not work :)
> I'd like to ask the group if any owners of HP cciss disk controllers
> have experienced similar locking, evident by 100% 'iowait' on both sides
> of a DRBD connection.  If so, can they mail be the version they have and
> how they fixed it please?
I did. I runned in really deep troubles few months ago with a p400 array
from dl320s servers. 100% util and very poor performance about 10-20 mb/s.
I didn't find a true solution, but i discovered 2 workaround:

1. changed the raid level from 5/6 to 10 because of the slow latency disk
operations. See the post "performance solved"

2 upgraded the p400 firmware twice. I had to download 2 cd's from hp with
very important fixes in their controller, like introducing sata command
queue and many others.

Hope this helps,

#Matteo Tescione
#RMnet srl

More information about the drbd-user mailing list