[DRBD-user] High io-wait

Phil Frost phil at macprofessionals.com
Thu Aug 16 15:47:25 CEST 2012

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

On 08/15/2012 10:26 PM, Dennis Jacobfeuerborn wrote:
> On 08/15/2012 03:54 AM, Phil Frost wrote:
>> I'd guess (and this is
>> just a guess, I've never examined DRBD internals beyond what's in the
>> manual) that the unusually high %util is due to the activity log [1] or
>> perhaps some other housekeeping function.
>> [1] http://www.drbd.org/users-guide/s-activity-log.html
> Thanks for the detailed explanation. I'm wondering though why something
> like this wouldn't be common knowledge and/or explained in the FAQ if this
> is a generic symptom of DRBD. Tomorrow I'm going to do some performance
> tests to see if this is a real problem or just a phantom issue.

It would probably be very educational to set up a DRBD device with 
external metadata, and then examine the disk activity during your 
benchmark with blktrace. This way you can see with great detail the 
requests being issued to the backing device, separate from those to the 
metadata device, and see how they compare when the DRBD device is 
presented with different loads. I'm sure plenty of people would 
appreciate the insight, myself included.

It is known more generally among storage professionals, that many of the 
numbers given by iostat are more like guidelines, and often assumed to 
have a more simple meaning than they actually do. It's simply not 
possible for the simple metrics provided by the kernel to give a 
detailed or even accurate measurement of the performance of a complex 
storage device, the topology and properties of which the kernel really 
knows nothing. Just search for "understanding iostat", and you will get 
pages with titles like "When iostat Leads You Astray".

I didn't really feel I had a grasp on what all iostat's numbers mean 
until I read more about the kernel counters that are the source of 
iostat's calculations [1] and then read the sysstat source code to see 
just what it was doing. I've attached a draft document I wrote 
attempting to explain what I learned. You may find it useful.

[1] http://www.kernel.org/doc/Documentation/iostats.txt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20120816/7673c90a/attachment.html>

More information about the drbd-user mailing list