<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Lars Ellenberg wrote:
<blockquote cite="mid:20090209124217.GB9658@barkeeper1-xen.linbit"
type="cite">
<pre wrap="">On Sat, Feb 07, 2009 at 04:55:10PM -0800, John Du wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I upgraded the kernel to the latest Red Hat release: 2.6.18-128.el5 and
that did not fix the problem. I installed 8.2.7 and it behaves the same
as 8.3. The system becomes too slow to use. I rolled back to 8.2.0.
8.2.0 works in terms of writing data to the underlying storage but it is
not stable in replicating data to the secondary node. Unloading the
DRBD module of 8.2.0 locks up the host. 8.2.7 and 8.3 do not log nay
errors when they are slow. The DRBD worker thread stays in state D and
other processes are blocked.
Here is some observation:
When running 8.2.0, the command "iostat" does not show the DRBD device.
With 8.3.0 or 8.2.7, "iostat" shows the DRBD device as one of the IO
devices. On all our other server where DRBD are used, iostat does not
display the DRBD devices.
In other words, when iostat shows a DRBD divce (/dev/drbd1 in my case)
as one of the IO devices, the host system becomes very slow.
</pre>
</blockquote>
<pre wrap=""><!---->
in January 2008, we started to use blkdev_issue_flush and BIO_RW_BARRIER
to increase reliability when using volatile disk caches.
these changes have been first released with 8.0.9 and 8.2.5.
8.0.9 (2008-02-04) and 8.0.10 (2008-02-07) have been bad releases,
and have been superseeded within days by 8.0.11 (2008-02-12).
8.0.12 (2008-04-24) was the first release that could switch off this
additional reliability feature again, as it turned out that the
"better", more "advanced", IO subsystem you have, the worse the
performance impact is when turning on barriers/flushes.
the options to do so are "no-disk-flushes" (backing store),
"no-md-flushes" (meta data store).
in the 8.2 series, 8.2.6 (2008-05-30) is the first release
to support these keywords to switch off these additional
reliability features again.
8.2.7, and 8.3.0 again add some more advanced handling
for additional performance while maintaining reliability
when running on volatile caches.
again, these features can be switched of.
They should only be switched off if you run on a "safe" device
(non-volatile, battery backed cache -- or no cache at all).
the important additional switch now is "no-disk-barrier".
see also "man drbd.conf"
all of this has _nothing_ to do with whether "iostats" shows
statistics for drbd or not.
</pre>
</blockquote>
Thank you Lars for your detailed explanation. I'll try the parameters
you mentioned above. As I mentioned in my original message the process
queue reaches 100 on this node when running 8.3 and it is under 1 with
8.2.0.<br>
<br>
I still do not understand why iostat only shows DRBD devices on this
particular node with 8.2.7 and 8.3.0 but not other nodes with the same
hardware, same Linux Kernel and same DRBD version.<br>
<br>
<br>
</body>
</html>