Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
HI Lars and folks, Il 09/02/09 13:42, "Lars Ellenberg" <[IND]> ha scritto: > On Sat, Feb 07, 2009 at 04:55:10PM -0800, John Du wrote: >> I upgraded the kernel to the latest Red Hat release: 2.6.18-128.el5 and >> that did not fix the problem. I installed 8.2.7 and it behaves the same >> as 8.3. The system becomes too slow to use. I rolled back to 8.2.0. >> 8.2.0 works in terms of writing data to the underlying storage but it is >> not stable in replicating data to the secondary node. Unloading the >> DRBD module of 8.2.0 locks up the host. 8.2.7 and 8.3 do not log nay >> errors when they are slow. The DRBD worker thread stays in state D and >> other processes are blocked. >> >> Here is some observation: >> >> When running 8.2.0, the command "iostat" does not show the DRBD device. >> With 8.3.0 or 8.2.7, "iostat" shows the DRBD device as one of the IO >> devices. On all our other server where DRBD are used, iostat does not >> display the DRBD devices. >> >> In other words, when iostat shows a DRBD divce (/dev/drbd1 in my case) >> as one of the IO devices, the host system becomes very slow. > > in January 2008, we started to use blkdev_issue_flush and BIO_RW_BARRIER > to increase reliability when using volatile disk caches. > these changes have been first released with 8.0.9 and 8.2.5. > 8.0.9 (2008-02-04) and 8.0.10 (2008-02-07) have been bad releases, > and have been superseeded within days by 8.0.11 (2008-02-12). > > 8.0.12 (2008-04-24) was the first release that could switch off this > additional reliability feature again, as it turned out that the > "better", more "advanced", IO subsystem you have, the worse the > performance impact is when turning on barriers/flushes. > > the options to do so are "no-disk-flushes" (backing store), > "no-md-flushes" (meta data store). > > in the 8.2 series, 8.2.6 (2008-05-30) is the first release > to support these keywords to switch off these additional > reliability features again. > > 8.2.7, and 8.3.0 again add some more advanced handling > for additional performance while maintaining reliability > when running on volatile caches. > > again, these features can be switched of. > > They should only be switched off if you run on a "safe" device > (non-volatile, battery backed cache -- or no cache at all). > > the important additional switch now is "no-disk-barrier". > > see also "man drbd.conf" > > all of this has _nothing_ to do with whether "iostats" shows > statistics for drbd or not. It's a long time since i've been using the no-disk-barrier, no-disk-flushes and no drain to optimize the performance of our datastores (3ware 9650 w/ BBU cache) but it has no apparent correlations btw the 2 mentioned drbd version (8.2.7 vs 8.3.0) We are using Centos5.2 x86_64 with a custom kernel and the read performance lost (about 20%) is only seen running 8.3.0 (same config&setup) If you don't believe me I can produce logs and iostats during some IO test. In the btw, iostats accounting for drbd devices is still crazy, even reverting back to 8.2.7 Any suggestions? Regards, -- So long and thank for all the fish -- #Matteo Tescione #RMnet srl