Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello, On Tue, 8 Apr 2008 17:20:37 +0200 Florian Haas wrote: > On Tuesday 08 April 2008 05:27:25 Christian Balzer wrote: > > These are (except for the 2 primaries) nearly exactly the same > > settings I came up with after a lot of testing. > > > > In my case it's a dual quad core box with 24GB RAM and 8 1TB SATA > > drives in a RAID-5 that give me about 120MB/s writes on the "bare" MD > > device. The first try with default values on DRBD gave me about 41MB/s > > writes and a big "WTH?" feeling. > > If your local I/O subsystem pulls 120MB/s, the expected max DRBD > throughput is around 105 MB/s: > > - disk does 120, > - Gigabit Ethernet realistically does 110, Yeah, various network tests get near that value. > - so the network is your bottleneck, It's a bonded dual GigE. Initially done to give DRBD enough breathing room and left in place to have more redundancy for it and heartbeat. > - so deduct about 5% DRBD throughput penalty, > - and you end up around 105 MB/s. > Not getting anywhere near that, obviously. > And that's a throughput we routinely tune DRBD to. > I'm at a loss at what more to tune here, seeing that the traffic on the wire(s) is the one thing that never goes higher than the above numbers. And the largest improvements so far were things that speed up writing on the ACTIVE (aka sender) node so given all this I'm more convinced that the bottleneck is local and not the network or the passive, receiving node. I have a feeling that the upcoming no-md-flushes might work some wonders here. > Also, don't confuse throughout and latency, and measure block device > performance before file system performance. > I've just been using bonnie++ with x2 RAM size on identically created file systems to get "real world" numbers and also to have something to compare to other systems of mine. Regards, Christian -- Christian Balzer Network/Systems Engineer NOC chibi at gol.com Global OnLine Japan/Fusion Network Services http://www.gol.com/