Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hey everyone!
We've been using DRBD for a while, so we've decided to test 8.4.1 for a
possible update on our cluster. So we built a couple test systems with a
really basic and very disposable RAID-0 with the following DRBD config:
resource edb {
options {
}
net {
verify-alg "md5";
}
_remote_host {
address ipv4 192.168.42.1:7788;
}
_this_host {
address ipv4 192.168.42.2:7788;
volume 0 {
device minor 0;
disk "/dev/cciss/c0d1";
meta-disk internal;
disk {
disk-barrier no;
disk-flushes no;
md-flushes no;
c-plan-ahead 1; # 1/10 seconds
c-fill-target 0s; # bytes
}
}
}
}
See? Nothing ridiculous. The thing is, this is what we get during an
online verify:
[=============>......] verified: 74.5% (29140/114148)M
finish: 0:18:27 speed: 26,932 (26,440) want: 102,400 K/sec
The "want" changes basically every second, so I'm not sure what to think
about that. But when I ran the initial sync, which I can force by doing
an invalidate, the speed averaged about 5x higher than that, at about
150MB/s, and peaked at 190MB/s a few times.
What I didn't expect, was this:
Device: r/s %util
cciss/c0d1 7274.50 94.20
That's from iostat on each machine. It bounces between 80 and 100%
utilized with several thousand read requests. Our read ahead is set to
8192 with blockdev, and like I said, the verify ran fine. This looks a
whole lot like random reads, or a lot of *really small* sequential
fetches. Either way, it's pretty terrible performance and it's actually
faster to re-sync the entire thing than to wait for the verify to finish.
Did something change that I'm missing? It seems counter-intuitive that a
verify would be several times slower than a full sync. We didn't have
this particular problem with 8.3.10 (yes, I know we need to upgrade it
to at least 8.3.12. That's why we're looking at 8.4.1.)
Anyway, I'd appreciate any input or ideas.
Thanks!
--
Shaun Thomas
OptionsHouse | 141 W. Jackson Blvd. | Suite 500 | Chicago IL, 60604
312-444-8534
sthomas at optionshouse.com
______________________________________________
See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email