Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
hi,
i have two node drbd in primary/secondary mode with this config:
Connection between nodes produced by back-to-back 1Gigabit Ethernet (cat 5)
DRBD Config:
resource "res" {
protocol C;
handlers {
pri-on-incon-degr "echo o > /proc/sysrq-trigger ; halt -f";
pri-lost-after-sb "echo o > /proc/sysrq-trigger ; halt -f";
local-io-error "echo o > /proc/sysrq-trigger ; halt -f";
outdate-peer "/usr/lib/heartbeat/drbd-peer-outdater -t 5";
}
startup {
wfc-timeout 20;
degr-wfc-timeout 20; # 2 minutes.
}
disk {
on-io-error detach;
}
net {
timeout 60; # 6 seconds (unit = 0.1 seconds)
connect-int 10; # 10 seconds (unit = 1 second)
ping-int 10; # 10 seconds (unit = 1 second)
ping-timeout 5; # 500 ms (unit = 0.1 seconds)
max-buffers 8000;
unplug-watermark 128;
max-epoch-size 8000;
ko-count 4;
cram-hmac-alg "sha1";
shared-secret "jecCOMirrorSystemWithPDCO&KVM";
after-sb-0pri disconnect;
after-sb-1pri disconnect;
after-sb-2pri disconnect;
rr-conflict disconnect;
}
syncer {
rate 50M;
al-extents 3389;
}
on primary {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.200.1:7788;
meta-disk /dev/VolGroup00/p[0];
}
on secondary {
device /dev/drbd0;
disk /dev/cciss/c0d0p7;
address 192.168.200.2:7788;
meta-disk /dev/cciss/c0d0p8[0];
}
}
OS: Sience Linux 6 (x86_64)
Kernel: 2.6.32-71.29.1.el6 IO Scheduler: DeadLine
DRBD: version: 8.3.10
Compiled From Source with this commands:
# tar xf drbd-8.3.10.tar.gz
# cd drbd-8.3.10
# make rpm
# make km-rpm KDIR=/usr/src/kernels/2.6.32-71.29.1.el6.x86_64
Hardware:
primary: HP G7, with 24Core CPU
secondary: HP G6, with 8Core CPU
---------------------
by this environment, synchronization speed is very very slow, about 5Mbit/s ...
by a simple scp command, i can achieve about 35Mbit/s ...
# cat /proc/drbd
version: 8.3.10 (api:88/proto:86-96)
GIT-hash: 5c0b0469666682443d4785d90a2c603378f9017b build by
rpmbuild at P2-SL6, 2011-06-07 11:05:12
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
ns:214112 nr:0 dw:212963132 dr:986726277 al:417500 bm:75663 lo:1
pe:320 ua:745 ap:28 ep:1 wo:b oos:1638452
[=>..................] sync'ed: 11.2% (1638452/1839516)K
finish: 0:04:39 speed: 5,844 (6,484) K/sec
when we connect primary to secondary throw "drbdadm connect", load of
primary grows ... at initial sync, load of primary been about 100%,
On drbd0 at primary side, we have six simple virtual machine run throw qemu+kvm.
After drbd connection, performance really goes down ...
Output of sar command before/after drbd connection is:
DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await
svctm %util
dev147-0 0.00 0.00 0.00 0.00 15.84 0.00
0.00 99.01
The question is Why?
Why we should experience this degradation in performance?
One note may help: at drbd startup (service drbd start) and after
connection of secondary to primary, we saw this error:
block drbd0: meta connection shut down by peer
on secondary:
block drbd0: Barriers not supported on meta data device - disabling
Best regards ...