Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Heya -
Same problem here!
With a 3ware SATA Raid with 7 Maxtor 7Y250M0 drives.
With scp rates are about 25MB/s (from machine1 raid to machine2
raid(without drbd))
with netcat up to 30MB. (from file on machine1 raid to file on
machine2 raid(without drbd))
Bonnie shows good performance on the HD's themselves.
BUT:
[root at topdog root]# cat /proc/drbd
version: 0.7.0 svn $Rev: 1438 $ (api:74/proto:74)
0: cs:SyncSource st:Secondary/Secondary ld:Consistent
ns:268888 nr:0 dw:0 dr:272000 al:0 bm:16 lo:0 pe:563 ua:778 ap:0
[===================>] sync'ed: 99.9% (1058590/1058850)M
finish: 68:26:02 speed: 4,396 (4,937) K/sec
[root at topdog root]# drbdsetup show /dev/nb0
/dev/nb0 is not a command
[root at topdog root]# drbdsetup /dev/nb0 show
Lower device: 08:33 (sdc1)
Meta device: 08:34 (sdc2)
Meta index: 0
Disk options:
on-io-error = detach
Local address: 10.100.100.1:7789
Remote address: 10.100.100.2:7789
Wire protocol: C
Net options:
timeout = 6.0 sec (default)
connect-int = 10 sec (default)
ping-int = 10 sec (default)
max-epoch-size = 2048 (default)
max-buffers = 2048 (default)
sndbuf-size = 1048576
ko-count = 0 (default)
Syncer options:
rate = 1048576 KB/sec
group = 1
al-extents = 127 (default)
Just like Jeff - I've been beating myself trying to find out where the
bottleneck is but to no avail. As far as I can tell anything *but* drbd
has no problem with network disk writes of up to 30MB/s.
I thought maybe /proc/drbd was reporting something false so I left the
darn thing running over a day and a night and it was still chugging
along at barely a trickle.
This is running on a vanilla 2.6.7 kernel.
iostat on machine1:
avg-cpu: %user %nice %sys %iowait %idle
0.13 0.00 0.69 0.68 98.50
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
md0 3.80 30.38 0.02 112072 56
sda 0.54 10.23 0.00 37740 16
sdb 0.55 10.21 0.00 37644 16
sdc 37.63 2601.03 0.08 9594276 291
sdd 1.36 30.69 11.74 113206 43290
sde 0.54 10.21 0.01 37660 24
on machine2:
avg-cpu: %user %nice %sys %iowait %idle
0.24 0.00 3.73 33.14 62.89
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.01 0.32 0.00 444 0
sdb 0.01 0.32 0.00 444 0
sdc 899.20 64.54 6681.54 90276 9345471
sdd 3.09 74.12 25.68 103670 35914
sde 0.01 0.32 0.00 444 0
Looks like there is a bit of iowait on the inconsistent machine2.
I tried bonding 2 GB nics together but ofcourse that didn't solve
anything.
I'm completely stumped. Am I just stuck with hardware that isn't happy
with the way DRBD does stuff. At the rate its going a full sync would
take a week! That's intense yo.
Thanks for any suggestions!
Cheers,
Tim