[DRBD-user] Slow syncall after kernel+drbd upgrade

Lars Ellenberg Lars.Ellenberg at linbit.com
Wed Jan 21 13:02:00 CET 2004

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


/ 2004-01-21 06:58:15 +0100
\ Christian Hammers:
> Hello
> 
> On Tue, Jan 20, 2004 at 12:51:28PM +0100, Lars Ellenberg wrote:
> > > I used to have 2.4.23 and drbd 0.6.4 running on two servers sharing
> > > 2x 5GB over a 100MBit crosslink.
> > > 
> > > Now I upgraded to 2.4.24 and drbd 0.6.10 and two Intel e1000.o gigabit
> > > cards and my syncall time drops from about 15mins to more than 1h!
> > >
> > > After the syncinc was done, I tested the link with "rsync -v -a
> > > --progress ..." and got at least 15MB/s = 120MBit/s which is not much 
> > > for a 1000MBit/s link but at least more than 2MBit/s that drbd got!
> > > 
> > > Below you find the "cat /proc/drbd" and the "drbdsetup /dev/nb0 show" 
> > > outputs. The sync-max values differ a bit as I played around before
> > > giving up...
> 
> ...
> > >  sync-min = 200 KB/sec
> > you should increase sync-min !!
> > >  sync-max = 5000 KB/sec
> > of course you want to increase sync-max again...
> > >  sync-nice = 10
> > you should probably sync-nice = -10 or so
> 
> And so I did. I also changed the devices MTU size to 5000 and verified
> with iptraf that this large packet size is actually used. It is.
> I could rsync a 100MB file over the very same crossover cable with
> 15MB/s and the servers are quite idle according to vmstat :-(
> 
> But the syncAll still is no bit faster!

if nothing else is in the syslog, 
then now please try my last suggestion:
sndbuf-size = 65534 or even less...


> 
> bye,
> 
> -christian-
> 
> root at mail3b:~# cat /proc/drbd 
> version: 0.6.10 (api:64/proto:62)
> 
> 0: cs:SyncingAll st:Secondary/Primary ns:0 nr:1435652 dw:1435652 dr:0 pe:0 ua:15
>         [=====>..............] sync'ed: 26.8% (3668/5004)M
>         finish: 0:47:57h speed: 1,319 (1,313) K/sec
> 1: cs:SyncingAll st:Secondary/Primary ns:0 nr:1367248 dw:1367248 dr:0 pe:0 ua:15
>         [=====>..............] sync'ed: 26.7% (3672/5004)M
>         finish: 0:47:40h speed: 1,336 (1,310) K/sec

hm. they still sync at the same time.
You are absolutely sure they don't kill performance with seektime
on the same spindle?

> root at mail3a:~# drbdsetup /dev/nb0 show	<--- same value on both hosts
> Lower device: 08:07   (/dev/sda7)            and both partitions.
> Disk options:
>  disk-size = 5124703 KB
>  do-panic
> Local address: 10.10.10.1:7788
> Remote address: 10.10.10.2:7788
> Wire protocol: C
> Net options:
>  timeout = 6.0 sec
>  tl-size = 5000
>  connect-int = 10 sec
>  ping-int = 10 sec
>  sndbuf-size = 131070
>  sync-min = 100000 KB/sec
>  sync-max = 122070 KB/sec
>  sync-nice = -10

looks absolutely fine.


	Lars Ellenberg



More information about the drbd-user mailing list