[DRBD-user] FW: Slow sync speed over gigabit

Lavender, Ben ben.lavender at mcdean-europe.com
Tue Aug 14 17:02:54 CEST 2007


The sysctl settings here gave a small boost, but they still aren't even
approaching what gigabit is capable of.

iperf does indeed give me 100% of the link, 990Mb/sec (the tcp settings
given here boosted it from about 940).

If I cat /proc/drbd quite a bit, I occasionally get interesting results
like the following, perhaps 1 out of 20 times.  Note the drastically
reduced speed reading:

[root at mail-2 ~]# cat /proc/drbd
version: 8.0.5 (api:86/proto:86)
SVN Revision: 3011 build by ben at m-2w9juvlmyfmj7.mcdean-europe.com,
2007-08-13 15:12:01
 0: cs:SyncTarget st:Secondary/Primary ds:Inconsistent/UpToDate C r---
    ns:0 nr:3844992 dw:3836800 dr:0 al:0 bm:231 lo:258 pe:6550 ua:256
ap:0
        [=>..................] sync'ed:  5.5% (65498/69245)M
        finish: 15:31:32 speed: 928 (17,600) K/sec
        resync: used:14/31 hits:246358 misses:248 starving:0 dirty:0
changed:248
        act_log: used:0/907 hits:0 misses:0 starving:0 dirty:0 changed:0


But usually, it's more like this:

[root at mail-2 ~]# cat /proc/drbd
version: 8.0.5 (api:86/proto:86)
SVN Revision: 3011 build by ben at m-2w9juvlmyfmj7.mcdean-europe.com,
2007-08-13 15:12:01
 0: cs:SyncTarget st:Secondary/Primary ds:Inconsistent/UpToDate C r---
    ns:0 nr:5081600 dw:5073408 dr:0 al:0 bm:308 lo:258 pe:4891 ua:256
ap:0
        [=>..................] sync'ed:  7.2% (64291/69245)M
        finish: 1:14:08 speed: 14,740 (15,752) K/sec
        resync: used:11/31 hits:321915 misses:320 starving:0 dirty:0
changed:320
        act_log: used:0/907 hits:0 misses:0 starving:0 dirty:0 changed:0


I'm not accusing drbd of limiting me, but everything else works as
expected and my ethernet interfaces don't have any errors.  Any
thoughts?

Ben



> -----Original Message-----
> From: drbd-user-bounces at lists.linbit.com [mailto:drbd-user-
> bounces at lists.linbit.com] On Behalf Of Leroy van Logchem
> Sent: Tuesday, August 14, 2007 3:16 PM
> To: drbd-user at lists.linbit.com
> Subject: Re: [DRBD-user] FW: Slow sync speed over gigabit
> 
> 
> > DRBD Partitions: Using a 70gb partition created by Logical Volume
> Manager
> > OS: RHEL5
> > DRBD: 8.0.5, made with 'make rpm' against RHEL kernel-source package
and
> installed with rpm.
> >         finish: 1:20:30 speed: 12,568 (11,900) K/sec
> >
> 
> # TCP adjustments for >1Gb/s
> net.ipv4.tcp_no_metrics_save = 1
> net.ipv4.tcp_rmem = 4096        87380   16777216
> net.ipv4.tcp_wmem = 4096        87380   16777216
> net.ipv4.tcp_reordering = 100
> net.core.netdev_max_backlog = 10000
> net.core.rmem_max = 16777216
> net.core.wmem_max = 16777216
> 
> Can you achieve full bandwidth using iperf?
> 
> DRBD doesnt limit you, we recently got the resync up to 160MB/s
> (sustained)
> using three interfaces of our quad gigabit nic.
> 
> 
> --
> Leroy
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user



More information about the drbd-user mailing list