[DRBD-user] FW: Slow sync speed over gigabit

[Lavender, Ben]

(Apologies if this is a repost)

Hi all,

I found a few entries in the mailing list with slow sync speeds, and I am going ahead and posting this as none of them helped me.

My problem, in short, is that I can't get sync speeds to go over 12MB/sec over gigabit.  Simple SCP will hit 45MB/sec and other protocols can do even better, on occasion, from the same partition that I'm trying to get drbd going on.  I am having a hard time finding the limitation.

I also notice something curious during my testing.  My fstab had this left over from installing RHEL5:
/dev/VolGroup00/LogVol02 /opt                    ext3    defaults        1 2

So when I booted this morning after testing things yesterday, /opt was properly mounted!  Interestingly, it did not only correctly mount as an ext3 file system, with the test files I had placed there yesterday, the test files were on the secondary machine!  I had thought that 'drbdadm create-md drbd-opt' would have destroyed the ext3 file system.  Am I misunderstanding something?

At any rate, all I'm doing is:
Primary: drbdadm up all;drbdadm primary drbd-opt
Secondary: drbdadm up all;

Syncing starts correctly but it goes much slower than I would expect.  It's certainly not hitting the gigabit ceiling.
Servers: Dell 1435's
Disks: 80GB hardware Raid-1
DRBD Partitions: Using a 70gb partition created by Logical Volume Manager
OS: RHEL5
DRBD: 8.0.5, made with 'make rpm' against RHEL kernel-source package and installed with rpm.


My drbd.conf:
resource drbd-opt {

  protocol C;


  syncer {
    rate 650M;

    al-extents 907;
  }

  on mail-1 {
    device     /dev/drbd0;
    disk       /dev/mapper/VolGroup00-LogVol02;
    address    192.168.65.119:7788;
    flexible-meta-disk  internal;
  }

  on mail-2 {
    device    /dev/drbd0;
    disk      /dev/mapper/VolGroup00-LogVol02;
    address   192.168.65.101:7788;
    meta-disk internal;
  }
}


Output of /proc/drbd:

[ben at mail-1 ~]$ cat /proc/drbd
version: 8.0.5 (api:86/proto:86)
SVN Revision: 3011 build by ben at m-2w9juvlmyfmj7.mcdean-europe.com, 2007-08-13 15:12:01
 0: cs:SyncSource st:Primary/Secondary ds:UpToDate/Inconsistent C r---
    ns:10052320 nr:0 dw:24 dr:10060433 al:0 bm:735 lo:1 pe:262 ua:249 ap:0
        [==>.................] sync'ed: 14.2% (59437/69245)M
        finish: 1:20:30 speed: 12,568 (11,900) K/sec
        resync: used:2/31 hits:627641 misses:615 starving:0 dirty:0 changed:615
        act_log: used:0/907 hits:6 misses:0 starving:0 dirty:0 changed:0

Any help is appreciated!


Ben Lavender
Systems Design
MC Dean Europe - Stuttgart
+49(0)711 849 50179
+1 703 803 6231 x 7179