[DRBD-user] slow sync speed of 12TB over 10GE

[Lars Täuber]

Hallo you!

Igor Neves <igor at 3gnt.net> schrieb:
> Hello,
> 
> Yes this is true. The metadata can be the problem, i would just use a
> simple raid setup, everything over hardware raid.

The metadata doesn't seem to be the problem. I hope this config puts the meta data onto a ramdisk:
# drbdadm dump
# /etc/drbd.conf
common {
    protocol               C;
    syncer {
        rate             1150M;
    }
    handlers {
        split-brain      "/usr/lib/drbd/notify-split-brain.sh root";
    }
}

# resource RAID61 on monosan: not ignored, not stacked
resource RAID61 {
    on monosan {
        device           /dev/drbd0;
        disk             /dev/md4;
        address          ipv4 10.9.8.7:7788;
        flexible-meta-disk /dev/ram0;
    }
    on duosan {
        device           /dev/drbd0;
        disk             /dev/md4;
        address          ipv4 10.9.8.6:7788;
        flexible-meta-disk /dev/ram0;
    }
    disk {
        no-disk-flushes;
        no-md-flushes;
    }
}

But the speed hasn't increased so far:
# cat /proc/drbd

version: 8.3.0 (api:88/proto:86-89)
GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by root at monosan, 2009-01-09 15:15:29
 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r---
    ns:0 nr:22204216 dw:22201608 dr:0 al:0 bm:1352 lo:654 pe:9889 ua:652 ap:0 ep:1 wo:b oos:6530218744  
        [>....................] sync'ed:  0.4% (6377166/6398848)M
        finish: 59:40:09 speed: 30,364 (88,804) K/sec


What are the other possible bottlenecks?

Thanks
Lars