[DRBD-user] Why so slow? Troubleshoot?

[Christian Völker]

Hi,

I have a nearly default setup running. I moved the secondary drbd to a
VM at it appears to be incredibly slow.

I need some hints to troubleshoot why it is so slow. See the cat /proc/drbd:
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by
phil at Build64R6, 2013-09-27 16:00:43
 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate A r-----
    ns:0 nr:38648 dw:38584 dr:0 al:0 bm:2 lo:16 pe:41 ua:1 ap:0 ep:1
wo:d oos:1677699908
        [>....................] sync'ed:  0.1% (1638376/1638412)M
        finish: 1792:24:51 speed: 244 (256) want: 250 K/sec

Speed of 244 is ways to slow. It ends up hours later with a io error on
the disk.

But when I check performance values I do not see any bottlenecks:
-CPU IO wait is below 2%
-CPU itself is ~98%idle
-network is not busy at all.

Anyone having a clue why this is so slow?
Here my drbd.res:

resource drbd0 {
  protocol A;
  startup {
        wfc-timeout 240;
        degr-wfc-timeout     120;
        become-primary-on backuppc;
        }
  disk {
        on-io-error detach;
        no-disk-flushes;
        no-md-flushes;
        use-bmbv;
        }
  net {
        cram-hmac-alg "sha1";
        shared-secret "ixxxxxxxxxxxxxxi";
        ping-timeout 100;
        }
#  syncer { rate 5M; }

  on backuppc {
    device /dev/drbd0;
    disk /dev/vg1/backuppc;
    address 192.168.34.1:7789;
    meta-disk internal;
  }
  on drbd {
    device /dev/drbd0;
    disk /dev/sda;
    address 192.168.34.16:7789;
    meta-disk internal;
  }
}

All hints are welcom!

Thanks

/Christian