[DRBD-user] Slow resync

Ricardo Branco ricardo at wenn.com
Mon Jan 4 17:48:53 CET 2016

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

I have an odd situation with resync.

Resync seems to always be limited to max 40MB/s, if I am doing both of my resources at the same time then both are at 
40MB/s (total 80M), but never goes above this.
If I copy a 20G file which is on drbd then the sync speed to the other server reaches 800GB/s, doing a dd gives similar 
results around 1GB/s.
Does not matter what param I tweek i can not get the sync speed to be above 40MB/s.
To test that cachecade is not getting in the way I have disabled it for tests but nothing changed.

This server has been running 8.4.3 up to recently, now 8.4.6.
Before the upgrade I am sure when I did a sync I saw much higher resync rates.

The server setup is:
16x 3TB SAS 7.2k RAID 6 on LSI Megaraid 9271-4i (FBWC + CacheCade)
10G Ethernet direct link between servers

resource d0 {
     options {
         cpu-mask                ""; # default
         on-no-data-accessible   io-error; # default
     net {
         protocol                C; # default
         timeout                 60; # 1/10 seconds, default
         max-epoch-size          2048; # default
         max-buffers             2048; # default
         unplug-watermark        128; # default
         connect-int             10; # seconds, default
         ping-int                10; # seconds, default
         sndbuf-size             0; # bytes, default
         rcvbuf-size             0; # bytes, default
         ko-count                7; # default
         allow-two-primaries     no; # default
         cram-hmac-alg           ""; # default
         shared-secret           ""; # default
         after-sb-0pri           disconnect; # default
         after-sb-1pri           disconnect; # default
         after-sb-2pri           disconnect; # default
         always-asbp             no; # default
         rr-conflict             disconnect; # default
         ping-timeout            5; # 1/10 seconds, default
         data-integrity-alg      ""; # default
         tcp-cork                yes; # default
         on-congestion           block; # default
         congestion-fill         0s; # bytes, default
         congestion-extents      1237; # default
         csums-alg               ""; # default
         csums-after-crash-only  no; # default
         verify-alg              "crc32c";
         use-rle                 yes; # default
         socket-check-timeout    0; # default
     _remote_host {
         address                 ipv4;
     _this_host {
         address                 ipv4;
         volume 0 {
             device                      minor 1;
             disk                        "/dev/vg_raid1/drbd0";
             meta-disk                   internal;
             disk {
                 size                    0s; # bytes, default
                 max-bio-bvecs           0; # default
                 on-io-error             detach; # default
                 fencing                 resource-only;
                 disk-barrier            no; # default
                 disk-flushes            no;
                 disk-drain              yes; # default
                 md-flushes              yes; # default
                 resync-rate             512000k; # bytes/second
                 resync-after            -1; # default
                 al-extents              1237; # default
                 al-updates              yes; # default
                 c-plan-ahead            0; # 1/10 seconds
                 c-delay-target          10; # 1/10 seconds, default
                 c-fill-target           100s; # bytes, default
                 c-max-rate              102400k; # bytes/second, default
                 c-min-rate              250k; # bytes/second, default
                 disk-timeout            0; # 1/10 seconds, default
                 read-balancing          prefer-local; # default

  0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----
     ns:0 nr:262361424 dw:325178528 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:11362364
         [==>.................] sync'ed: 15.5% (11096/13116)M
         finish: 0:05:15 speed: 35,944 (31,844) want: 512,000 K/sec

The information contained in or attached to this email is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain any message or any part of it. If you have received an email in error, please contact the sender and delete the material from any computer.  The contents of this email are not for publication unless specifically stated.  Furthermore, the information contained in this message, and any attachment(s) thereto, is for information purposes only and may contain the personal views and opinions of the author, which are not necessarily the views and opinions of WENN or its subsidiaries and associated companies.  We make every effort to keep our network free from viruses. However, you do need to check this e-mail and any attachments to it for viruses as we can take no responsibility for any computer virus which may be transferred by way!
  of this e-mail.

WENN Ltd | Registered Office: 4a Tileyard Studios, Tileyard Road, London, N7 9AH, England.  Registered No: 4375163.  Place of Registration: United Kingdom
USA Entertainment News Inc (d/b/a WENN) | Registered Office: 352 7th Avenue, Suite 1105, New York, NY 10001-5657, USA
WENN GmbH |  Schoenhauser Allee 149, 10435 Berlin, Germany | Geschäftsführer: Lloyd Beiny, Heiko Schoenborn | HRB 150939 B | Berlin-Charlottenburg | USt-IdNr: DE815442660

The WENN name, design and related marks are trademarks of the WENN group of companies.  (c)All Rights Reserved.

More information about the drbd-user mailing list