[DRBD-user] DRBD9 sync speed

Piero Baudino pbaudino at formatnet.it
Tue Jun 21 14:15:11 CEST 2016

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hi Tytus,

thanks!

I have just did some test and the slow sync occurs only with the first 
syncro of the virtual disks between the 2 nodes...
When the disks are syncronized and you write data on the virtual disk, 
the speed is very high!
For example: just tried to install ubuntu server on a virtual machine 
with virtual disks syncronized, the sync speed was about 1 Gbit/sec !

Regards
Piero

Il 21/06/2016 11:01, Tytus Rogalewski ha scritto:
> Lol i made it :)
>
>
> Ok so i thought that it was buggy drbd9 but after 3 days of lost time 
> i finally made it :)
> Before that fix i had 20mbit/s syncing but after that i have 800mbit/s 
> on 1GBit :)
>
> Code:
> NET | bond0    80% | pcki   13298 | pcko  133286 | si 3879 Kbps | so  807 Mbps | coll       0 | mlti      12 | erri       0 | erro       0 | drpi       0 | drpo       0 |
> NET | eth1     80% | pcki   13298 | pcko  133286 | si 3879 Kbps | so  807 Mbps | coll       0 | mlti      12 | erri       0 | erro       0 | drpi       0 | drpo       0 |
> NET | tap0      0% | pcki       4 | pcko       4 | si    0 Kbps | so   12 Kbps | coll       0 | mlti       0 | erri       0 | erro       0 | drpi       0 | drpo       0 |
> NET | eth2
> it was all about stupid config file ..... and accually you need to 
> edit proper config ....
>
> First of all if you have working drbd9 cluster first stop all VMs and 
> then all resources on both drbd nodes:
> Code:
> drbdadm disconnect all
> Then you need to edit config (and this is what i was looking for so 
> long)* /var/lib/drbd.d/drbdmanage_global_common.conf* because editing 
> /etc/drbd.d/global_common.conf doesnt work at all :)
>
> Enter those settings to */var/lib/drbd.d/drbdmanage_global_common.conf 
> ON BOTH NODES
> Code:
> # it must be content of /var/lib/drbd.d/drbdmanage_global_common.conf !!!!
> common {
> disk {
>          on-io-error             detach;
>          no-disk-flushes ;
>          no-disk-barrier;
>          c-plan-ahead 10;
>          c-fill-target 24M;
>          c-min-rate 10M;
>          c-max-rate 100M;
> }.
> net {
>          # max-epoch-size          20000;
>          max-buffers             36k;
>          sndbuf-size            1024k ;
>          rcvbuf-size            2048k;
> }
> }
> *
> Then simply restart drbd by:
> *
> Code:
> /etc/init.d/drbd  restart
> *
> And thats it. Syncing should be at 800mbits :)
> In you have 10GBit just change c-max-rate to 1000M
> Play with those settings
>
> -- 
> sky-link.net <http://sky-link.net> Tytus Rogalewski
> eMail: tytanick at gmail.com <mailto:tytanick at gmail.com>
> strona: http://sky-link.net
> kom. 790 202 300
> gg: 210533
> skype: tytanick
> NIP: 9570976234
> REGON: 221197388
>
>
> 2016-06-21 11:00 GMT+02:00 Tytus Rogalewski <tytanick at gmail.com 
> <mailto:tytanick at gmail.com>>:
>
>     Lol i made it :)
>
>
>     Ok so i thought that it was buggy drbd9 but after 3 days of lost
>     time i finally made it :)
>     Before that fix i had 20mbit/s syncing but after that i have
>     800mbit/s on 1GBit :)
>
>     Code:
>
>     NET | bond0    80% | pcki   13298 | pcko  133286 | si 3879 Kbps | so  807 Mbps | coll       0 | mlti      12 | erri       0 | erro       0 | drpi       0 | drpo       0 |
>     NET | eth1     80% | pcki   13298 | pcko  133286 | si 3879 Kbps | so  807 Mbps | coll       0 | mlti      12 | erri       0 | erro       0 | drpi       0 | drpo       0 |
>     NET | tap0      0% | pcki       4 | pcko       4 | si    0 Kbps | so   12 Kbps | coll       0 | mlti       0 | erri       0 | erro       0 | drpi       0 | drpo       0 |
>     NET | eth2
>
>     it was all about stupid config file ..... and accually you need to
>     edit proper config ....
>
>     First of all if you have working drbd9 cluster first stop all VMs
>     and then all resources on both drbd nodes:
>     Code:
>
>     drbdadm disconnect all
>
>     Then you need to edit config (and this is what i was looking for
>     so long)* /var/lib/drbd.d/drbdmanage_global_common.conf* because
>     editing /etc/drbd.d/global_common.conf doesnt work at all :)
>
>     Enter those settings to
>     */var/lib/drbd.d/drbdmanage_global_common.conf ON BOTH NODES
>     Code:
>
>     # it must be content of /var/lib/drbd.d/drbdmanage_global_common.conf !!!!
>     common {
>     disk {
>              on-io-error             detach;
>              no-disk-flushes ;
>              no-disk-barrier;
>              c-plan-ahead 10;
>              c-fill-target 24M;
>              c-min-rate 10M;
>              c-max-rate 100M;
>     }.
>     net {
>              # max-epoch-size          20000;
>              max-buffers             36k;
>              sndbuf-size            1024k ;
>              rcvbuf-size            2048k;
>     }
>     }
>
>     *
>     Then simply restart drbd by:
>     *
>     Code:
>
>     /etc/init.d/drbd restart
>
>     *
>     And thats it. Syncing should be at 800mbits :)
>     In you have 10GBit just change c-max-rate to 1000M
>     Play with those settings :)
>
>     -- 
>     sky-link.net <http://sky-link.net> Tytus Rogalewski
>     eMail: tytanick at gmail.com <mailto:tytanick at gmail.com>
>     strona: http://sky-link.net
>     kom. 790 202 300
>     gg: 210533
>     skype: tytanick
>     NIP: 9570976234
>     REGON: 221197388
>
>
>     2016-06-21 8:02 GMT+02:00 Tytus Rogalewski <tytanick at gmail.com
>     <mailto:tytanick at gmail.com>>:
>
>         Drbd 8.x was totally fine.
>         9.0 in proxmox 4.2 is bugged i suppose.
>         I will try 8.x in 4.2
>
>         21 cze 2016 01:00 "Adam Goryachev"
>         <mailinglists at websitemanagers.com.au
>         <mailto:mailinglists at websitemanagers.com.au>> napisa?(a):
>
>             On 20/06/16 22:47, Emmanuel Florac wrote:
>
>                 Le Sun, 19 Jun 2016 08:56:18 +0200
>                 Piero Baudino <pbaudino at formatnet.it
>                 <mailto:pbaudino at formatnet.it>> écrivait:
>
>                     Hi all,
>
>                     I have a cluster with 2 PVE servers with DRBD9.
>                     The replicated volume is about 7 Tb, the disks are
>                     7200 rpm near line
>                     SAS. The communications channel for DRBD9 is done
>                     via 4 Gbit
>                     dedicated network cards in bond configuration
>                     (round robbin).
>                     I have created a VM with 1 Tb disk for test
>                     purpuse in the DRBD
>                     volume. After that, DRBD starts sync...
>                     The sync speed is about 10 Mbyte/sec
>                     The effective network speed misured with IPERF is
>                     about 3,6 Gbit/sec
>
>                     Is there a way to increse sync speed ?
>                     All the docs about this topic are related to 8.x.y
>                     DRBD version....
>
>
>                 See this thread maybe:
>                 http://www.gossamer-threads.com/lists/drbd/users/27675
>
>
>             Stupid question perhaps, but doesn't that thread also
>             relate to 8.4.x rather than 9.0.x ?
>
>
>             -- 
>             Adam Goryachev Website Managers www.websitemanagers.com.au
>             <http://www.websitemanagers.com.au>
>             _______________________________________________
>             drbd-user mailing list
>             drbd-user at lists.linbit.com <mailto:drbd-user at lists.linbit.com>
>             http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
>
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20160621/bccfc0be/attachment.htm>


More information about the drbd-user mailing list