[DRBD-user] Speed Sync seems to be limited on gigabit ethernet

[Marcello Lupo]

Hi,
thanks for answers.
Comments inline

Siim Vahtre wrote:
> On Wed, 20 Jun 2007, Marcello Lupo wrote:
>
>> If i force the sync of the partitions to drbd with invalidate_remote 
>> it do not go over 12 Mb/sec.
>
> This doesn't seem normal.
>
>
> Some obligatory questions:
>
> * Are you sure you actually have working fullduplex Gbit link? (check 
> with ethtool)
>
Yes.
Settings for eth1:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x000000ff (255)
        Link detected: yes

> * Are you sure you actually can write to your disks faster than 
> 12Mbytes/sec? (try with benchmarking tool that comes with drbd 'dm.c')
>
I tried to copy with scp from one machine to the other a 4GB file and it 
went at ~40Mb/s.

Tool you said dm.c the documentation says that is untouched from version 
0.6 and not working currently with 0.7 version.
SOme times in cat /proc/drbd i see some peak of speed around 18/20 
Mb/sec but it is very rare. the average is everytime around 14/15 Mb/sec.

Now i tried with bonnie++ and this is the result

Version  @version@      ------Sequential Output------ --Sequential 
Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  
/sec %CP
host0            4G 40987  99 85422  33 45074  11 53951  99 120726  15 
1152.6   2
host0            4G 41007  99 88090  34 44630  11 53664  99 122682  16 
1169.1   1
                    ------Sequential Create------ --------Random 
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
host0            16 17641  99 +++++ +++ 16958  99 19706 100 +++++ +++ 
16110  99
host0            16 15767  76 +++++ +++ 17099  99 19778  99 +++++ +++ 
16256  10

> * Are you sure nothing else is writing to disks at the same time?
>
Yes the servers are not used at the moment. I'm just making all 
preinstall tests of speed and so on before install.

>> i tried with rate option to 50M,40M,30M and it is the same.
>
> * Did you drbdadm adjust after changing config?
>
I made drbd completely stop and start to be sure all the times i changed 
the config.

>
> * Have you searched the mailing-list archives? I think I have seen 
> several questions like yours, maybe someone has already solved problem 
> like this.
>
>

I searched for something but found nothing that was usefull to me.. May 
be i didn't searched so well.

Thanks,
Bye,
Marcello