[DRBD-user] Speed Sync seems to be limited on gigabit ethernet

Marcello Lupo mlist at itspecialist.it
Wed Jun 20 15:19:21 CEST 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Hi to all,
i'm making a new installation on SLES 10 x86_64 on 2 HP Proliant DL380 
multiprocessor servers with 2 Ultra 320 SCSI disks of 146 Gb in raid1 on 
each using Smart Array 6i hardware raid controller. Both servers have 
2GB ram.
The 2 servers are connected on public network on eth0 and connected with 
cross cable on eth1 dedicated for sync of drbd, both Gigabit ethernet.
When i make scp of one big file from one server to the other i reach 
50Mb/sec speed. If i force the sync of the partitions to drbd with 
invalidate_remote it do not go over 12 Mb/sec.

version: 0.7.21 (api:79/proto:74)
SVN Revision: 2341 build by lmb at dale, 2006-08-10 14:19:54
  0: cs:SyncSource st:Primary/Secondary ld:Consistent
     ns:41370104 nr:0 dw:0 dr:41371732 al:0 bm:6109 lo:0 pe:201 ua:407 ap:0
         [==============>.....] sync'ed: 70.5% (16949/57349)M
         finish: 0:22:15 speed: 12,968 (12,672) K/sec

It seems like it is limited to network connection of 10Mbit.
i tried with rate option to 50M,40M,30M and it is the same.
I'm using protocol C.

Partitions are as follows:

       Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14         536     4200997+  82  Linux 
swap / Solaris
/dev/cciss/c0d0p3             537        3147    20972857+  83  Linux
/dev/cciss/c0d0p4            3148       17848   118085782+   f  W95 
Ext'd (LBA)
/dev/cciss/c0d0p5            3148       10458    58725576   83  Linux
/dev/cciss/c0d0p6           10459       17769    58725576   83  Linux
/dev/cciss/c0d0p7           17770       17848      634536   83  Linux

Partitions for the system are p1/p2/p3, partitions to be synced are p5 
and p6 (now testing only with p5), i'm using the last partition p7 as 
metadisk partition.

DRBD version 0.7.21 and kernel x86_64

If you have any clue on the possible cause please let me know. May be 
this speed is normal for this kind of environment or may be i have to 
tune some network parameters.

More information about the drbd-user mailing list