<p dir="ltr"><br>
On 1 Jul 2016 3:48 pm, "T.J. Yang" <<a href="mailto:tjyang2001@gmail.com">tjyang2001@gmail.com</a>> wrote:<br>
><br>
> Hi All<br>
><br>
> I am new to drbd performance turning and I have been study (R0). <br>
> Aso I am browsing others effort in drbd-user archive (R1).<br>
> I was able to get 350MB/s rsync rate (R2) for two Centos 7.2 VMs(A and B) when they are on same LAN with turning from (R1) thread. <br>
><br>
> My goal is to have C Centos 7.2 VM paired with A VM that go over a fast WAN pipe(R3). but when I reuse B's drbd config and change the IP info. The rsync rate back to 30-40M rsync rate(See R4).<br>
><br>
> I tried the jumbo frame turning to raise MTU from 1500 to 8000(RH Support article recommend 8000, not 9000). But this change on VM A and C doesn't not improve rsync rate.<br>
><br>
><br>
> Does networking team need do any change on their swith/router for drbd case ?<br>
><br>
You need jumbo frames enabled on the switch too.</p>
<p dir="ltr">><br>
> References:<br>
> R0: <a href="https://www.drbd.org/en/doc/users-guide-84/p-performance">https://www.drbd.org/en/doc/users-guide-84/p-performance</a><br>
> R1: <a href="http://lists.linbit.com/pipermail/drbd-user/2016-January/022611.html">http://lists.linbit.com/pipermail/drbd-user/2016-January/022611.html</a><br>
> R2:<br>
><br>
> #this is between vmA(10.65.184.1) and vmB(10.65.184.3), same subnet.<br>
><br>
> [root@vmA ~]# ./drbd-pm-test.bash wandk0 # script from R0<br>
><br>
> testing wandk0 on /dev/drbd1<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 8.00925 s, 67.0 MB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 1.72338 s, 312 MB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 1.84181 s, 291 MB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 1.66079 s, 323 MB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 1.69359 s, 317 MB/s<br>
><br>
> testing wandk0 on backing device:/dev/centos/wandk0<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 0.504366 s, 1.1 GB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 0.550144 s, 976 MB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 0.502675 s, 1.1 GB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 0.473032 s, 1.1 GB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 0.470139 s, 1.1 GB/s<br>
><br>
> [root@vmA ~]#<br>
><br>
><br>
><br>
> R3:<br>
><br>
> [root@vmC ~]# iperf3 -s -p 5900<br>
><br>
> warning: this system does not seem to support IPv6 - trying IPv4<br>
><br>
> -----------------------------------------------------------<br>
><br>
> Server listening on 5900<br>
><br>
> -----------------------------------------------------------<br>
><br>
> Accepted connection from 10.65.184.1(vmA), port 56750<br>
><br>
> [ 5] local 10.64.5.245 port 5900 connected to 10.65.184.1 port 56754<br>
><br>
> [ ID] Interval Transfer Bandwidth<br>
><br>
> [ 5] 0.00-1.00 sec 73.9 MBytes 620 Mbits/sec<br>
><br>
> [ 5] 1.00-2.00 sec 100 MBytes 842 Mbits/sec<br>
><br>
> [ 5] 2.00-3.00 sec 107 MBytes 895 Mbits/sec<br>
><br>
> [ 5] 3.00-4.00 sec 113 MBytes 947 Mbits/sec<br>
><br>
> [ 5] 4.00-5.00 sec 117 MBytes 984 Mbits/sec<br>
><br>
> [ 5] 5.00-6.00 sec 120 MBytes 1.01 Gbits/sec<br>
><br>
> [ 5] 6.00-7.00 sec 123 MBytes 1.03 Gbits/sec<br>
><br>
> [ 5] 7.00-8.00 sec 124 MBytes 1.04 Gbits/sec<br>
><br>
> [ 5] 8.00-9.00 sec 124 MBytes 1.04 Gbits/sec<br>
><br>
> [ 5] 9.00-10.00 sec 125 MBytes 1.04 Gbits/sec<br>
><br>
> [ 5] 10.00-10.04 sec 5.25 MBytes 1.25 Gbits/sec<br>
><br>
> - - - - - - - - - - - - - - - - - - - - - - - - -<br>
><br>
> [ ID] Interval Transfer Bandwidth<br>
><br>
> [ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender<br>
><br>
> [ 5] 0.00-10.04 sec 1.11 GBytes 947 Mbits/sec receiver<br>
><br>
> -----------------------------------------------------------<br>
><br>
> Server listening on 5900<br>
><br>
> -----------------------------------------------------------<br>
><br>
> ^Ciperf3: interrupt - the server has terminated<br>
><br>
> [root@vmC ~]# date<br>
><br>
> Wed Jun 29 12:26:37 EDT 2016<br>
><br>
> [root@vmC ~]#<br>
><br>
><br>
><br>
> R4: only getting 35MB/s when cross WAN network.<br>
><br>
> [root@vmA ~]# ./scratch-test.bash wandk0 # from R1<br>
><br>
> 1: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r-----<br>
><br>
> Writing via wandk0 on /dev/drbd1<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 15.1859 s, 35.4 MB/s<br>
><br>
> <snipped><br>
><br>
> 536870912 bytes (537 MB) copied, 15.598 s, 34.4 MB/s<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 16.9145 s, 31.7 MB/s<br>
><br>
> Writing directly into backing device:/dev/centos/wandk0<br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 0.625202 s, 859 MB/s<br>
><br>
> <snipped><br>
><br>
> 1+0 records in<br>
><br>
> 1+0 records out<br>
><br>
> 536870912 bytes (537 MB) copied, 0.566128 s, 948 MB/s<br>
><br>
> [root@vmA ~]# date<br>
><br>
> Thu Jun 30 13:07:16 EDT 2016<br>
><br>
> [root@vmA ~]#<br>
><br>
><br>
><br>
> -- <br>
> T.J. Yang<br>
><br>
> _______________________________________________<br>
> drbd-user mailing list<br>
> <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>
> <a href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
><br>
</p>