<div dir="ltr">Hi All<div><br></div><div>I am new to drbd performance turning and I have been study (R0). </div><div>Aso I am browsing others effort in drbd-user archive (R1).</div><div>I was able to get 350MB/s rsync rate (R2) for two Centos 7.2 VMs(A and B) when they are on same LAN with turning from (R1) thread. </div><div><br></div><div>My goal is to have C Centos 7.2 VM paired with A VM that go over a fast WAN pipe(R3). but when I reuse B's drbd config and change the IP info. The rsync rate back to 30-40M rsync rate(See R4).</div><div><br></div><div>I tried the jumbo frame turning to raise MTU from 1500 to 8000(RH Support article recommend 8000, not 9000). But this change on VM A and C doesn't not improve rsync rate.</div><div><br></div><div><br></div><div>Does networking team need do any change on their swith/router for drbd case ?<br></div><div><br></div><div><br></div><div>References:</div><div>R0: <a href="https://www.drbd.org/en/doc/users-guide-84/p-performance">https://www.drbd.org/en/doc/users-guide-84/p-performance</a></div><div>R1: <a href="http://lists.linbit.com/pipermail/drbd-user/2016-January/022611.html">http://lists.linbit.com/pipermail/drbd-user/2016-January/022611.html</a></div><div>R2:</div><div><span id="docs-internal-guid-794ce825-a26c-a924-a228-f9c0eaacd9e1"><p style="margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><font color="#000000" face="Consolas"><span style="font-size:10.6667px;line-height:14.72px;white-space:pre-wrap">#this is between vmA(10.65.184.1) and vmB(10.65.184.3), same subnet.</span></font></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span id="docs-internal-guid-794ce825-a270-4265-0b88-52385d81e796"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmA ~]# ./drbd-pm-test.bash wandk0 # script from R0</span></span><br></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">testing wandk0 on /dev/drbd1</span></p><p style="margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span id="docs-internal-guid-794ce825-a26f-e0a7-1d97-e0fd471192f7"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></span><font color="#000000" face="Consolas"><span style="font-size:10.6667px;line-height:14.72px;white-space:pre-wrap"><br></span></font></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 8.00925 s, 67.0 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 1.72338 s, 312 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 1.84181 s, 291 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 1.66079 s, 323 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 1.69359 s, 317 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">testing wandk0 on backing device:/dev/centos/wandk0</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 0.504366 s, 1.1 GB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 0.550144 s, 976 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 0.502675 s, 1.1 GB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 0.473032 s, 1.1 GB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 0.470139 s, 1.1 GB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmA ~]# </span></p><div><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><br></span></div></span></div><div><br></div><div>R3:</div><div><br></div><div><span id="docs-internal-guid-794ce825-a26a-c3d1-f00a-ba8d13245728"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmC ~]# iperf3 -s -p 5900</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">warning: this system does not seem to support IPv6 - trying IPv4</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-----------------------------------------------------------</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Server listening on 5900</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-----------------------------------------------------------</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Accepted connection from 10.65.184.1(vmA), port 56750</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] local 10.64.5.245 port 5900 connected to 10.65.184.1 port 56754</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ ID] Interval Transfer Bandwidth</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 0.00-1.00 sec 73.9 MBytes 620 Mbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 1.00-2.00 sec 100 MBytes 842 Mbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 2.00-3.00 sec 107 MBytes 895 Mbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 3.00-4.00 sec 113 MBytes 947 Mbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 4.00-5.00 sec 117 MBytes 984 Mbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 5.00-6.00 sec 120 MBytes 1.01 Gbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 6.00-7.00 sec 123 MBytes 1.03 Gbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 7.00-8.00 sec 124 MBytes 1.04 Gbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 8.00-9.00 sec 124 MBytes 1.04 Gbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 9.00-10.00 sec 125 MBytes 1.04 Gbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 10.00-10.04 sec 5.25 MBytes 1.25 Gbits/sec</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">- - - - - - - - - - - - - - - - - - - - - - - - -</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ ID] Interval Transfer Bandwidth</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[ 5] 0.00-10.04 sec 1.11 GBytes 947 Mbits/sec receiver</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-----------------------------------------------------------</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Server listening on 5900</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-----------------------------------------------------------</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">^Ciperf3: interrupt - the server has terminated</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmC ~]# date</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Wed Jun 29 12:26:37 EDT 2016</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:36pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmC ~]#</span></p><div><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><br></span></div></span></div><div><br></div><div>R4: only getting 35MB/s when cross WAN network.</div><div><br></div><div><span id="docs-internal-guid-794ce825-a271-1b1d-b377-580b4440ed3b"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmA ~]# ./scratch-test.bash wandk0 # from R1</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> 1: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r-----</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Writing via wandk0 on /dev/drbd1</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 15.1859 s, 35.4 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><snipped></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 15.598 s, 34.4 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 16.9145 s, 31.7 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Writing directly into backing device:/dev/centos/wandk0</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 0.625202 s, 859 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><snipped></span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records in</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">1+0 records out</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">536870912 bytes (537 MB) copied, 0.566128 s, 948 MB/s</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmA ~]# date</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Thu Jun 30 13:07:16 EDT 2016</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:72pt"><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent">[root@vmA ~]#</span></p><div><span style="font-size:10.6667px;font-family:Consolas;color:rgb(0,0,0);vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><br></span></div></span><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">T.J. Yang</div>
</div></div>