<html><head><meta http-equiv="content-type" content="text/html; charset=GB2312"><style>body { line-height: 1.5; }body { font-size: 10.5pt; font-family: 微软雅黑; color: rgb(0, 0, 0); line-height: 1.5; }</style></head><body>
<div><span></span>Hi,</div><div><br></div><div>I use a PCI-E flash card and 10Gb ib card to test drbd9. Test tool is fio. I tested 4k randread, 4k read, 512k read, 4k randwrite, 4k write, 512k write, on flash card device, drbd device in standalone status, and drbd device in sync status.</div><div><br></div><div><!--StartFragment-->
<table width="673" height="189" style="border-collapse:collapse;width:504.75pt;">
<colgroup><col width="244">
<col width="112">
<col width="116">
<col width="201">
</colgroup><tbody><tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">1. read (bandwidth/iops)</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;"><br></td><td height="19" style="font-size: 11pt; font-family: 宋体; vertical-align: middle;"></td><td height="19" style="font-size: 11pt; font-family: 宋体; vertical-align: middle;"></td>
</tr>
<tr height="19">
<td height="19" style="font-size: 11pt; font-family: 宋体; vertical-align: middle;"></td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">/dev/dfa</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">/dev/drbd0</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">/dev/drbd0 standalone</td>
</tr>
<tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">4k randread</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">630m/158k</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">445m/111k</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">428m/107k</td>
</tr>
<tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">4k read</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">804m/201k</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">441m/110k</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">423m/105k</td>
</tr>
<tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">512k read</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">2282m</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">2457m</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">2279m</td>
</tr>
<tr height="18">
<td height="18" style="font-size: 11pt; font-family: 宋体; vertical-align: middle;"></td><td height="18" style="font-size: 11pt; font-family: 宋体; vertical-align: middle;"></td><td height="18" style="font-size: 11pt; font-family: 宋体; vertical-align: middle;"></td><td height="18" style="font-size: 11pt; font-family: 宋体; vertical-align: middle;"></td>
</tr>
<tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">2. write</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">/dev/dfa</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">/dev/drbd0</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">/dev/drbd0 standalone</td>
</tr>
<tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">4k randwrite</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">1424m/364k</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">93m/23k</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">328m/82k</td>
</tr>
<tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">4k write</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">1455m/372k</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">286m/71k</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">369m/92k</td>
</tr>
<tr height="19">
<td class="et1" x:str="" height="19" width="244" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 183pt; font-family: 宋体;">512k write</td><td class="et1" x:str="" height="19" width="112" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 84pt; font-family: 宋体;">1785m</td><td class="et1" x:str="" height="19" width="116" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 87pt; font-family: 宋体;">860m</td><td class="et1" x:str="" height="19" width="201" style="font-size: 11pt; vertical-align: middle; height: 14.25pt; width: 150.75pt; font-family: 宋体;">1799m</td>
</tr>
</tbody></table>
<!--EndFragment--></div><div><br></div>
<div>Seems write performance droped a lot. Bandwitdh of 4k randwrite is only 93M. I read user guide and tried the recommanded parameter, but it's still no use. Anyone can give some advise?</div><div><br></div><div>My fio command:</div><div><span style="color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);">fio --filename=/dev/drbd0 --direct=1 --rw=randwrite --randrepeat=0 --ioengine=libaio --group_reporting --bs=4k --iodepth=4 --numjobs=16 --runtime=20 --name=fiotest1 --size=20g<br></span></div><div><span style="color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);"><br></span></div><div>My r0.res config:</div><div><span style="color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);">~<br>[root@app2 drbd.d]# vi r0.res <br>resource r0 {<br> disk {<br> on-io-error detach;<br> disk-flushes no;<br> disk-barrier no;<br> c-plan-ahead 40;<br> c-fill-target 24M;<br> c-min-rate 10M;<br> c-max-rate 1000M;<br> al-extents 6007;<br> }<br><br> net {<br> protocol A;<br> max-buffers 20000;<br> max-epoch-size 20000;<br> sndbuf-size 0;<br> rcvbuf-size 0;<br> }<br> device /dev/drbd0;<br> disk /dev/dfa;<br> meta-disk internal;<br> on app2 {<br> address 10.1.10.104:7789;<br> }<br> on app3 {<br> address 10.1.10.105:7789;<br> }<br></span></div><div><br></div><div>Thanks!</div><div>Richard</div><hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span><div style="MARGIN: 10px; FONT-FAMILY: verdana; FONT-SIZE: 10pt"><div>ligong_qiu@sina.com</div></div></span></div>
</body></html>