All,<div><br></div><div>I'm setting up a drbd setup for my system, but the poor disk write performance is really throttling my system, even with Protocol A. The write speed needs to be 20MB/s to not bottleneck my system. Anyone has advice tuning for small writes? 10kb writes is the most common operations my system does.</div>
<div><br></div><div>It doesn't make much sense to me Protocol A performance is so much off native. Anyone has a theory?</div><div><br></div><div><div># for i in $(seq 5); do dd if=/dev/zero of=/dev/drbd1 bs=10K count=1000 oflag=direct; done</div>
<div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 8.43085 s, 1.2 MB/s</div><div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 8.48178 s, 1.2 MB/s</div>
<div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 8.99026 s, 1.1 MB/s</div><div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 8.80776 s, 1.2 MB/s</div>
<div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 10.2134 s, 1.0 MB/s</div><div><br></div><div><div>for i in $(seq 5); do dd if=/dev/zero of=/dev/sda2 bs=10K count=1000 oflag=direct; done</div>
<div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 0.17393 s, 58.9 MB/s</div><div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 0.157572 s, 65.0 MB/s</div>
<div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 0.157324 s, 65.1 MB/s</div><div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 0.157506 s, 65.0 MB/s</div>
<div>1000+0 records in</div><div>1000+0 records out</div><div>10240000 bytes (10 MB) copied, 0.157329 s, 65.1 MB/s</div></div><div><br></div><div>---------------------------------------------------------------------------------</div>
<div><br></div><div><div>dump of drbd.conf:</div><div><div>global {</div><div> usage-count yes;</div><div>}</div><div><br></div><div>common {</div><div> protocol A;</div><div> syncer {</div><div> rate 400M;</div><div>
}</div><div>}</div><div><br></div><div>resource r0 {</div><div><br></div><div> on mbus16 {</div><div> device /dev/drbd1;</div><div> disk /dev/sda2;</div><div> address <a href="http://10.20.76.55:7789">10.20.76.55:7789</a>;</div>
<div> meta-disk internal;</div><div> }</div><div> on mbus16-backup {</div><div> device /dev/drbd1;</div><div> disk /dev/md2;</div><div> address <a href="http://10.20.40.100:7789">10.20.40.100:7789</a>;</div>
<div> meta-disk internal;</div><div> }</div><div><br></div><div>}</div></div><div><br></div><div>----------------------------------------------------------------------------------------------------------------------------------------------------------------</div>
<div><br></div><div>Network latency is negligible:</div><div><div># ping mbus16-backup.snc1</div><div>PING mbus16-backup.snc1 (10.20.40.100) 56(84) bytes of data.</div><div>64 bytes from mbus16-backup.snc1 (10.20.40.100): icmp_seq=1 ttl=62 time=0.158 ms</div>
<div>64 bytes from mbus16-backup.snc1 (10.20.40.100): icmp_seq=2 ttl=62 time=0.187 ms</div><div>64 bytes from mbus16-backup.snc1 (10.20.40.100): icmp_seq=3 ttl=62 time=0.186 ms</div><div>64 bytes from mbus16-backup.snc1 (10.20.40.100): icmp_seq=4 ttl=62 time=0.192 ms</div>
<div>64 bytes from mbus16-backup.snc1 (10.20.40.100): icmp_seq=5 ttl=62 time=0.152 ms</div></div><div><br></div>-- <br>Lin Zhao<br>Data Platform Engineer<br>3101 Park Blvd, Palo Alto, CA 94306<br>
</div></div>