<html><head><style type='text/css'>p { margin: 0; }</style><style type='text/css'>body { font-family: 'Times New Roman'; font-size: 10pt; color: #000000}</style></head><body>Hi All,<br><br>Putting together a system to compete with netapp and bluearc. I say 90% of it can be done with drbd and supermicro + LVM + ext3.<br><br>Problem: slow write when server1 and server2 are connected.<br><br>Server profile:<br>server1: dualcore, 2gb ram, 1x500gb with 10 gb drbd0 on /dev/sda6. ext3 formatted.<br>server2: dualcore, 4gb ram, 1x500gb with 10 gb drbd0 on /dev/sda6. ext3 formatted.<br><br>Network: 1 gigabit connections, server to server direct.<br><br>OS: Centos5.2 x 64<br><br>Config file:<br>==============<br>global { usage-count yes; }<br>common { <br> syncer { <br> rate 50M; <br> } <br>}<br>resource r0 {<br> protocol C;<br> disk {<br> no-disk-flushes;<br> no-md-flushes;<br> }<br> net {<br> cram-hmac-alg sha1;<br> shared-secret "Foofsfafsfdtory";<br> }<br> device /dev/drbd0;<br> disk /dev/sda6;<br> meta-disk internal;<br><br> on server1 {<br> address 10.12.62.10:7789;<br> }<br><br> on server2 {<br> address 10.12.63.20:7789;<br> }<br>}<br>==============<br><br>Here are my benchmarks: <br>dd if=/dev/zero bs=4096 count=10000 oflag=dsync of=/data/file1<br>
<br>DRBD Active and connected:<br>40960000 bytes (41 MB) copied, 24.8387 seconds, 1.6 MB/s<br><br>DRBD Active and NOT connected:<br>40960000 bytes (41 MB) copied, 10.8259 seconds, 3.8 MB/s<br><br>File system partition, NON DRBD, same disk:<br>40960000 bytes (41 MB) copied, 10.3686 seconds, 4.0 MB/s<br><br>These are very consistent and repeatable.<br><br>DRBD: built from source 8.2.6. installed as a module. tried the rpms from yum, similar results.<br>
<br>Output of /proc/drbd:<br>version: 8.2.6 (api:88/proto:86-88)<br>GIT-hash: 3e6982sdfsdfsdffs28c1bfdf7d647169eba7d2eb4 build by root@server2, 2008-09-24 13:00:25<br> 0: cs:WFConnection st:Primary/Unknown ds:UpToDate/DUnknown C r---<br> ns:10055268 nr:0 dw:1680584 dr:9775393 al:112 bm:598 lo:0 pe:0 ua:0 ap:0 oos:211180<br><br>Both hard drives are the same speed, confirmed with the same test from above.<br><br>I've recompiled network drivers, recompiled drbd from source, now thinking to recompile the kernel.<br><br>Checked that network is clear with wireshark, no errors. I've adjusted the buffers and other settings in drbd.conf and nothing i seem to do makes any difference. <br><br>Would welcome any ideas or suggestions.<br><br>Best Regards,<br><br>Artur<br>eBoundHost.com<br>http://www.eboundhost.com<br></body></html>