<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Artur,<div><br><div><div>On 25/09/2008, at 7:10 AM, eBoundHost: Artur wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="font-family: 'Times New Roman'; font-size: 10pt; color: rgb(0, 0, 0); ">Hi All,<br><br>Putting together a system to compete with netapp and bluearc. I say 90% of it can be done with drbd and supermicro + LVM + ext3.<br><br>Problem: slow write when server1 and server2 are connected.<br><br>Here are my benchmarks:<span class="Apple-converted-space"> </span><br>dd if=/dev/zero bs=4096 count=10000 oflag=dsync of=/data/file1<br><br>DRBD Active and connected:<br>40960000 bytes (41 MB) copied, 24.8387 seconds, 1.6 MB/s<br><br>DRBD Active and NOT connected:<br>40960000 bytes (41 MB) copied, 10.8259 seconds, 3.8 MB/s<br><br>File system partition, NON DRBD, same disk:<br>40960000 bytes (41 MB) copied, 10.3686 seconds, 4.0 MB/s<br><br>These are very consistent and repeatable.<br></div></span></blockquote><br></div><div>I am in no way surprised by these results. First note that you are writing out 4K blocks, then forcing a sync. So you can see already this is a slow operation from the non-DRBD results. As each time you write 4k you need to wait for it to sync before it can do the next 4k.</div><div><br></div><div>Problem with doing a force sync on DRBD is you introduce network latency into the 'sync' pipeline.. so it writes 4K.. sends a sync and you have to wait for that to go over the network, get synced, and come back, before it can move onto the next 4k block.</div><div><br></div><div>There isn't a whole lot you can do about this, but im not sure there is a 'fantastic' benchmark for real workload situations... anyway.</div><div><br></div><div>You could try some crazy things to reduce your latency.. remove switches try better drivers and stuff (does that really work!? - but ultimately I think you will need to stop doing 4k synced writes.. or get some low latency interconnect gear - but I suspect it's rather expensive:</div><div><a href="http://fghaas.wordpress.com/2008/09/12/got-drbd-got-dolphin-combine-em/">http://fghaas.wordpress.com/2008/09/12/got-drbd-got-dolphin-combine-em/</a></div><div><br></div><div>Something about this seem silly/wrong to you?</div><div><br></div>Thanks,<br><div> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><div><div>-- </div><div>Trent Lloyd</div></div></div></div></div></span> </div><br></div></body></html>