<html><head><style type='text/css'>p { margin: 0; }</style><style type='text/css'>body { font-family: 'Times New Roman'; font-size: 10pt; color: #000000}</style></head><body>----- "Trent Lloyd" <lathiat@bur.st> wrote:
<br>| Hi Artur,<div><br>| <div><div>On 25/09/2008, at 7:10 AM, eBoundHost: Artur wrote:</div><br class="Apple-interchange-newline"><blockquote><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><div style="font-family: 'Times New Roman'; font-size: 10pt; color: rgb(0, 0, 0);">Hi All,<br>| <br>| Putting together a system to compete with netapp and bluearc. I say 90% of it can be done with drbd and supermicro + LVM + ext3.<br>| <br>| Problem: slow write when server1 and server2 are connected.<br>| <br>| Here are my benchmarks:<span class="Apple-converted-space"> </span><br>| dd if=/dev/zero bs=4096 count=10000 oflag=dsync of=/data/file1<br>| <br>| DRBD Active and connected:<br>| 40960000 bytes (41 MB) copied, 24.8387 seconds, 1.6 MB/s<br>| <br>| DRBD Active and NOT connected:<br>| 40960000 bytes (41 MB) copied, 10.8259 seconds, 3.8 MB/s<br>| <br>| File system partition, NON DRBD, same disk:<br>| 40960000 bytes (41 MB) copied, 10.3686 seconds, 4.0 MB/s<br>| <br>| These are very consistent and repeatable.<br>| </div></span></blockquote><br>| </div><div>I am in no way surprised by these results. First note that you are writing out 4K blocks, then forcing a sync. So you can see already this is a slow operation from the non-DRBD results. As each time you write 4k you need to wait for it to sync before it can do the next 4k.</div><div><br>| </div><div>Problem with doing a force sync on DRBD is you introduce network latency into the 'sync' pipeline.. so it writes 4K.. sends a sync and you have to wait for that to go over the network, get synced, and come back, before it can move onto the next 4k block.</div><div><br>| </div><div>There isn't a whole lot you can do about this, but im not sure there is a 'fantastic' benchmark for real workload situations... anyway.</div><div><br>| </div><div>You could try some crazy things to reduce your latency.. remove switches try better drivers and stuff (does that really work!? - but ultimately I think you will need to stop doing 4k synced writes.. or get some low latency interconnect gear - but I suspect it's rather expensive:</div><div><a href="http://fghaas.wordpress.com/2008/09/12/got-drbd-got-dolphin-combine-em/" target="_blank">http://fghaas.wordpress.com/2008/09/12/got-drbd-got-dolphin-combine-em/</a></div><div><br>| </div><div>Something about this seem silly/wrong to you?</div><div><br>| </div>Thanks,<br>| <div> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><div style=""><div><div><div><div>-- </div><div>Trent Lloyd</div></div></div></div></div></span> </div><br>|<br>===============================<br><br>I don't know if it's silly, seems to be the faster the better! Once I have a better handle on this, we're going to be binding 3 or 4 interfaces to remove a bottleneck. Who knows, a rack full of hard drives may be just the thing for Dolphin. I think this is very cool and necessary to take DRBD to the enterprise.<br><br>To get back to what you're saying about the latency but take a look at this: <br>http://lists.linbit.com/pipermail/drbd-user/2007-December/008128.html<br><br>To my understanding we should be lagging by some percentage, but not by such a margin.<br><br>Best Regards,<br><br>Artur<br>eBoundHost.com<br>http://www.eboundhost.com<br></div></lathiat@bur.st></body></html>