Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
----- "Trent Lloyd" wrote: | Hi Artur, | On 25/09/2008, at 7:10 AM, eBoundHost: Artur wrote: Hi All, | | Putting together a system to compete with netapp and bluearc. I say 90% of it can be done with drbd and supermicro + LVM + ext3. | | Problem: slow write when server1 and server2 are connected. | | Here are my benchmarks: | dd if=/dev/zero bs=4096 count=10000 oflag=dsync of=/data/file1 | | DRBD Active and connected: | 40960000 bytes (41 MB) copied, 24.8387 seconds, 1.6 MB/s | | DRBD Active and NOT connected: | 40960000 bytes (41 MB) copied, 10.8259 seconds, 3.8 MB/s | | File system partition, NON DRBD, same disk: | 40960000 bytes (41 MB) copied, 10.3686 seconds, 4.0 MB/s | | These are very consistent and repeatable. | | I am in no way surprised by these results. First note that you are writing out 4K blocks, then forcing a sync. So you can see already this is a slow operation from the non-DRBD results. As each time you write 4k you need to wait for it to sync before it can do the next 4k. | Problem with doing a force sync on DRBD is you introduce network latency into the 'sync' pipeline.. so it writes 4K.. sends a sync and you have to wait for that to go over the network, get synced, and come back, before it can move onto the next 4k block. | There isn't a whole lot you can do about this, but im not sure there is a 'fantastic' benchmark for real workload situations... anyway. | You could try some crazy things to reduce your latency.. remove switches try better drivers and stuff (does that really work!? - but ultimately I think you will need to stop doing 4k synced writes.. or get some low latency interconnect gear - but I suspect it's rather expensive: http://fghaas.wordpress.com/2008/09/12/got-drbd-got-dolphin-combine-em/ | Something about this seem silly/wrong to you? | Thanks, | -- Trent Lloyd | =============================== I don't know if it's silly, seems to be the faster the better! Once I have a better handle on this, we're going to be binding 3 or 4 interfaces to remove a bottleneck. Who knows, a rack full of hard drives may be just the thing for Dolphin. I think this is very cool and necessary to take DRBD to the enterprise. To get back to what you're saying about the latency but take a look at this: http://lists.linbit.com/pipermail/drbd-user/2007-December/008128.html To my understanding we should be lagging by some percentage, but not by such a margin. Best Regards, Artur eBoundHost.com http://www.eboundhost.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20080924/d2a11aa1/attachment.htm>