Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
> -----Original Message----- > From: drbd-user-bounces at lists.linbit.com > [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Ralf Gross > Sent: Monday, January 15, 2007 12:56 PM > To: drbd-user at lists.linbit.com > Subject: Re: [DRBD-user] drbd performance with GbE in connected mode > > Ross S. W. Walker schrieb: > > > Both Raid's are able to write with >120MB/s in disconnected mode. > > > I'm just wondering because I have seen people on the list that get > > > ~95MB/s with a similar setup. > > > > There's lies, damn lies, then benchmarks. Pump a big enough > block size > > to the storage till you get the throughput you like. Some > people throw > > 1MB block size and get 115MB/s throughput. > > > > I always like to use the block sizes my application uses. > If it's ext2 > > then 4K is a good benchmark, but different apps use different block > > sizes. > > > > Always ask the block size and io pattern for the benchmarks > people throw > > around. > > It seems that > > a. changing the net parameters did help > > sndbuf-size 240k; > max-buffers 20480; > max-epoch-size 16384; > unplug-watermark 20480; > > b. changing the bonding mode of the interfaces from balance-rr to > balance-xor did help too. > > I now get about 85MB/s. Maybe that's by accident, but I could watch > the write performance go down when increasing the sndbuf-size. Well I can see increasing sndbuf as increasing latency, so it makes sense that decreasing it would also decrease latency a bit, why not try 128K and see where that puts you. If you have direct connections and a fast network, there is no real need to have a large sndbuf. If I was using Prot A and a slow network of T1s then I would use a very large sndbuf. Statistically speaking though when doing a benchmark over a short period of time 82-83-85 MB/s are about the same. I find that a 15 minute run will normally get rid of the 3-5 MB/s swings between runs and narrow it down to 1-2 MB/s swings. It looks like you are approaching the part of tuning were you are receiving diminishing returns and will need to do more and more tuning to squeeze less and less out, so I would say that 85 MB/s is what your gonna see unless you can find a way to run drbd with multiple paths, which I don't think it has the capability to do. Well let me know if you can squeeze any more out of it. You might want to see if there is any filesystem optimizations you can do now to get some extra performance out of it. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.