Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Wed, Oct 21, 2015 at 11:42:52AM +0300, Vladislav Bogdanov wrote: > >>Throughput is concurrency times request size by request completion latency. > >> > >>dd has no concurrency, in the given example, request size is 1M, > >>if you get 450 MB/s, your latency apparently is in the order of 2.3 ms. > >> > >>If you want more throughput, > >>you need to decrease latency, or increase concurrency. > > > >Any suggestions on how to improve that then? > > There was advise on some of related lists (may be even this) to tune > following parameters: > net/max-buffers=80k > net/max-epoch-size=20000 Tuning max-buffers is certainly needed if you have a capable backend and replication link. Epoch size as only a very minor effect, typical epoch sizes will be rather in the small hundreds at most, anyways, so the new-epoch-because-we-exceeded-the-configured-maximum will very likely never be hit. > I additionally use > disk/c-max-rate=1500m > disk/c-fill-target=0 I strongly recommend using c-fill-target, and not c-delay target, but you may need to tune it to a value suitable for your setup. c-max-rate is by default at 100M, so yes, if you want more than that, you need to change it. To clarify, both are meaningful only in the context of a *resync*, so are meaningless for *replication*, *cannot* have any effect on the appliaction throughput during normal operation. > and get >800MB/s writes with protocol C over 10Gbps Ethernet > (ConnectX-3 in ethernet or IPoIB mode) with IO depth >=32 and IO > size 512k (I use disktest from LTP project). That is what 6 Intel DC > S3610 can provide in HW RAID-10 with disk cache turned on. > > And, of course, Lars is absolutely correct that it is merely > impossible to get high IO speeds with just one IO thread. -- : Lars Ellenberg : http://www.LINBIT.com | Your Way to High Availability : DRBD, Linux-HA and Pacemaker support and consulting DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed