Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
> On 20 Oct 2015, at 20:47, Lars Ellenberg <lars.ellenberg at linbit.com> wrote: > > On Tue, Oct 20, 2015 at 01:07:48PM +0200, Joeri Casteels wrote: >> Dear All, >> >> >> Doing a new set-up of DRBD on Ubuntu LTS 14.04 with stock DRBD >> When Setting to protocol A i get double the bandwidth as protocol C anybody an idea? not changing anything else of the settings except the protocol >> >> PROTOCOL C: >> d if=/dev/zero of=/dev/drbd0 bs=1M count=20000 oflag=direct >> 20000+0 records in >> 20000+0 records out >> 20971520000 bytes (21 GB) copied, 46.3583 s, 452 MB/s >> >> PROTOCOL A: >> dd if=/dev/zero of=/dev/drbd0 bs=1M count=20000 oflag=direct >> 20000+0 records in >> 20000+0 records out >> 20971520000 bytes (21 GB) copied, 23.8356 s, 880 MB/s > > dd does: > write one block size # which takes some time > write next bs # which again takes some time > ... > > Without oflag=direct, > you write into the linux page cache, the write itself takes microseconds > (at most), and you get many such writes per second, resulting usually in > a throughput of several gigabytes per second (until you hit the > boundaries of the page cache and are throttled down to the rate of the > background write-out). > > With oflag=direct, > it bypasses the linux page cache, > the single write call takes longer, > and you do no longer get that many writes per second. > > With protocol B and C, the single write latency even includes > the network latency and remote disk latency, possibly flush latencies, > and you get even less writes per second. > > Throughput is concurrency times request size by request completion latency. > > dd has no concurrency, in the given example, request size is 1M, > if you get 450 MB/s, your latency apparently is in the order of 2.3 ms. > > If you want more throughput, > you need to decrease latency, or increase concurrency. Any suggestions on how to improve that then? > > > -- > : Lars Ellenberg > : http://www.LINBIT.com | Your Way to High Availability > : DRBD, Linux-HA and Pacemaker support and consulting > > DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. > __ > please don't Cc me, but send to list -- I'm subscribed > _______________________________________________ > drbd-user mailing list > drbd-user at lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user