[DRBD-user] Protocol A vs Protocol C performance issue?

Joeri Casteels joeri.casteels at intec.ugent.be
Tue Oct 20 20:52:27 CEST 2015

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.



> On 20 Oct 2015, at 20:47, Lars Ellenberg <lars.ellenberg at linbit.com> wrote:
> 
> On Tue, Oct 20, 2015 at 01:07:48PM +0200, Joeri Casteels wrote:
>> Dear All,
>> 
>> 
>> Doing a new set-up of DRBD on Ubuntu LTS 14.04 with stock DRBD 
>> When Setting to protocol A i get double the bandwidth as protocol C anybody an idea? not changing anything else of the settings except the protocol
>> 
>> PROTOCOL C:
>> d if=/dev/zero of=/dev/drbd0 bs=1M count=20000 oflag=direct
>> 20000+0 records in
>> 20000+0 records out
>> 20971520000 bytes (21 GB) copied, 46.3583 s, 452 MB/s
>> 
>> PROTOCOL A:
>> dd if=/dev/zero of=/dev/drbd0 bs=1M count=20000 oflag=direct
>> 20000+0 records in
>> 20000+0 records out
>> 20971520000 bytes (21 GB) copied, 23.8356 s, 880 MB/s
> 
> dd does:
>  write one block size # which takes some time
>  write next bs # which again takes some time
>  ...
> 
> Without oflag=direct,
> you write into the linux page cache, the write itself takes microseconds
> (at most), and you get many such writes per second, resulting usually in
> a throughput of several gigabytes per second (until you hit the
> boundaries of the page cache and are throttled down to the rate of the
> background write-out).
> 
> With oflag=direct,
> it bypasses the linux page cache,
> the single write call takes longer,
> and you do no longer get that many writes per second.
> 
> With protocol B and C, the single write latency even includes
> the network latency and remote disk latency, possibly flush latencies,
> and you get even less writes per second.
> 
> Throughput is concurrency times request size by request completion latency.
> 
> dd has no concurrency, in the given example, request size is 1M,
> if you get 450 MB/s, your latency apparently is in the order of 2.3 ms.
> 
> If you want more throughput,
> you need to decrease latency, or increase concurrency.

Any suggestions on how to improve that then?

> 
> 
> -- 
> : Lars Ellenberg
> : http://www.LINBIT.com | Your Way to High Availability
> : DRBD, Linux-HA  and  Pacemaker support and consulting
> 
> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
> __
> please don't Cc me, but send to list   --   I'm subscribed
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user




More information about the drbd-user mailing list