[DRBD-user] drbd-0.7.0 with linux-2.4. slow?

[Lars Ellenberg]

/ 2004-07-30 01:34:12 +0200
\ Bernd Schubert:
> 
> > (latency figures...)
> 
> Data from a nfs-mounted directory:
> 
> 
> 1.) async exported :	2.) sync exported
>                    :	

[*] shows clearly an average overhead of nfs sync exports
    of factor 10! (max even almost factor 20...)
    these measure how long a write([record size]) call takes,
    in milliseconds.  (with -c -e, add in how long the fflush/fsync
    takes.) here record size is 4k, so the actual data transfer time is
    neglectable. that gives good figures for "round trip time" of

    drbd protocol C:
      application
         `nfs-client -> nfs-server -> drbd -> drbd-peer
                                       >dio      >dio   <--- disk io
         ,nfs-client <- nfs-server <- drbd <- drbd-peer
      application

    without -e and no sync export, you probably only get the time
    application -> nfs-client. 

    btw, with drbd proto B, you could reduce it to
      application
         `nfs-client -> nfs-server -> drbd -> drbd-peer
                                       >dio /
         ,nfs-client <- nfs-server <- drbd <
      application

    and drbd proto A is (when you use fsync)
      application
         `nfs-client -> nfs-server -> drbd -> [ drbd-peer ]
                                       >dio
         ,nfs-client <- nfs-server <- drbd
      application

I assume: -s 1m -r 4k ? with or without -c -e ?
drbd proto C ?

> N:          1024   :	N:          1024
> min:        1203   :	min:        1810
> avg:        1529   :	avg:       17103   <== [*]
> max:       37856   :	max:      695349
                        (roughly 1.5M/sec)

this may be -s 1g -r 1m ?
> N:          1024   :	N:          1024
> min:       91182   :	min:       95326
> avg:      110153   :	avg:      238704
> max:    18805320   :	max:      989322

for larger transfers, with sync export,
we have 1024k/0.24 sec ==> avg. throughput: 4.16M/sec


this may be -s 1g -r 4m ?
> N:           256   :	N:           256
> min:      364772   :	min:      495749
> avg:      384725   :	avg:      962792 [ factor 2.5]
> max:     4927510   :	max:     2502739

for even larger transfers, this still holds:
4M/0.96 sec ==> avg. throughput: 4.16M/sec

both max values degrade 

now, what is this:
only different drbd protocol?
other iozone options?

same assumptions: -s 1m -r 4k ? with or without -c -e ?
> N:          1024   :	N:          1024
> min:         983   :	min:        3957
> avg:        1494   :	avg:        4292 [still factor 4]
> max:        1944   :	max:      224765


this may be -s 1g -r 1m ?
> N:          1024   :	N:          1024
> min:       91151   :	min:       94015
> avg:      108411   :	avg:      410282
> max:    17054200   :	max:     1587899

this may be -s 1g -r 4m ?
> N:           256   :	N:           256
> min:      366288   :	min:      509567
> avg:      371261   :	avg:     1650030
> max:     1373654   :	max:     3948684

here numbers for short transfers have decreased
see the diagrams above.
but the numbers for large transfers has increased considerably...
throughput drops to 2.x MB/sec.  this is strange...

wait, this was with 0.7.0, right?
drbd 0.7.1 removed a serios handicap from protocols other than 'C' ...
and you already reported that you have an average throughput now
on a single nfs client of 9 - 10 MB/sec
(iirc, with nfs sync exports of reiserfs,
 drbd proto A, kernel 2.4.27-rcX, reiserfs)
so these figures should look very different now...

we should invent some generic "drbd_bench", I think, to get reliable
comparable values of different setups on different hardware...


	Lars Ellenberg

-- 
please use the "List-Reply" function of your email client.