Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
/ 2007-01-08 23:39:37 -0600
\ Weilin Gong:
> Hi Lars,
>
> Thanks so much for your help.
>
> >* try again with a higher numruns, to get better statistics
> >
> >* try again with 0.7.22
> >* try again with drbd 8 (well, rcX, or svn trunk)
> >* try again with large "al-extends"
> > and then again in a second run...
> >
> I have not been able to run 0.7.22 and 8 yet due to our system constraint. I did try a large
> "al-extents" value (2570) with numruns 5 (it was 3 before), which resulted in at least the
> better max latency data:
>
> ┌─────────────────┬─────┬─────┬────┬─────┬───────┬───────┬─────────┬───────┬───────┬─────┐
> │Identifier │File │Blk │Num │Rate │Maximum│Avg │Max │Lat% > │Lat% > │CPU │
> │ │Size │Size │Thr │ │CPU% │Latency│Latency │2s │10s │Eff │
> ├─────────────────┼─────┼─────┼────┼─────┼───────┼───────┼─────────┼───────┼───────┼─────┤
> │Raw-device │5000 │4096 │4 │32.19│28.27% │0.347 │5956.10 │0.00244│0.00000│114 │
> ├─────────────────┼─────┼─────┼────┼─────┼───────┼───────┼─────────┼───────┼───────┼─────┤
> │Drbd-disconnected│5000 │4096 │4 │31.52│29.85% │0.365 │5054.72 │0.00080│0.00000│106 │
> ├─────────────────┼─────┼─────┼────┼─────┼───────┼───────┼─────────┼───────┼───────┼─────┤
> │Drbd-connected │5000 │4096 │4 │28.28│40.48% │0.394 │16223.82 │0.00122│0.00003│70 │
> └─────────────────┴─────┴─────┴────┴─────┴───────┴───────┴─────────┴───────┴───────┴─────┘
^^^^^^^
if you compare the percentage of latency > 2s, it actually decreases,
compared to the "raw" statistics.
so, if you envison a histogram of latency,
your raw device would look like
|
| ...
| .... .....
|.. ... .
+--------------------------------------------->
and the effect of drbd would be
| .
| ...
| .. ..
|.. .. . . . .
+------------------------------------------->
so it in fact seems to "compact" most of the requests,
but then has a few requests that have additional latency
because of transactional meta data updates.
> According the comments in drbd.conf, increasing "al-extents" would
> cause longer resync time if the primary crashes. Now
> here are my questions:
>
> 1) Is this the only side-effect of a large "al-extents" value?
yes. bigger al-extents --> less likely for a particular request to cause
additional meta data updates, but larger "active set" to be synced on
primary crash.
make sure you use 0.7.22,
we have had bugs in the apply-al-after-primary-crash code!
> 2) Does the activity log works the way similar to the ext3 file system
> journaling, each transaction is written to the log until
> it is full.
not at all. different concept altogether.
its more like a "cache" where we remember which "extents" are "active".
so in case we recover from a primary crash, we know that any extent
_not_ mentioned in this cache may use the dirty bitmap directly (it was not active),
and any extent that _is_ in this cache is assumed to be completely
dirty, because we cannot possibly know in more detail.
whenever the set of active elements changes, we need a meta data
transaction. as long as your requests stay in the same area,
we don't. the configuratino parameter al-extents tunes the size of this
"set of active elements".
> >* what is the real storage device? sdX ? mdX ? lvm something?
> >* what is the meta data storage device?
>
> Our storage device is the SCSI disk sda. LVM is being considered. The
> meta data resides on the same storage device (internal).
unless that "SCSI disk" is some better hw-raid box with lots of spindles,
you probably can decrease maximum latency by using external meta data on
some other disk (think seek time from data area to meta data area).
> One more thing. When running "tiobench" with DRBD in a connected mode,
> I noticed the throughput would increase ~4-6% if "jnettop" tool is
> turned on to monitor the interface carrying the traffic. Do we know
> what happens?
network voodoo.
:->
--
: Lars Ellenberg Tel +43-1-8178292-0 :
: LINBIT Information Technologies GmbH Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe http://www.linbit.com :
__
please use the "List-Reply" function of your email client.