[Drbd-dev] Behaviour of verify: false positives -> true positives

[Lars Ellenberg]

On Wed, Oct 01, 2008 at 11:16:41AM +0200, Thomas Schoebel-Theuer wrote:
> Hi Lars,
> 
> I have two different patches, both based on your git version from 2008-09-11.

great.

> The first one implements the checksumming method while the second one 
> implements the copy method proposed by you.
> 
> I found that your suggestion was much more easier to implement than the 
> checksumming method. However, the copy method works completely silently (you 
> get no response whether a race on the data is actually taking place or not), 
> while the checksumming method writes reports to the kernel log (so you can 
> observe the behavior).

I briefly had a look at the copy method.

as private_bio is a clone of master_bio,
it shares its bvec, thus the pages.

you assign the copy pages to the bvec->bv_page
(you have to do that, master_bio->bvec->bv_page is used in sendpage)
but you don't restore the original pages before completion,
even free the pages already.
you
 * potentially leak the original pages
 * potentially double free the copy pages
as the submitter (file system, page cache, whatever)
in its completion callback may reference or free those.

did you try this on a debug kernel,
memory poisoning and page allocation debugging switched on?
(of course, that is not the kernel you want to use for
 performance benchmarks).

you need to remember the original pages, and you need to restore them
before completion of the master bio.

for diagnostics, you can "memcmp" on that occasion as well.

and you probably need to increase the "emergency reserve" amount of
pre-allocated pages, to avoid resource starvation deadlocks.

you should use copy_page or copy_highpage.

for performance impact, assuming that your memory bandwidth is much
higher than your IO bandwidth, this will "only" increase CPU cycles.
which is probably ok on a generic file server, but may become critical
when running a data base directly on drbd.
so, when doing benchmarks, have a look at the cpu usage as well.

I'll have a look at the checksumming method later.

thanks again,

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.