[Drbd-dev] Checksum based resync block size

[Eric Wheeler]

On Thu, 27 Jun 2019, Robert Altnoeder wrote:

> On 6/26/19 9:20 PM, Eric Wheeler wrote:
> > On Mon, 24 Jun 2019, Lars Ellenberg wrote:
> >
> >> As our in-sync/out-of-sync bitmap tracks 4k blocks,
> >> we want to compare 4k checkesums.
> >>
> >> Yes, that generates "a lot" of requests, and if these are not merged by
> >> some IO scheduler on the lower layers, that may seriously suck.
> >>
> >> make_ov_request() is what generates the online-verify requests.
> >>
> >> What we potentially could do is issue the requests in larger chunks,
> >> like (1 MiB) to the backends, then calculate and communicate the
> >> checksum per each 4k, as well as the result.
>
> > What if it were to calculate 1MiB chunks (configurable) and then 
> > invalidate all 4k bitmap entries in that 1MiB range if the hash 
> > mismatches?

This could also help resync by checksuming contiguous dirty bitmap entries 
(up to a chunk size limit) and resyncing the whole series instead of each 
4k block.
 
> Is your intention to reduce the number of packets with checksums that
> are being sent, and/or the number of checksum comparisons for the same
> amount of data?

Reduce the number of packets, but also, crypto transforms perform better 
on larger data chunks. You make another good point: fewer hash comparisons 
will help too.

> Both could have a positive impact on performance, but the question is,
> whether the difference is big enough to be relevant. On the other hand,
> hashing more data per checksum increases the chance of hash collisions.

I'm not too concerned about hash collisions.  That might be a problem with 
small CRC32-based sums, but assuming crypto hashes with a hash size of 
2^128, then the birthday paradox gives us a collision betwen some two 
hashes after 2^64 hashes---that is, any 2 hashes---not specifically the 
two being compared.  The probability of two chosen hashes colliding is 
even lower than 1/(2^64).

-Eric

> 
> br,
> Robert
> 
> _______________________________________________
> drbd-dev mailing list
> drbd-dev at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-dev
>