Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
2016-05-20 17:29 GMT+08:00 Lars Ellenberg <lars.ellenberg at linbit.com>:
> * no "stable pages"
>
> If the content of the buffer that was originally submitted changes
> after that has been submitted, but before we notify completion, the
> local IO subsystem and the network may DMA (or otherwise use)
> different generations of that buffer to their respective destinations.
>
> Assumption would be that if someone "changes" such a buffer while it
> is being written, that someone would later re-submit the changes, and
> eventually the affected block would become identical again,
> because there would be no further change during the last write.
>
> In that case this would only be a temporary state.
>
> But sometimes that re-submit does not really happen.
> Which may even be "legal", for example if that was for some
> already-deleted temp file, file system optimizations could
> decide that there is no reason to do the actual write-out,
> unless there is memory pressure. Or some application manages
> its own data buffers, and uses direct-IO to submit them,
> then does further changes, without explicitly re-submitting,
> and without waiting for the write to be complete.
> Or someone else has mmap'ed the same stuff,
> and keeps modifying it, without ever explicitly syncing.
> BTW, behaviour changes with kernel versions.
>
> In which case this can become a permanent state -- until the next
> stable write to the same block.
>
> Differing data caused by lack of stable pages is usually harmless,
> because it typically affects areas which are "unreachable".
> But I know of no easy-to-automate way to decide which of these caused
> the data to be different just by looking at it during online-verify.
>
> So there.
> What next?
Hi:
thanks a lot for so detailed explanation. the information should
put into document or at least some blog.
the "no stable page" part is something maybe I am suffering, since
the under layer of drbd is software raid1. I know software raid1 may
have inconsistent sectors between disks. previously I only know that
swap partition or file will cause the software raid1 inconsistent. but
it seems there are other situations.
I now try to force repair the software raid1, to see if I can make
drbd verify more happy. if this is the reason, then next time I should
use software raid5 or raid6 for drbd to make sure data-integrity.
thanks again for your kindly help!!
Regards,
tbskyd