[DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

[Stanislav German-Evtushenko]

Just to make things clearer. These results are not false-positive, they are
real. False-positive also happen but rarely. I do check for false-positive
using the following script:
----------------------------------------------------------
#!/bin/bash

# Usage: cat /var/log/kern.log | drbd_out_of_sync_compare.sh

#echo 'Mar 31 10:24:04 virt1 kernel: block drbd0: Out of sync:
start=1036171232, size=8 (sectors)'
while read line; do
        if [[ $line =~ Out\ of\ sync:\ start=([0-9]+),\ size=([0-9]+) ]];
then
                start=${BASH_REMATCH[1]}
                size=${BASH_REMATCH[2]}
                echo $start - $size
                sum1=$(ssh 10.10.10.1 dd iflag=direct if=/dev/drbd0 bs=512
skip=$start count=$size 2>/dev/null < /dev/null | md5sum | awk '{print $1}')
                sum2=$(ssh 10.10.10.2 dd iflag=direct if=/dev/drbd0 bs=512
skip=$start count=$size 2>/dev/null < /dev/null | md5sum | awk '{print $1}')
                if [[ $sum1 = $sum2 ]]; then
                        echo OK: $sum1 - $sum2
                else
                        echo ERR: $sum1 - $sum2
                        ssh 10.10.10.1 dd iflag=direct if=/dev/drbd0 bs=512
skip=$start count=$size 2>/dev/null < /dev/null > /tmp/${start}_${size}_1
                        ssh 10.10.10.2 dd iflag=direct if=/dev/drbd0 bs=512
skip=$start count=$size 2>/dev/null < /dev/null > /tmp/${start}_${size}_2
                fi
        fi
done
----------------------------------------------------------

And results look like:
----------------------------------------------------------
253182888 - 16
OK: 0829f71740aab1ab98b33eae21dee122 - 0829f71740aab1ab98b33eae21dee122
253182904 - 8
OK: 620f0b67a91f7f74151bc5be745b7110 - 620f0b67a91f7f74151bc5be745b7110
253182952 - 8
OK: 620f0b67a91f7f74151bc5be745b7110 - 620f0b67a91f7f74151bc5be745b7110
253250344 - 8
OK: 620f0b67a91f7f74151bc5be745b7110 - 620f0b67a91f7f74151bc5be745b7110
253259336 - 8
OK: 620f0b67a91f7f74151bc5be745b7110 - 620f0b67a91f7f74151bc5be745b7110
719214256 - 8
OK: 0132ffdc961a93ab39f3687b2168b326 - 0132ffdc961a93ab39f3687b2168b326
719214264 - 8
OK: e824f6f1a60c23fea04cfb5d080747c2 - e824f6f1a60c23fea04cfb5d080747c2
719299576 - 8
OK: a969c6562450baa0c5306fe89fe6d4f9 - a969c6562450baa0c5306fe89fe6d4f9
1085832880 - 8
OK: 9da8849288dcaa863b96d6cf5d9fee09 - 9da8849288dcaa863b96d6cf5d9fee09
1085972048 - 8
ERR: 708d5019b36d8bc6ef68fbdf431efbb3 - bffe661e808e1b42a4c5e1cad490ec0c
1085972072 - 8
ERR: a381fea0de0a34d01db0e4d7a9f9e824 - d1d6d30932ba15611cfac831e337e634
1086079632 - 8
ERR: 75e2d49f51a691998d1e9023b252aa51 - d367cfbd482fde9827ccef063b4b55a9
1086079528 - 8
ERR: 08637fb9b63c59db91c8179a22c9e4f7 - e421dc61e71d95bc63d9ab3fd834aa3e
1086079592 - 8
ERR: 483c379d346769f711721b4df154415b - 132f45d198b603e3f33cee750a21602d
1086079440 - 16
ERR: a8cb7da5e9da13d910b5afdbdb2721d9 - 717acb3d5b7403690f4f33340855a14c
1086128384 - 8
ERR: 31a1740dce4b305eb7a888a35de48ac8 - 53bc81a20e9ab5bddba35d742d3a7551
----------------------------------------------------------

Most of the time (99%) I see ERR for the swap space of virtual machines.

Best regards,
Stanislav
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20140130/621419c8/attachment.htm>