[DRBD-user] DRBD 8.4.2: "drbdadm verify" just do not work

Markus Müller drbd2012 at priv.de
Sun Sep 23 12:18:57 CEST 2012

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Hello DRBD Users,

I have a drbd two-node setup running, and got alarmed by the LINBIT mail 
about sync problems with newer kernels. So I updated to 8.4.2 and tried 
to make sure anything is fine now.

Even if the mail of LINBIT says that no action is required after 
upgrading, I tried the "drbdadm verify" feature. And it found "oos", 
means blocks not in sync. I thought okay, good that you thought for 
that, and tried to fix this as described in the LINBIT mail by doing 
"drbdadm disconnect/connect". It synced the found "oos:" and I thought 
everything is fine, so I did rerun the "drbdadm verify" to just be sure. 
And I saw... just found more "oos:"! I did again a "drbdadm 
disconnect/connect" but there were still more "oos:" after the next 
"drbdadm verify". I made this some loops and saw that this is not 
working at all to fix this!

To spot the estimation of damage I stopped drbd, made a network block 
device via "nbd-client / nbd-server" and run the following perl script 
on the devices on primary and secondary from drbd directly. The perl 
program verifies the differences in 1 MB blocks, and writes a line for 
each 1 MB block wich is not fully in sync by each bit. This shows that 
in the 500 GB partition there are still 18 blocks of 1 MB which have one 
or more bits different on my secondary to my primary. The rerun of the 
"drbdadm verify/disconnect/connect" don't change anything here!

By the way: Further I saw that the increasing of the MTU size of the 
network interface above 1500 leads that drbd cannot be connected at all! 
Network block device of linux just works fine, also with a MTU of 9.000 
and is in RAID1 much more faster than drbd ever was. Hmm... Does anybody 
see real bennefits of drbd against a solution with nbd and raid1?

$deva = "/dev/md2";
$devb = "/dev/nbd0";
$| = 1;
$size = 1*1000*1000;
$gb = 1000*1000*1000;
$pos = 0; $bad = 0;
system("blockdev --setra 200000 ".$deva);
system("blockdev --setra 200000 ".$devb);
open(A, "<", $deva) || die($!);
open(B, "<", $devb) || die($!);
while((my $reada = sysread(A, $bufa, $size)) == $size) {
    my $readb = sysread(B, $bufb, $size);
    die "a != b\t'".$reada."!=".$readb."'"
       if ($reada != $readb);
    $bad++ if ($bufa ne $bufb);
    print $bad." bad   ".(int(++$pos)/($gb/$size))." GB ".(($lastbad == 
$bad) ? "\r" : "\n");
    $lastbad = $bad;

My primary is ubuntu with kernel 3.0, my secondary is debian wWheezy 
with kernel 3.2. Both have drbd 8.4.2 and are 64 bit.

Much regards,
Markus Müller

More information about the drbd-user mailing list