[DRBD-user] weird behavior with metadata

Alex Vasilenko aa.vasilenko at gmail.com
Mon Jan 26 15:03:54 CET 2015

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hello Lars,

Here's mine sectors data:

$ sfdisk -d

Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
# partition table of /dev/sda
unit: sectors


/dev/sda1 : start= 2048, size= 25165825, Id=83
/dev/sda2 : start= 25169920, size= 1048577, Id=83
/dev/sda3 : start= 26220544, size=2147483649, Id=83
/dev/sda4 : start=2173706240, size=3685736448, Id= f
/dev/sda5 : start=2173708288, size=1860173825, Id=83
/dev/sda6 : start=4033884160, size=1825558528, Id=83

So looks like you're right. Just 1 sector above your number.
Will try to compile HEAD drbd. Is it stable enough to be used on production
servers?

Regards,
Alex

On Mon Jan 26 2015 at 11:27:18 Lars Ellenberg <lars.ellenberg at linbit.com>
wrote:

> On Sat, Jan 24, 2015 at 04:01:11PM +0200, Alex Vasilenko wrote:
> > Hello,
> >
> > Having a strange behavior using drbd 8.4.5. After restart one partition
> of
>
> TL;DR: for the fix scroll ahead.
>
> You should always tell both kernel *and* userland versions.
>
>
> > Jan 24 14:42:39 de kernel: block drbd0: disk( UpToDate -> Failed )
>
> That may look scary, but is normal: during detach (down),
> we move via "Failed" to "Diskless" to avoid various potential races.
>
> > Now trying to restart backup partition:
> >
> > > $ sudo drbdadm down backup
> > > $ sudo drbdadm up backup
> > > strange bm_offset -65608 (expected: -65600)
> > > No valid meta data found
> > > Command 'drbdmeta 1 v08 /dev/sda3 internal apply-al' terminated with
> exit
> > > code 255
>
>
> > Somehow metadata is not there anymore (?!). Note warning saying "strange
> > bm_offset -65608 (expected: -65600)".
> > Ok, how about recreate metadata and resync full partition? Trying to wipe
> > old metadata shows that metadata is not present. Ok, let's pretend it's
> not
> > there:
> >
> > > $ sudo drbdadm create-md backup
> > > strange bm_offset -65608 (expected: -65600)
> > > strange bm_offset -65608 (expected: -65600)
> > > md_offset 1099511623680
> > > al_offset 1099511590912
> > > bm_offset 1099478036480
> > > Found ext3 filesystem
> > >   1073709016 kB data area apparently used
> > >   1073709020 kB left usable by current configuration
> > > Even though it looks like this would place the new meta data into
> > > unused space, you still need to confirm, as this is only a guess.
> > > Do you want to proceed?
> > > [need to type 'yes' to confirm] yes
> >
> > initializing activity log
> > > NOT initializing bitmap
> > > Writing meta data...
> > > New drbd meta data block successfully created.
> > > success
> >
> >
> > Same warning twice
>
> So what.  During "probing", some validation is done with various
> potential contexts.  But it keeps being "strange", which is reported.
>
> > possible wrong fs detection (in fact it's ext4) and
>
> ext2, 3, 4, who cares.  for this purpose, keeping you from shooting
> yourself by truncating an existing file system, they are the same.
> They don't differ in their "master magic",
> but only in by feature flags, which are irrelevant for us.
>
> > incorrect data usage detection (in fact takes 533GB of 1Tb partition)
>
> How much "payload" you stored in the file system is irrelevant,
> we care about the size of the file system itself,
> and that you won't truncate it.
>
> It may well still be completely "empty", it still would be very unhappy
> if you truncated the data area it thinks it owns exclusively.
>
> > Wiping and creating metadata again fixes bm_offset warning until next
> drbd
> > service restart.
>
> Yup. The kernel sanity checks are less picky when attaching the first
> time, and just fixes the broken offsets drbdmeta put there.
>
> > Having latest Centos 6.5. Kernel 2.6.32-504.1.3.el6.x86_64
> > Disks are based on hardware RAID 1.
> >
> > $ sudo fdisk -l
>
> Next time use "sfdisk -d" (to get the exact sectors,
> not some rounded last-millenium cylinder numbers),
> or better yet, show "blockdev --getsize64 /dev/X"
>
> > backup uses /dev/sda3 and main uses /dev/sda6
>
> > Can someone hint me what's wrong am I doing? Will provide any additional
> > info on request.
>
> The fix:
> http://git.linbit.com/gitweb.cgi?p=drbd-utils.git;a=commit;h=
> 7969b9dc6636b5543d16b7b09ae267abc64ca111
> | drbdmeta: fix regression corner cases in bitmap size calculation
> |
> | Regression was introduced with the kernel/userland split.
> | Regression was caused underestimating the number of bytes by
> | not aligning the number of bits to 64 before converting to bytes
> | (dividing by 8).
>
> Apparently, your sda3 just happens to be of some "odd" size,
> just above 2147483648 sectors, (hex: 0x80000000).
> (if it was exactly that, you would not have triggered the bug).
>
>
> --
> : Lars Ellenberg
> : http://www.LINBIT.com | Your Way to High Availability
> : DRBD, Linux-HA  and  Pacemaker support and consulting
>
> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
> __
> please don't Cc me, but send to list   --   I'm subscribed
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20150126/8f84f30a/attachment.htm>


More information about the drbd-user mailing list