[DRBD-user] weird behavior with metadata

Alex Vasilenko aa.vasilenko at gmail.com
Wed Jan 28 22:27:23 CET 2015

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


By saying logs I've meant git logs

On Tue Jan 27 2015 at 01:45:24 Alex Vasilenko <aa.vasilenko at gmail.com>
wrote:

> Hi all,
>
> I've been trying to recompile latest drbd-utils and apparently failed:
>
> $ autoconf
> $ ./configure --prefix=/usr --without-83support --with-pacemaker
> ./configure: line 1958: PKG_PROG_PKG_CONFIG: command not found
> configure: Could not detect systemd unit directory
> Using systemd unit directory:
> configure: Could not detect udev rules directory, using default
> Using udev rules directory: /lib/udev
> checking for gcc... gcc
> checking for C compiler default output file name... a.out
> checking whether the C compiler works... yes
> checking whether we are cross compiling... no
> checking for suffix of executables...
> checking for suffix of object files... o
> checking whether we are using the GNU C compiler... yes
> checking whether gcc accepts -g... yes
> checking for gcc option to accept ISO C89... none needed
> checking whether ln -s works... yes
> checking for sed... /bin/sed
> checking for grep... /bin/grep
> checking for flex... /usr/bin/flex
> checking for rpmbuild... /usr/bin/rpmbuild
> checking for xsltproc... /usr/bin/xsltproc
> checking for tar... /bin/tar
> checking for git... /usr/bin/git
> checking for dpkg-buildpackage... no
> checking for udevadm... /sbin/udevadm
> checking for udevinfo... false
> configure: WARNING: No dpkg-buildpackage found, building Debian packages
> is disabled.
> checking for /etc/gentoo-release... no
> checking for /etc/redhat-release... yes
> checking for /etc/slackware-version... no
> checking for /etc/debian_version... no
> checking for /etc/SuSE-release... no
> configure: configured for Red Hat (includes Fedora, RHEL, CentOS).
> checking for /etc/fedora-release... no
> configure: creating ./config.status
> config.status: creating Makefile
> config.status: creating user/shared/Makefile
> config.status: creating user/v9/Makefile
> config.status: creating user/v83/Makefile
> config.status: creating user/v84/Makefile
> config.status: creating scripts/Makefile
> config.status: creating documentation/v9/Makefile
> config.status: creating documentation/v83/Makefile
> config.status: creating documentation/v84/Makefile
> config.status: creating scripts/drbd.rules
> config.status: error: cannot find input file: user/shared/config.h.in
>
> config.log <https://gist.github.com/mente/5707963576be475774aa>
>
> Checked logs - config.h.in was never there. Where can I find it?
>
> Thanks!
> Alex
>
> On Mon Jan 26 2015 at 16:03:53 Alex Vasilenko <aa.vasilenko at gmail.com>
> wrote:
>
>> Hello Lars,
>>
>> Here's mine sectors data:
>>
>> $ sfdisk -d
>>
>> Warning: extended partition does not start at a cylinder boundary.
>> DOS and Linux will interpret the contents differently.
>> # partition table of /dev/sda
>> unit: sectors
>>
>>
>> /dev/sda1 : start= 2048, size= 25165825, Id=83
>> /dev/sda2 : start= 25169920, size= 1048577, Id=83
>> /dev/sda3 : start= 26220544, size=2147483649, Id=83
>> /dev/sda4 : start=2173706240, size=3685736448, Id= f
>> /dev/sda5 : start=2173708288, size=1860173825, Id=83
>> /dev/sda6 : start=4033884160, size=1825558528, Id=83
>>
>> So looks like you're right. Just 1 sector above your number.
>> Will try to compile HEAD drbd. Is it stable enough to be used on
>> production servers?
>>
>> Regards,
>> Alex
>>
>> On Mon Jan 26 2015 at 11:27:18 Lars Ellenberg <lars.ellenberg at linbit.com>
>> wrote:
>>
>>> On Sat, Jan 24, 2015 at 04:01:11PM +0200, Alex Vasilenko wrote:
>>> > Hello,
>>> >
>>> > Having a strange behavior using drbd 8.4.5. After restart one
>>> partition of
>>>
>>> TL;DR: for the fix scroll ahead.
>>>
>>> You should always tell both kernel *and* userland versions.
>>>
>>>
>>> > Jan 24 14:42:39 de kernel: block drbd0: disk( UpToDate -> Failed )
>>>
>>> That may look scary, but is normal: during detach (down),
>>> we move via "Failed" to "Diskless" to avoid various potential races.
>>>
>>> > Now trying to restart backup partition:
>>> >
>>> > > $ sudo drbdadm down backup
>>> > > $ sudo drbdadm up backup
>>> > > strange bm_offset -65608 (expected: -65600)
>>> > > No valid meta data found
>>> > > Command 'drbdmeta 1 v08 /dev/sda3 internal apply-al' terminated with
>>> exit
>>> > > code 255
>>>
>>>
>>> > Somehow metadata is not there anymore (?!). Note warning saying
>>> "strange
>>> > bm_offset -65608 (expected: -65600)".
>>> > Ok, how about recreate metadata and resync full partition? Trying to
>>> wipe
>>> > old metadata shows that metadata is not present. Ok, let's pretend
>>> it's not
>>> > there:
>>> >
>>> > > $ sudo drbdadm create-md backup
>>> > > strange bm_offset -65608 (expected: -65600)
>>> > > strange bm_offset -65608 (expected: -65600)
>>> > > md_offset 1099511623680
>>> > > al_offset 1099511590912
>>> > > bm_offset 1099478036480
>>> > > Found ext3 filesystem
>>> > >   1073709016 kB data area apparently used
>>> > >   1073709020 kB left usable by current configuration
>>> > > Even though it looks like this would place the new meta data into
>>> > > unused space, you still need to confirm, as this is only a guess.
>>> > > Do you want to proceed?
>>> > > [need to type 'yes' to confirm] yes
>>> >
>>> > initializing activity log
>>> > > NOT initializing bitmap
>>> > > Writing meta data...
>>> > > New drbd meta data block successfully created.
>>> > > success
>>> >
>>> >
>>> > Same warning twice
>>>
>>> So what.  During "probing", some validation is done with various
>>> potential contexts.  But it keeps being "strange", which is reported.
>>>
>>> > possible wrong fs detection (in fact it's ext4) and
>>>
>>> ext2, 3, 4, who cares.  for this purpose, keeping you from shooting
>>> yourself by truncating an existing file system, they are the same.
>>> They don't differ in their "master magic",
>>> but only in by feature flags, which are irrelevant for us.
>>>
>>> > incorrect data usage detection (in fact takes 533GB of 1Tb partition)
>>>
>>> How much "payload" you stored in the file system is irrelevant,
>>> we care about the size of the file system itself,
>>> and that you won't truncate it.
>>>
>>> It may well still be completely "empty", it still would be very unhappy
>>> if you truncated the data area it thinks it owns exclusively.
>>>
>>> > Wiping and creating metadata again fixes bm_offset warning until next
>>> drbd
>>> > service restart.
>>>
>>> Yup. The kernel sanity checks are less picky when attaching the first
>>> time, and just fixes the broken offsets drbdmeta put there.
>>>
>>> > Having latest Centos 6.5. Kernel 2.6.32-504.1.3.el6.x86_64
>>> > Disks are based on hardware RAID 1.
>>> >
>>> > $ sudo fdisk -l
>>>
>>> Next time use "sfdisk -d" (to get the exact sectors,
>>> not some rounded last-millenium cylinder numbers),
>>> or better yet, show "blockdev --getsize64 /dev/X"
>>>
>>> > backup uses /dev/sda3 and main uses /dev/sda6
>>>
>>> > Can someone hint me what's wrong am I doing? Will provide any
>>> additional
>>> > info on request.
>>>
>>> The fix:
>>> http://git.linbit.com/gitweb.cgi?p=drbd-utils.git;a=commit;h=
>>> 7969b9dc6636b5543d16b7b09ae267abc64ca111
>>> | drbdmeta: fix regression corner cases in bitmap size calculation
>>> |
>>> | Regression was introduced with the kernel/userland split.
>>> | Regression was caused underestimating the number of bytes by
>>> | not aligning the number of bits to 64 before converting to bytes
>>> | (dividing by 8).
>>>
>>> Apparently, your sda3 just happens to be of some "odd" size,
>>> just above 2147483648 sectors, (hex: 0x80000000).
>>> (if it was exactly that, you would not have triggered the bug).
>>>
>>>
>>> --
>>> : Lars Ellenberg
>>> : http://www.LINBIT.com | Your Way to High Availability
>>> : DRBD, Linux-HA  and  Pacemaker support and consulting
>>>
>>> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
>>> __
>>> please don't Cc me, but send to list   --   I'm subscribed
>>> _______________________________________________
>>> drbd-user mailing list
>>> drbd-user at lists.linbit.com
>>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20150128/ed5c450a/attachment.htm>


More information about the drbd-user mailing list