Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Thu, Jan 10, 2008 at 01:59:22PM +0100, Tomasz Chmielewski wrote: > Lars Ellenberg schrieb: > >On Thu, Jan 10, 2008 at 01:13:42PM +0100, Tomasz Chmielewski wrote: > >>Tomasz Chmielewski schrieb: > >>>It seems to me that for some reason it's not possible to create a > >>>physical volume on a DRBD device (at least on my machine, that is). > >>> > >>>Below, we can see that running "pvcreate" succeeds and than fails, then > >>>it succeeds again etc.: > >>> > >>># pvcreate -d -v /dev/drbd0 > >>> Set up physical volume for "/dev/drbd0" with 976544952 available > >>>sectors > >>> Zeroing start of device /dev/drbd0 > >>> Physical volume "/dev/drbd0" successfully created > > > >there, "Physical volume "/dev/drbd0" successfully created". > >so, what gives? > > It just says so. Later on it is needed to create a volume group - and it > always fails, no matter if the previous pvcreate was successful or not: > > # vgcreate san2_data /dev/drbd0 > No physical volume label read from /dev/drbd0 > /dev/drbd0 not identified as an existing physical volume > Unable to add physical volume '/dev/drbd0' to volume group 'san2_data'. > > > >>># pvcreate -d -v /dev/drbd0 > >>> Wiping cache of LVM-capable devices > >>> Set up physical volume for "/dev/drbd0" with 976544952 available > >>>sectors > >>> Zeroing start of device /dev/drbd0 > >>> Failed to write physical volume "/dev/drbd0" > > > >why would you want to recreate a pv > >on a pv you just now successfully created? > > Why not? It shouldn't fail if I try to recreate it. I think it does not overwrite a previous label, unless you use -f or -ff. but that is not the problem here. > >>Looks like LVM doesn't like unpartitioned DRBD devices? > >> > >>I added a partition to /dev/drbd0 with fdisk. > >> > >>Then, I added a drive mapping with kpartx. > >> > >>It works fine now. > >> > >># kpartx -a -v /dev/drbd0 > >>add map drbd0p1 : 0 976543092 linear /dev/drbd0 63 > >> > >># pvcreate -d -v /dev/mapper/drbd0p1 > >> Set up physical volume for "/dev/mapper/drbd0p1" with 976542708 > >>available sectors > >> Zeroing start of device /dev/mapper/drbd0p1 > >> Physical volume "/dev/mapper/drbd0p1" successfully created > >> > >> > >>Anyway, why does it fail if I want to set up LVM on raw DRBD device? A > >>bug or a feature? > > > >works for me. > >we do have several "lvm on top of drbd" (or, drbd as pv for lvm) > >in production. and we for sure don't "partition" them first, > >that would be nonsense. > >so neither drbd nor lvm is the problem here. > >check your lvm configuration. > > For me, it sits first on an encrypted device, which is then partitioned, > maybe this confuses LVM? > > If my drawing will not mess up when I send it, this is how it looks like > (data partition is to be replicated with DRBD to another server; swap > and non-crucial data is not to be replicated): > > /dev/mapper/san2_crypt1 - > > / - drbd0 - LVM2 (san2_data) > / > Encrypted /dev/sdb - /dev/mapper/san2_crypt > \ > /dev/mapper/san2_crypt2 - > - LVM2 (san2_swap) > > I can't create LVM volume on unpartitioned drbd0. > > > >try adding -vvvv (and, if you want to, -dddd) to your pvcreate > >commandline, the output, should give you a hint what actually goes on, > >and that should be enough to find out what you are doing wrong. > > Ouch, it creates a 120 kB log. :) > Still, no clue why it fails (it mentions duplicate PV, but it doesn't > tell me anything) - here are some parts of it, link to the full log below: > > #lvmcmdline.c:871 Processing: pvcreate -dddd -vvvv /dev/drbd0 > #lvmcmdline.c:874 O_DIRECT will be used > #config/config.c:846 Setting global/locking_type to 1 > #config/config.c:823 Setting global/locking_dir to /var/lock/lvm > #locking/locking.c:139 File-based locking enabled. > #locking/file_locking.c:164 Locking /var/lock/lvm/P_orphans WB > #device/dev-io.c:439 Opened /dev/drbd0 RW O_DIRECT > #device/dev-io.c:134 /dev/drbd0: block size is 4096 bytes > #label/label.c:158 /dev/drbd0: lvm2 label detected > #cache/lvmcache.c:655 lvmcache: /dev/drbd0: now orphaned > #format_text/format-text.c:915 <backtrace> > #label/label.c:158 /dev/drbd0: lvm2 label detected > #format_text/format-text.c:915 <backtrace> > #filters/filter-persistent.c:55 Wiping cache of LVM-capable devices > > (...) > > #device/dev-cache.c:208 /dev/drbd0: Already in device cache > > (...) > > #device/dev-io.c:264 /dev/drbd0: size is 976545336 sectors > #device/dev-io.c:264 /dev/drbd0: size is 976545336 sectors > #filters/filter-composite.c:31 Using /dev/drbd0 > #label/label.c:158 /dev/drbd0: lvm2 label detected > > (...) > > #device/dev-io.c:439 Opened /dev/dm-95 RO > #device/dev-io.c:264 /dev/dm-95: size is 977272832 sectors > #device/dev-io.c:485 Closed /dev/dm-95 > #device/dev-io.c:264 /dev/dm-95: size is 977272832 sectors > #device/dev-io.c:439 Opened /dev/dm-95 RW O_DIRECT > #device/dev-io.c:134 /dev/dm-95: block size is 4096 bytes > #device/dev-io.c:485 Closed /dev/dm-95 > #filters/filter-composite.c:31 Using /dev/dm-95 > #device/dev-io.c:439 Opened /dev/dm-95 RW O_DIRECT > #device/dev-io.c:134 /dev/dm-95: block size is 4096 bytes > #label/label.c:158 /dev/dm-95: lvm2 label detected > #cache/lvmcache.c:791 Duplicate PV > Ndrdet3xirFKZfBIqtqjX1BAiXxW6dtd on /dev/drbd0 - using dm /dev/dm-95 that is it, right there. pv label already detected on /dev/dm-95, (which is likely your de-crypted dm-crypt target?). you'd have to filter out that device in the lvm.conf. preferably, you'd have a filter that only accepts drbd, and rejects anything else. did you know about LVM_SYSTEM_DIR ? try this: copy your /etc/lvm to /etc/lvm.on-drbd, edit the lvm.conf there to place the cache there, too, or preferably: remove the cache, and disable it), edit the filter to something like a|^/dev/drbd|, r|.|, then ( export LVM_SYSTEM_DIR=/etc/lvm.on-drbd pvcreate... vgcreate... ) if that works out, figure out how you can make that dm-XY device apear always as the same XY after reboot, then filter it out specifically. your "partitioning" of the drbd changed the offset of the to-become pv, to something where lvm would normally not look. but if you want to go that route, rather partition the dm-XY below drbd, and put drbd on top of one of those. your problem apears to be that you cannot decide whether your lvm lives above or below drbd, or on both sides :) using lvs as pvs for nested vgs is tricky. -- : Lars Ellenberg http://www.linbit.com : : DRBD/HA support and consulting sales at linbit.com : : LINBIT Information Technologies GmbH Tel +43-1-8178292-0 : : Vivenotgasse 48, A-1120 Vienna/Europe Fax +43-1-8178292-82 : __ please use the "List-Reply" function of your email client.