Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Lars Ellenberg schrieb:
> On Thu, Jan 10, 2008 at 01:13:42PM +0100, Tomasz Chmielewski wrote:
>> Tomasz Chmielewski schrieb:
>>> It seems to me that for some reason it's not possible to create a
>>> physical volume on a DRBD device (at least on my machine, that is).
>>>
>>> Below, we can see that running "pvcreate" succeeds and than fails, then
>>> it succeeds again etc.:
>>>
>>> # pvcreate -d -v /dev/drbd0
>>> Set up physical volume for "/dev/drbd0" with 976544952 available
>>> sectors
>>> Zeroing start of device /dev/drbd0
>>> Physical volume "/dev/drbd0" successfully created
>
> there, "Physical volume "/dev/drbd0" successfully created".
> so, what gives?
It just says so. Later on it is needed to create a volume group - and it
always fails, no matter if the previous pvcreate was successful or not:
# vgcreate san2_data /dev/drbd0
No physical volume label read from /dev/drbd0
/dev/drbd0 not identified as an existing physical volume
Unable to add physical volume '/dev/drbd0' to volume group 'san2_data'.
>>> # pvcreate -d -v /dev/drbd0
>>> Wiping cache of LVM-capable devices
>>> Set up physical volume for "/dev/drbd0" with 976544952 available
>>> sectors
>>> Zeroing start of device /dev/drbd0
>>> Failed to write physical volume "/dev/drbd0"
>
> why would you want to recreate a pv
> on a pv you just now successfully created?
Why not? It shouldn't fail if I try to recreate it.
>> Looks like LVM doesn't like unpartitioned DRBD devices?
>>
>> I added a partition to /dev/drbd0 with fdisk.
>>
>> Then, I added a drive mapping with kpartx.
>>
>> It works fine now.
>>
>> # kpartx -a -v /dev/drbd0
>> add map drbd0p1 : 0 976543092 linear /dev/drbd0 63
>>
>> # pvcreate -d -v /dev/mapper/drbd0p1
>> Set up physical volume for "/dev/mapper/drbd0p1" with 976542708
>> available sectors
>> Zeroing start of device /dev/mapper/drbd0p1
>> Physical volume "/dev/mapper/drbd0p1" successfully created
>>
>>
>> Anyway, why does it fail if I want to set up LVM on raw DRBD device? A
>> bug or a feature?
>
> works for me.
> we do have several "lvm on top of drbd" (or, drbd as pv for lvm)
> in production. and we for sure don't "partition" them first,
> that would be nonsense.
> so neither drbd nor lvm is the problem here.
> check your lvm configuration.
For me, it sits first on an encrypted device, which is then partitioned,
maybe this confuses LVM?
If my drawing will not mess up when I send it, this is how it looks like
(data partition is to be replicated with DRBD to another server; swap
and non-crucial data is not to be replicated):
/dev/mapper/san2_crypt1 -
/ - drbd0 - LVM2 (san2_data)
/
Encrypted /dev/sdb - /dev/mapper/san2_crypt
\
/dev/mapper/san2_crypt2 -
- LVM2 (san2_swap)
I can't create LVM volume on unpartitioned drbd0.
> try adding -vvvv (and, if you want to, -dddd) to your pvcreate
> commandline, the output, should give you a hint what actually goes on,
> and that should be enough to find out what you are doing wrong.
Ouch, it creates a 120 kB log.
Still, no clue why it fails (it mentions duplicate PV, but it doesn't
tell me anything) - here are some parts of it, link to the full log below:
#lvmcmdline.c:871 Processing: pvcreate -dddd -vvvv /dev/drbd0
#lvmcmdline.c:874 O_DIRECT will be used
#config/config.c:846 Setting global/locking_type to 1
#config/config.c:823 Setting global/locking_dir to /var/lock/lvm
#locking/locking.c:139 File-based locking enabled.
#locking/file_locking.c:164 Locking /var/lock/lvm/P_orphans WB
#device/dev-io.c:439 Opened /dev/drbd0 RW O_DIRECT
#device/dev-io.c:134 /dev/drbd0: block size is 4096 bytes
#label/label.c:158 /dev/drbd0: lvm2 label detected
#cache/lvmcache.c:655 lvmcache: /dev/drbd0: now orphaned
#format_text/format-text.c:915 <backtrace>
#label/label.c:158 /dev/drbd0: lvm2 label detected
#format_text/format-text.c:915 <backtrace>
#filters/filter-persistent.c:55 Wiping cache of LVM-capable devices
(...)
#device/dev-cache.c:208 /dev/drbd0: Already in device cache
(...)
#device/dev-io.c:264 /dev/drbd0: size is 976545336 sectors
#device/dev-io.c:264 /dev/drbd0: size is 976545336 sectors
#filters/filter-composite.c:31 Using /dev/drbd0
#label/label.c:158 /dev/drbd0: lvm2 label detected
(...)
#device/dev-io.c:439 Opened /dev/dm-95 RO
#device/dev-io.c:264 /dev/dm-95: size is 977272832 sectors
#device/dev-io.c:485 Closed /dev/dm-95
#device/dev-io.c:264 /dev/dm-95: size is 977272832 sectors
#device/dev-io.c:439 Opened /dev/dm-95 RW O_DIRECT
#device/dev-io.c:134 /dev/dm-95: block size is 4096 bytes
#device/dev-io.c:485 Closed /dev/dm-95
#filters/filter-composite.c:31 Using /dev/dm-95
#device/dev-io.c:439 Opened /dev/dm-95 RW O_DIRECT
#device/dev-io.c:134 /dev/dm-95: block size is 4096 bytes
#label/label.c:158 /dev/dm-95: lvm2 label detected
#cache/lvmcache.c:791 Duplicate PV
Ndrdet3xirFKZfBIqtqjX1BAiXxW6dtd on /dev/drbd0 - using dm /dev/dm-95
#format_text/format-text.c:915 <backtrace>
(...)
#device/dev-io.c:485 Closed /dev/drbd0
#device/dev-io.c:439 Opened /dev/drbd0 RW O_EXCL O_DIRECT
#device/dev-io.c:485 Closed /dev/drbd0
#device/dev-io.c:264 /dev/drbd0: size is 976545336 sectors
#device/dev-io.c:439 Opened /dev/drbd0 RW O_DIRECT
#device/dev-io.c:134 /dev/drbd0: block size is 4096 bytes
#config/config.c:851 metadata/pvmetadatasize not found in config:
defaulting to 255
#config/config.c:851 metadata/pvmetadatacopies not found in
config: defaulting to 1
#device/dev-io.c:264 /dev/drbd0: size is 976545336 sectors
#device/dev-io.c:619 Wiping /dev/drbd0 at 4096 length 1
#pvcreate.c:213 Set up physical volume for "/dev/drbd0" with
976544952 available sectors
#label/label.c:199 Scanning for labels to wipe from /dev/drbd0
#label/label.c:240 /dev/drbd0: Wiping label at sector 1
#pvcreate.c:222 Zeroing start of device /dev/drbd0
#device/dev-io.c:623 Wiping /dev/drbd0 at sector 0 length 4 sectors
#pvcreate.c:237 Writing physical volume data to disk "/dev/drbd0"
#cache/lvmcache.c:778 Ignoring duplicate PV
vNkxW9mSGlLkzZzsgukfNlu7EvwsaZi4.^[ on /dev/drbd0 - using dm /dev/dm-95
#format_text/format-text.c:1152 <backtrace>
#metadata/metadata.c:1369 <backtrace>
#pvcreate.c:240 Failed to write physical volume "/dev/drbd0"
#locking/file_locking.c:59 Unlocking /var/lock/lvm/P_orphans
#device/dev-io.c:485 Closed /dev/dm-95
#device/dev-io.c:485 Closed /dev/drbd0
Full log available on http://wpkg.org/pvcreate-on-drbd-fails.txt
--
Tomasz Chmielewski