Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Lars Ellenberg schrieb: > On Mon, Jan 14, 2008 at 05:29:01PM +0100, Tomasz Chmielewski wrote: >> After a couple of days of struggling I got to the point where I have to >> conclude that using LVM reliably over DRBD devices is impossible. > > after a couple of seconds of contemplating I got to the point where I > have to conclude that this would mean that a non-negligible number of > our managed production deployments are impossible, there for non > existent. I wonder why I hear no customer complaints, though. > ;) Did any of them use dm-crypt? :) >> 7. pvs confirms it - LVM sits on /dev/sdb, not on /dev/drbd0 - >> certainly, nothing will be replicated by DRBD: > > as pointed out earlier, you have to fix your lvm conf, specifically the > filter setting. if you don't, it won't work. That I know. How I don't :| >> As device names are not necessarily persistent (if /dev/sdb is a iSCSI >> drive, next time one boots it can be /dev/sdc) and a similar is for >> device mapper devices (think of crypted devices), setting up filters in >> lvm.conf will not be reliable. > > you did hear about udev and scsi id, did you? For iSCSI, true, I could use a link like /dev/iscsi/blah -> /dev/sdX and put that /dev/iscsi/blah as a filter for lvm.conf. dm-crypt devices don't have any ID or anything unique (that I know). One way to overcome seem to do all these steps during the system startup: 1. Put a filter like /etc/drbd-disk-dont-touch-AB to lvm.conf 2. On system startup, find out which one of /dev/dm-XY devices is our crypted device. How to do it? If the device has unique size, just parse the output of fdisk -l. If there are multiple devices with the same size, it gets problematic. 3. Point /etc/drbd-disk-dont-touch-AB symlink to the correct /dev/dm-XY device - this way, it would be exluded for LVM. All that makes the things a bit crude and error-prone, especially when it's possible that there are more devices with the same size, which have to be treated differently. Or, is there an easier way? >> How can I use LVM over DRBD devices *reliably*? > > you make your device nodes persistent. > how to do that is detailed in depth elsewhere. > this problem generic, and not specific to drbd at all. It is, a bit. For example, md (Linux software RAID) can store its metadata (called superblock) at different locations on the device (at start, or at the end, or 4K from the start). This way, by choosing a correct metadata format, one can be sure of data integrity - with metadata at start, mounting or using an underlying device accidentally is impossible. Storing DRBD metadata at the beginning of the device would effectively make some administration tasks easier (no need to write custom scripts for detecting correct dm-XY device) and/or more reliable (not possible to mount or use a device accidentally) - this thread wouldn't have to exist I guess (and some of split-brain posts as well, I guess). Just a thought. -- Tomasz Chmielewski http://wpkg.org