[DRBD-user] LVM snapshot not getting removed after resync

Jake Smith jsmith at argotec.com
Thu Aug 11 18:41:05 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


I did some further testing by running the commands manually

I edited "/usr/lib/drbd/unsnapshot-resync-target-lvm.sh" and commented out the exec line that redirects the output then ran this:

$ drbdadm before-resync-target minor-1

Which yielded the following output:

invoked for ldap
File descriptor 3 (/etc/drbd.conf) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 4 (/home/jsmith) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 5 (/home/jsmith) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 6 (/home/jsmith) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 7 (/etc) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 9 (/etc) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 10 (/etc) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 11 (/etc) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 12 (/etc) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 13 (/etc) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 14 (/etc) leaked on lvdisplay invocation. Parent PID 21208: /bin/bash
File descriptor 3 (/etc/drbd.conf) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 4 (/home/jsmith) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 5 (/home/jsmith) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 6 (/home/jsmith) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 7 (/etc) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 9 (/etc) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 10 (/etc) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 11 (/etc) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 12 (/etc) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 13 (/etc) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
File descriptor 14 (/etc) leaked on lvcreate invocation. Parent PID 21214: /bin/bash
  Rounding up size to full physical extent 20.00 MiB
  Logical volume "ldap-before-resync" already exists in volume group "Drbd"

Then I ran:

$ drbdadm after-resync-target minor-1

Which yielded nothing...

not even the "invoked for" line.

Maybe this is a bug?

DRBD version:
ii  drbd8-source                    2:8.3.7-1ubuntu2.3                RAID 1 over tcp/ip for Linux module source
ii  drbd8-utils                     2:8.3.7-1ubuntu2.3                RAID 1 over tcp/ip for Linux utilities





----- Original Message ----- 

> From: "Jake Smith" <jsmith at argotec.com>
> To: drbd-user at lists.linbit.com
> Sent: Tuesday, August 9, 2011 4:56:14 PM
> Subject: [DRBD-user] LVM snapshot not getting removed after resync

> I have my DRBD resources on top of LVM2 in a Pacemaker cluster.
> I have it setup to take a snapshot prior to resync and then remove
> after using the included scripts.
> The snapshots are being taken but not removed after resync.

> global_common.conf:

> before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p
> 15 -- -c 16k";
> after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;

> If I put one node in standby and then bring it back online the proper
> scripts are invoked. The "before" output is shown and the snapshots
> are created. After the resync completes and the "after" script is
> invoked and returns no error but the output isn't shown
> /var/log/messages and the snapshot(s) are still there.

> DRBD status is connected - Secondary/Primary.

> messages:

> Aug 8 16:08:09 Condor kernel: [2261509.078128] block drbd4: Starting
> worker thread (from cqueue [1597])
> Aug 8 16:08:09 Condor kernel: [2261509.078199] block drbd4: disk(
> Diskless -> Attaching )
> Aug 8 16:08:09 Condor kernel: [2261509.078667] block drbd4: Found 4
> transactions (4 active extents) in activity log.
> Aug 8 16:08:09 Condor kernel: [2261509.078671] block drbd4: Method to
> ensure write ordering: drain
> Aug 8 16:08:09 Condor kernel: [2261509.078676] block drbd4: Backing
> device's merge_bvec_fn() = ffffffff81439bc0
> Aug 8 16:08:09 Condor kernel: [2261509.078679] block drbd4:
> max_segment_size ( = BIO size ) = 4096
> Aug 8 16:08:09 Condor kernel: [2261509.078683] block drbd4:
> drbd_bm_resize called with capacity == 65456
> Aug 8 16:08:09 Condor kernel: [2261509.078689] block drbd4: resync
> bitmap: bits=8182 words=128
> Aug 8 16:08:09 Condor kernel: [2261509.078692] block drbd4: size = 32
> MB (32728 KB)
> Aug 8 16:08:09 Condor kernel: [2261509.078743] block drbd4:
> recounting of set bits took additional 0 jiffies
> Aug 8 16:08:09 Condor kernel: [2261509.078746] block drbd4: 0 KB (0
> bits) marked out-of-sync by on disk bit-map.
> Aug 8 16:08:09 Condor kernel: [2261509.078750] block drbd4: disk(
> Attaching -> Outdated )
> Aug 8 16:08:09 Condor kernel: [2261509.093225] block drbd4: conn(
> StandAlone -> Unconnected )
> Aug 8 16:08:09 Condor kernel: [2261509.093246] block drbd4: Starting
> receiver thread (from drbd4_worker [23998])
> Aug 8 16:08:09 Condor kernel: [2261509.093303] block drbd4: receiver
> (re)started
> Aug 8 16:08:09 Condor kernel: [2261509.093308] block drbd4: conn(
> Unconnected -> WFConnection )
> Aug 8 16:08:09 Condor kernel: [2261509.188172] block drbd4: Handshake
> successful: Agreed network protocol version 91
> Aug 8 16:08:09 Condor kernel: [2261509.188179] block drbd4: conn(
> WFConnection -> WFReportParams )
> Aug 8 16:08:09 Condor kernel: [2261509.188318] block drbd4: Starting
> asender thread (from drbd4_receiver [24045])
> Aug 8 16:08:09 Condor kernel: [2261509.188392] block drbd4:
> data-integrity-alg: <not-used>
> Aug 8 16:08:09 Condor kernel: [2261509.188921] block drbd4:
> drbd_sync_handshake:
> Aug 8 16:08:09 Condor kernel: [2261509.188926] block drbd4: self
> 142A4F0F4025D354:0000000000000000:6D30156DB75DBAB4:7F22ED3EC52A5F1C
> bits:0 flags:0
> Aug 8 16:08:09 Condor kernel: [2261509.188930] block drbd4: peer
> 94813C0B3B6F6EB1:142A4F0F4025D355:6D30156DB75DBAB5:7F22ED3EC52A5F1C
> bits:0 flags:0
> Aug 8 16:08:09 Condor kernel: [2261509.188934] block drbd4:
> uuid_compare()=-1 by rule 50
> Aug 8 16:08:09 Condor kernel: [2261509.188943] block drbd4: peer(
> Unknown -> Primary ) conn( WFReportParams -> WFBitMapT ) pdsk(
> DUnknown -> UpToDate )
> Aug 8 16:08:09 Condor kernel: [2261509.207454] block drbd4: conn(
> WFBitMapT -> WFSyncUUID )
> Aug 8 16:08:09 Condor kernel: [2261509.208080] block drbd4: helper
> command: /sbin/drbdadm before-resync-target minor-4
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: invoked
> for bind
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 3 (/) leaked on lvdisplay invocation. Parent PID 24161:
> /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 4 (/etc) leaked on lvdisplay invocation. Parent PID
> 24161: /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 6 (/etc) leaked on lvdisplay invocation. Parent PID
> 24161: /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 7 (/etc) leaked on lvdisplay invocation. Parent PID
> 24161: /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 8 (/etc) leaked on lvdisplay invocation. Parent PID
> 24161: /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 9 (/etc) leaked on lvdisplay invocation. Parent PID
> 24161: /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 10 (/etc) leaked on lvdisplay invocation. Parent PID
> 24161: /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 3 (/) leaked on lvcreate invocation. Parent PID 24231:
> /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 4 (/etc) leaked on lvcreate invocation. Parent PID 24231:
> /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 6 (/etc) leaked on lvcreate invocation. Parent PID 24231:
> /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 7 (/etc) leaked on lvcreate invocation. Parent PID 24231:
> /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 8 (/etc) leaked on lvcreate invocation. Parent PID 24231:
> /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 9 (/etc) leaked on lvcreate invocation. Parent PID 24231:
> /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: File
> descriptor 10 (/etc) leaked on lvcreate invocation. Parent PID
> 24231: /bin/bash
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: Rounding
> up size to full physical extent 16.00 MiB
> Aug 8 16:08:09 Condor snapshot-resync-target-lvm.sh[24161]: Logical
> volume "bind-before-resync" created
> Aug 8 16:08:09 Condor kernel: [2261509.340447] block drbd4: helper
> command: /sbin/drbdadm before-resync-target minor-4 exit code 0
> (0x0)
> Aug 8 16:08:09 Condor kernel: [2261509.340452] block drbd4: conn(
> WFSyncUUID -> SyncTarget ) disk( Outdated -> Inconsistent )
> Aug 8 16:08:09 Condor kernel: [2261509.340459] block drbd4: Began
> resync as SyncTarget (will sync 0 KB [0 bits set]).
> Aug 8 16:08:10 Condor crm-unfence-peer.sh[24552]: invoked for bind
> Aug 8 16:08:10 Condor kernel: [2261509.878140] block drbd4: Resync
> done (total 1 sec; paused 0 sec; 0 K/sec)
> Aug 8 16:08:10 Condor kernel: [2261509.878149] block drbd4: conn(
> SyncTarget -> Connected ) disk( Inconsistent -> UpToDate )
> Aug 8 16:08:10 Condor kernel: [2261509.878158] block drbd4: helper
> command: /sbin/drbdadm after-resync-target minor-4
> Aug 8 16:08:10 Condor kernel: [2261509.942702] block drbd4: helper
> command: /sbin/drbdadm after-resync-target minor-4 exit code 0 (0x0)

> I would expect something like this to be logged after the
> after-resync-target is issued:

> Aug 8 xx:xx:xx Condor unsnapshot-resync-target-lvm.sh[xxxxxxx]:
> Logical volume "bind-before-resync" removed

> Thanks!

> Jake Smith
> Network Admin
> Argotec, Inc.
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user



More information about the drbd-user mailing list