Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello.
no-md-flushes's default value or behavior was changed
on drbd-8.4 ?
I use DRBD on LVM and barriers are not supported.
drbd-8.3.10 with "no-disk-barrier;" works
but drbd-8.4.0 with "no-disk-barrier;" (or "disk-barrier: no;")
does not work.
Aug 11 18:39:41 XXX kernel: block drbd0: Method to ensure write
ordering: flush
***
Aug 11 18:39:41 XXX kernel: block drbd0: attached to UUIDs
9FA234B78651E7A6:0000000000000000:01D5349BD9ABE16B:01D4349BD9ABE16B
Aug 11 18:39:41 XXX kernel: block drbd0: Barriers not supported on
meta data device - disabling
Aug 11 18:39:41 XXX kernel: block drbd0: meta data update failed!
Aug 11 18:39:41 XXX kernel: block drbd0: disk( Consistent -> Failed )
Aug 11 18:39:41 XXX kernel: block drbd0: Local IO failed in
drbd_md_sync. Detaching...
Aug 11 18:39:41 XXX kernel: block drbd0: disk( Failed -> Diskless )
drbd-8.4.0 with "no-disk-barrier; no-md-flushes;"
(or "disk-barrier no; md-flushes no;") works.
Aug 11 18:53:05 XXX kernel: block drbd0: Method to ensure write
ordering: flush
***
Aug 11 18:53:05 XXX kernel: block drbd0: attached to UUIDs
9FA234B78651E7A6:0000000000000000:01D5349BD9ABE16B:01D4349BD9ABE16B
Aug 11 18:53:05 XXX kernel: d-con r0: conn( StandAlone -> Unconnected )
Aug 11 18:53:05 XXX kernel: d-con r0: Starting receiver thread
(from drbd_w_r0 [2881])
Is this a change of "no-md-flushes"'s behavior?
("no-md-flushes" is not mentioned in ChangeLog of 8.4.0.)
Is meta data safe on this settings ("no-disk-barrier; no-md-flushes;") ?
Thanks.
Logs are below...
------------------------------------------------------------------
drbd-8.3.10
no-disk-barrier;
WORKS
common {
protocol C;
syncer {
verify-alg md5;
rate 40M;
}
disk {
on-io-error detach;
fencing resource-only;
no-disk-barrier;
# no-md-flushes;
}
handlers {
*****
}
}
resource r0 {
device /dev/drbd0;
disk /dev/VolGroup00/lv_drbd0;
meta-disk internal;
on XXXXX1 {
address *****;
}
on XXXXX2 {
address *****;
}
}
Aug 11 18:32:11 XXX kernel: block drbd0: Starting worker thread (from
cqueue/0 [116])
Aug 11 18:32:11 XXX kernel: block drbd0: disk( Diskless -> Attaching )
Aug 11 18:32:11 XXX kernel: block drbd0: Found 4 transactions (11
active extents) in activity log.
Aug 11 18:32:11 XXX kernel: block drbd0: Method to ensure write ordering: flush
Aug 11 18:32:11 XXX kernel: block drbd0: max BIO size = 65536
Aug 11 18:32:11 XXX kernel: block drbd0: drbd_bm_resize called with
capacity == 44038776
Aug 11 18:32:11 XXX kernel: block drbd0: resync bitmap: bits=5504847
words=86014 pages=168
Aug 11 18:32:11 XXX kernel: block drbd0: size = 21 GB (22019388 KB)
Aug 11 18:32:11 XXX kernel: block drbd0: bitmap READ of 168 pages took
44 jiffies
Aug 11 18:32:11 XXX kernel: block drbd0: recounting of set bits took
additional 1 jiffies
Aug 11 18:32:11 XXX kernel: block drbd0: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug 11 18:32:11 XXX kernel: block drbd0: disk( Attaching -> Consistent )
Aug 11 18:32:11 XXX kernel: block drbd0: attached to UUIDs
3389ACBB8429C508:0000000000000000:9DAB8165891B5F66:9DAA8165891B5F66
Aug 11 18:32:11 XXX kernel: block drbd0: Barriers not supported on
meta data device - disabling
Aug 11 18:32:11 XXX kernel: block drbd0: conn( StandAlone -> Unconnected )
Aug 11 18:32:11 XXX kernel: block drbd0: Starting receiver thread
(from drbd0_worker [16439])
Aug 11 18:32:11 XXX kernel: block drbd0: receiver (re)started
Aug 11 18:32:11 XXX kernel: block drbd0: conn( Unconnected -> WFConnection )
Aug 11 18:32:11 XXX kernel: block drbd0: Handshake successful: Agreed
network protocol version 96
Aug 11 18:32:11 XXX kernel: block drbd0: conn( WFConnection -> WFReportParams )
Aug 11 18:32:11 XXX kernel: block drbd0: Starting asender thread (from
drbd0_receiver [16444])
Aug 11 18:32:11 XXX kernel: block drbd0: data-integrity-alg: <not-used>
Aug 11 18:32:11 XXX kernel: block drbd0: max BIO size = 65536
Aug 11 18:32:11 XXX kernel: block drbd0: drbd_sync_handshake:
Aug 11 18:32:11 XXX kernel: block drbd0: self
3389ACBB8429C508:0000000000000000:9DAB8165891B5F66:9DAA8165891B5F66
bits:0 flags:0
Aug 11 18:32:11 XXX kernel: block drbd0: peer
801951088EEAC11F:3389ACBB8429C508:9DAB8165891B5F66:9DAA8165891B5F66
bits:55 flags:0
Aug 11 18:32:11 XXX kernel: block drbd0: uuid_compare()=-1 by rule 50
Aug 11 18:32:11 XXX kernel: block drbd0: peer( Unknown -> Primary )
conn( WFReportParams -> WFBitMapT ) disk( Consistent -> Outdated )
pdsk( DUnknown -> UpToDate )
Aug 11 18:32:11 XXX kernel: block drbd0: receive bitmap stats
[Bytes(packets)]: plain 0(0), RLE 46(1), total 46; compression: 100.0%
Aug 11 18:32:11 XXX kernel: block drbd0: conn( WFBitMapT -> WFSyncUUID )
Aug 11 18:32:11 XXX kernel: block drbd0: updated sync uuid
338AACBB8429C508:0000000000000000:9DAB8165891B5F66:9DAA8165891B5F66
Aug 11 18:32:11 XXX kernel: block drbd0: helper command: /sbin/drbdadm
before-resync-target minor-0
Aug 11 18:32:11 XXX snapshot-resync-target-lvm.sh[16459]: invoked for r0
Aug 11 18:32:11 XXX lvm[4658]: Monitoring snapshot
VolGroup00-lv_drbd0--before--resync
Aug 11 18:32:11 XXX snapshot-resync-target-lvm.sh[16459]: Rounding
up size to full physical extent 1.06 GB
Aug 11 18:32:11 XXX snapshot-resync-target-lvm.sh[16459]: Logical
volume "lv_drbd0-before-resync" created
Aug 11 18:32:11 XXX kernel: block drbd0: helper command: /sbin/drbdadm
before-resync-target minor-0 exit code 0 (0x0)
Aug 11 18:32:11 XXX kernel: block drbd0: conn( WFSyncUUID ->
SyncTarget ) disk( Outdated -> Inconsistent )
Aug 11 18:32:11 XXX kernel: block drbd0: Began resync as SyncTarget
(will sync 220 KB [55 bits set]).
Aug 11 18:32:11 XXX kernel: block drbd0: Resync done (total 1 sec;
paused 0 sec; 220 K/sec)
------------------------------------------------------------------
drbd-8.3.10
no-disk-barrier;
no-md-flushes;
WORKS
Aug 11 18:34:28 XXX kernel: block drbd0: Starting worker thread (from
cqueue/0 [116])
Aug 11 18:34:28 XXX kernel: block drbd0: disk( Diskless -> Attaching )
Aug 11 18:34:28 XXX kernel: block drbd0: Found 4 transactions (11
active extents) in activity log.
Aug 11 18:34:28 XXX kernel: block drbd0: Method to ensure write ordering: flush
Aug 11 18:34:28 XXX kernel: block drbd0: max BIO size = 65536
Aug 11 18:34:28 XXX kernel: block drbd0: drbd_bm_resize called with
capacity == 44038776
Aug 11 18:34:28 XXX kernel: block drbd0: resync bitmap: bits=5504847
words=86014 pages=168
Aug 11 18:34:28 XXX kernel: block drbd0: size = 21 GB (22019388 KB)
Aug 11 18:34:28 XXX kernel: block drbd0: bitmap READ of 168 pages took
47 jiffies
Aug 11 18:34:28 XXX kernel: block drbd0: recounting of set bits took
additional 1 jiffies
Aug 11 18:34:28 XXX kernel: block drbd0: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug 11 18:34:28 XXX kernel: block drbd0: disk( Attaching -> Outdated )
Aug 11 18:34:28 XXX kernel: block drbd0: attached to UUIDs
801951088EEAC11E:0000000000000000:338AACBB8429C508:3389ACBB8429C508
Aug 11 18:34:28 XXX kernel: block drbd0: conn( StandAlone -> Unconnected )
Aug 11 18:34:28 XXX kernel: block drbd0: Starting receiver thread
(from drbd0_worker [16665])
Aug 11 18:34:28 XXX kernel: block drbd0: receiver (re)started
Aug 11 18:34:28 XXX kernel: block drbd0: conn( Unconnected -> WFConnection )
Aug 11 18:34:28 XXX kernel: block drbd0: Handshake successful: Agreed
network protocol version 96
Aug 11 18:34:28 XXX kernel: block drbd0: conn( WFConnection -> WFReportParams )
Aug 11 18:34:28 XXX kernel: block drbd0: Starting asender thread (from
drbd0_receiver [16673])
Aug 11 18:34:28 XXX kernel: block drbd0: data-integrity-alg: <not-used>
Aug 11 18:34:28 XXX kernel: block drbd0: max BIO size = 65536
Aug 11 18:34:28 XXX kernel: block drbd0: drbd_sync_handshake:
Aug 11 18:34:28 XXX kernel: block drbd0: self
801951088EEAC11E:0000000000000000:338AACBB8429C508:3389ACBB8429C508
bits:0 flags:0
Aug 11 18:34:28 XXX kernel: block drbd0: peer
01D4349BD9ABE16B:801951088EEAC11F:338AACBB8429C508:3389ACBB8429C508
bits:0 flags:0
Aug 11 18:34:28 XXX kernel: block drbd0: uuid_compare()=-1 by rule 50
Aug 11 18:34:28 XXX kernel: block drbd0: peer( Unknown -> Primary )
conn( WFReportParams -> WFBitMapT ) pdsk( DUnknown -> UpToDate )
Aug 11 18:34:28 XXX kernel: block drbd0: receive bitmap stats
[Bytes(packets)]: plain 0(0), RLE 15(1), total 15; compression: 100.0%
Aug 11 18:34:28 XXX kernel: block drbd0: conn( WFBitMapT -> WFSyncUUID )
Aug 11 18:34:28 XXX kernel: block drbd0: updated sync uuid
801A51088EEAC11E:0000000000000000:338AACBB8429C508:3389ACBB8429C508
Aug 11 18:34:28 XXX kernel: block drbd0: helper command: /sbin/drbdadm
before-resync-target minor-0
Aug 11 18:34:28 XXX snapshot-resync-target-lvm.sh[16688]: invoked for r0
Aug 11 18:34:29 XXX lvm[4658]: Monitoring snapshot
VolGroup00-lv_drbd0--before--resync
Aug 11 18:34:29 XXX snapshot-resync-target-lvm.sh[16688]: Rounding
up size to full physical extent 1.06 GB
Aug 11 18:34:29 XXX snapshot-resync-target-lvm.sh[16688]: Logical
volume "lv_drbd0-before-resync" created
Aug 11 18:34:29 XXX kernel: block drbd0: helper command: /sbin/drbdadm
before-resync-target minor-0 exit code 0 (0x0)
Aug 11 18:34:29 XXX kernel: block drbd0: conn( WFSyncUUID ->
SyncTarget ) disk( Outdated -> Inconsistent )
Aug 11 18:34:29 XXX kernel: block drbd0: Began resync as SyncTarget
(will sync 0 KB [0 bits set]).
Aug 11 18:34:29 XXX kernel: block drbd0: Resync done (total 1 sec;
paused 0 sec; 0 K/sec)
------------------------------------------------------------------
drbd-8.4.0
no-disk-barrier; (or disk-barrier no;)
NOT WORKS
Aug 11 18:39:41 XXX kernel: d-con r0: Starting worker thread (from
drbdsetup [31590])
Aug 11 18:39:41 XXX kernel: block drbd0: disk( Diskless -> Attaching )
Aug 11 18:39:41 XXX kernel: block drbd0: Method to ensure write ordering: flush
Aug 11 18:39:41 XXX kernel: block drbd0: max BIO size = 65536
Aug 11 18:39:41 XXX kernel: block drbd0: drbd_bm_resize called with
capacity == 44038776
Aug 11 18:39:41 XXX kernel: block drbd0: resync bitmap: bits=5504847
words=86014 pages=168
Aug 11 18:39:41 XXX kernel: block drbd0: size = 21 GB (22019388 KB)
Aug 11 18:39:41 XXX kernel: block drbd0: bitmap READ of 168 pages took 2 jiffies
Aug 11 18:39:41 XXX kernel: block drbd0: recounting of set bits took
additional 1 jiffies
Aug 11 18:39:41 XXX kernel: block drbd0: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug 11 18:39:41 XXX kernel: block drbd0: disk( Attaching -> Consistent )
Aug 11 18:39:41 XXX kernel: block drbd0: attached to UUIDs
9FA234B78651E7A6:0000000000000000:01D5349BD9ABE16B:01D4349BD9ABE16B
Aug 11 18:39:41 XXX kernel: block drbd0: Barriers not supported on
meta data device - disabling
Aug 11 18:39:41 XXX kernel: block drbd0:
drbd_md_sync_page_io(,44040184s,WRITE) failed!
Aug 11 18:39:41 XXX kernel: block drbd0: meta data update failed!
Aug 11 18:39:41 XXX kernel: block drbd0: disk( Consistent -> Failed )
Aug 11 18:39:41 XXX kernel: block drbd0: Local IO failed in
drbd_md_sync. Detaching...
Aug 11 18:39:41 XXX kernel: block drbd0: disk( Failed -> Diskless )
Aug 11 18:39:41 XXX kernel: d-con r0: conn( StandAlone -> Unconnected )
Aug 11 18:39:41 XXX kernel: d-con r0: Starting receiver thread (from
drbd_w_r0 [31591])
Aug 11 18:39:41 XXX kernel: d-con r0: receiver (re)started
Aug 11 18:39:41 XXX kernel: d-con r0: conn( Unconnected -> WFConnection )
Aug 11 18:39:41 XXX kernel: d-con r0: Handshake successful: Agreed
network protocol version 96
Aug 11 18:39:41 XXX kernel: d-con r0: conn( WFConnection -> WFReportParams )
Aug 11 18:39:41 XXX kernel: d-con r0: Starting asender thread (from
drbd_r_r0 [31594])
Aug 11 18:39:41 XXX kernel: block drbd0: peer( Unknown -> Primary )
conn( WFReportParams -> Connected ) pdsk( DUnknown -> UpToDate )
------------------------------------------------------------------
drbd-8.4.0
no-disk-barrier; (or disk-barrier no;)
no-md-flushes; (or md-flushes no;)
WORKS
Aug 11 18:53:05 XXX kernel: d-con r0: Starting worker thread (from
drbdsetup [2880])
Aug 11 18:53:05 XXX kernel: block drbd0: disk( Diskless -> Attaching )
Aug 11 18:53:05 XXX kernel: block drbd0: Method to ensure write ordering: flush
Aug 11 18:53:05 XXX kernel: block drbd0: max BIO size = 65536
Aug 11 18:53:05 XXX kernel: block drbd0: drbd_bm_resize called with
capacity == 44038776
Aug 11 18:53:05 XXX kernel: block drbd0: resync bitmap: bits=5504847
words=86014 pages=168
Aug 11 18:53:05 XXX kernel: block drbd0: size = 21 GB (22019388 KB)
Aug 11 18:53:05 XXX kernel: block drbd0: bitmap READ of 168 pages took
13 jiffies
Aug 11 18:53:05 XXX kernel: block drbd0: recounting of set bits took
additional 1 jiffies
Aug 11 18:53:05 XXX kernel: block drbd0: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug 11 18:53:05 XXX kernel: block drbd0: disk( Attaching -> Consistent )
Aug 11 18:53:05 XXX kernel: block drbd0: attached to UUIDs
9FA234B78651E7A6:0000000000000000:01D5349BD9ABE16B:01D4349BD9ABE16B
Aug 11 18:53:05 XXX kernel: d-con r0: conn( StandAlone -> Unconnected )
Aug 11 18:53:05 XXX kernel: d-con r0: Starting receiver thread (from
drbd_w_r0 [2881])
Aug 11 18:53:05 XXX kernel: d-con r0: receiver (re)started
Aug 11 18:53:05 XXX kernel: d-con r0: conn( Unconnected -> WFConnection )
Aug 11 18:53:05 XXX kernel: d-con r0: Handshake successful: Agreed
network protocol version 96
Aug 11 18:53:05 XXX kernel: d-con r0: conn( WFConnection -> WFReportParams )
Aug 11 18:53:05 XXX kernel: d-con r0: Starting asender thread (from
drbd_r_r0 [2889])
Aug 11 18:53:05 XXX kernel: block drbd0: drbd_sync_handshake:
Aug 11 18:53:05 XXX kernel: block drbd0: self
9FA234B78651E7A6:0000000000000000:01D5349BD9ABE16B:01D4349BD9ABE16B
bits:0 flags:0
Aug 11 18:53:05 XXX kernel: block drbd0: peer
1837D5C3DBCEF949:9FA234B78651E7A6:01D5349BD9ABE16A:01D4349BD9ABE16B
bits:52 flags:0
Aug 11 18:53:05 XXX kernel: block drbd0: uuid_compare()=-1 by rule 50
Aug 11 18:53:05 XXX kernel: block drbd0: peer( Unknown -> Primary )
conn( WFReportParams -> WFBitMapT ) disk( Consistent -> Outdated )
pdsk( DUnknown -> UpToDate )
Aug 11 18:53:05 XXX kernel: block drbd0: send bitmap stats
[Bytes(packets)]: plain 0(0), RLE 47(1), total 47; compression: 100.0%
Aug 11 18:53:05 XXX kernel: block drbd0: conn( WFBitMapT -> WFSyncUUID )
Aug 11 18:53:05 XXX kernel: block drbd0: updated sync uuid
9FA334B78651E7A6:0000000000000000:01D5349BD9ABE16B:01D4349BD9ABE16B
Aug 11 18:53:05 XXX kernel: block drbd0: helper command: /sbin/drbdadm
before-resync-target minor-0
Aug 11 18:53:05 XXX snapshot-resync-target-lvm.sh[2899]: invoked for
r0/0 (drbd0)
Aug 11 18:53:05 XXX snapshot-resync-target-lvm.sh[2899]:
Insufficient free extents (24) in volume group VolGroup00: 34 required
Aug 11 18:53:05 XXX snapshot-resync-target-lvm.sh[2899]: Rounding up
size to full physical extent 1.06 GB
Aug 11 18:53:05 XXX kernel: block drbd0: helper command: /sbin/drbdadm
before-resync-target minor-0 exit code 0 (0x0)
Aug 11 18:53:05 XXX kernel: block drbd0: conn( WFSyncUUID ->
SyncTarget ) disk( Outdated -> Inconsistent )
Aug 11 18:53:05 XXX kernel: block drbd0: Began resync as SyncTarget
(will sync 208 KB [52 bits set]).
Aug 11 18:53:06 XXX kernel: block drbd0: Resync done (total 1 sec;
paused 0 sec; 208 K/sec)