Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi all,
With drbd-8.4.3, I noticed that performing `drbdadm attach r0' with a
newly-initialized empty disk causes initial full synchronization.
In the following document, it seems it is necessary to perform `drbdadm
primary --force resource' to start the initial full synchronization.
http://www.drbd.org/users-guide-emb/s-initial-full-sync.html
However, it starts without performing `drbdadm primary --force
resource'.
I'd like to know whether this is an expected behavior or not.
* enviroment
OS: Scientific Linux release 6.2 (Carbon)
DRBD: drbd-8.4.3
* configuration
# /etc/drbd.conf
common {
}
# resource r0 on nodea: not ignored, not stacked
# defined at /etc/drbd.d/r0.res:1
resource r0 {
on nodea {
volume 0 {
device /dev/drbd0 minor 0;
disk /dev/sdc;
meta-disk internal;
}
address ipv4 192.168.0.1:7789;
}
on nodeb {
volume 0 {
device /dev/drbd0 minor 0;
disk /dev/sdc;
meta-disk internal;
}
address ipv4 192.168.0.2:7789;
}
net {
protocol C;
}
disk {
resync-rate 2g;
c-plan-ahead 0;
}
}
* How To reproduce
(1) cleanup metadata in /dev/sdc
[root at nodea ~]# MegaCli ldinit start l2 a0
Start Initialization on Virtual Drive 2 (target id: 2) Success.
Exit Code: 0x00
[root at nodeb ~]# MegaCli ldinit start l2 a0
Start Initialization on Virtual Drive 2 (target id: 2) Success.
Exit Code: 0x00
(2) drbd start
[root at nodea ~]# service drbd start
Starting DRBD resources: [
create res: r0
prepare disk: r0
adjust disk: r0:failed(apply-al:255)
adjust net: r0
]
....
[root at nodeb ~]# service drbd start
Starting DRBD resources: [
create res: r0
prepare disk: r0
adjust disk: r0:failed(apply-al:255)
adjust net: r0
]
.
[root at nodea ~]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
[root at nodeb ~]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
=> Disk states is Diskless
(3) create metadata
[root at nodea ~]# drbdadm create-md r0
Writing meta data...
initializing activity log
NOT initializing bitmap
New drbd meta data block successfully created.
success
[root at nodeb ~]# drbdadm create-md r0
Writing meta data...
initializing activity log
NOT initializing bitmap
New drbd meta data block successfully created.
success
(4) attach backing device
[root at nodea ~]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
[root at nodea ~]# drbdadm attach r0
[root at nodea ~]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Diskless C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2147417308
[root at nodeb ~]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
0: cs:Connected ro:Secondary/Secondary ds:Diskless/Inconsistent C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
[root at nodeb ~]# drbdadm attach r0
[root at nodeb ~]# cat /proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
0: cs:SyncSource ro:Secondary/Secondary ds:UpToDate/Inconsistent C r---n-
ns:460616 nr:0 dw:0 dr:468248 al:0 bm:28 lo:6 pe:11 ua:80 ap:0 ep:1 wo:f oos:2146957764
[>....................] sync'ed: 0.1% (2096636/2097084)M
finish: 1:17:51 speed: 459,544 (459,544) K/sec
(5) log
[nodea]
Jul 10 10:22:03 nodea kernel: events: mcg drbd: 2
Jul 10 10:22:03 nodea kernel: drbd: initialized. Version: 8.4.3 (api:1/proto:86-101)
Jul 10 10:22:03 nodea kernel: drbd: GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
Jul 10 10:22:03 nodea kernel: drbd: registered as block device major 147
Jul 10 10:22:03 nodea kernel: d-con r0: Starting worker thread (from drbdsetup [32273])
Jul 10 10:22:03 nodea kernel: d-con r0: conn( StandAlone -> Unconnected )
Jul 10 10:22:03 nodea kernel: d-con r0: Starting receiver thread (from drbd_w_r0 [32275])
Jul 10 10:22:03 nodea kernel: d-con r0: receiver (re)started
Jul 10 10:22:03 nodea kernel: d-con r0: conn( Unconnected -> WFConnection )
Jul 10 10:22:05 nodea kernel: d-con r0: Handshake successful: Agreed network protocol version 101
Jul 10 10:22:05 nodea kernel: d-con r0: conn( WFConnection -> WFReportParams )
Jul 10 10:22:05 nodea kernel: d-con r0: Starting asender thread (from drbd_r_r0 [32276])
Jul 10 10:22:05 nodea kernel: block drbd0: max BIO size = 4096
Jul 10 10:22:05 nodea kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> Connected ) pdsk( DUnknown -> Diskless )
Jul 10 10:22:43 nodea kernel: block drbd0: disk( Diskless -> Attaching )
Jul 10 10:22:43 nodea kernel: d-con r0: Method to ensure write ordering: flush
Jul 10 10:22:43 nodea kernel: block drbd0: max BIO size = 286720
Jul 10 10:22:43 nodea kernel: block drbd0: drbd_bm_resize called with capacity == 4294834616
Jul 10 10:22:43 nodea kernel: block drbd0: resync bitmap: bits=536854327 words=8388349 pages=16384
Jul 10 10:22:43 nodea kernel: block drbd0: size = 2048 GB (2147417308 KB)
Jul 10 10:22:43 nodea kernel: block drbd0: Writing the whole bitmap, size changed
Jul 10 10:22:43 nodea kernel: block drbd0: bitmap WRITE of 16384 pages took 20 jiffies
Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
Jul 10 10:22:43 nodea kernel: block drbd0: bitmap READ of 16384 pages took 18 jiffies
Jul 10 10:22:43 nodea kernel: block drbd0: recounting of set bits took additional 28 jiffies
Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
Jul 10 10:22:43 nodea kernel: block drbd0: disk( Attaching -> Negotiating )
Jul 10 10:22:43 nodea kernel: block drbd0: attached to UUIDs 0000000000000004:0000000000000000:0000000000000000:0000000000000000
Jul 10 10:22:43 nodea kernel: block drbd0: disk( Negotiating -> Inconsistent )
Jul 10 10:22:59 nodea kernel: block drbd0: real peer disk state = Inconsistent
Jul 10 10:22:59 nodea kernel: block drbd0: drbd_sync_handshake:
Jul 10 10:22:59 nodea kernel: block drbd0: self 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:0
Jul 10 10:22:59 nodea kernel: block drbd0: peer 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:4
Jul 10 10:22:59 nodea kernel: block drbd0: uuid_compare()=0 by rule 10
Jul 10 10:22:59 nodea kernel: block drbd0: No resync, but 536854327 bits in bitmap!
Jul 10 10:22:59 nodea kernel: block drbd0: pdsk( Diskless -> Inconsistent )
Jul 10 10:22:59 nodea kernel: block drbd0: Resync of new storage after online grow
Jul 10 10:22:59 nodea kernel: block drbd0: conn( Connected -> WFSyncUUID ) pdsk( Inconsistent -> UpToDate )
Jul 10 10:22:59 nodea kernel: block drbd0: updated sync uuid 087F262FA3DE1BDE:0000000000000000:0000000000000000:0000000000000000
Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0
Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 exit code 0 (0x0)
Jul 10 10:22:59 nodea kernel: block drbd0: conn( WFSyncUUID -> SyncTarget )
Jul 10 10:22:59 nodea kernel: block drbd0: Began resync as SyncTarget (will sync 2147417308 KB [536854327 bits set]).
[nodeb]
Jul 10 10:22:05 nodeb kernel: events: mcg drbd: 2
Jul 10 10:22:05 nodeb kernel: drbd: initialized. Version: 8.4.3 (api:1/proto:86-101)
Jul 10 10:22:05 nodeb kernel: drbd: GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
Jul 10 10:22:05 nodeb kernel: drbd: registered as block device major 147
Jul 10 10:22:05 nodeb kernel: d-con r0: Starting worker thread (from drbdsetup [28581])
Jul 10 10:22:05 nodeb kernel: d-con r0: conn( StandAlone -> Unconnected )
Jul 10 10:22:05 nodeb kernel: d-con r0: Starting receiver thread (from drbd_w_r0 [28583])
Jul 10 10:22:05 nodeb kernel: d-con r0: receiver (re)started
Jul 10 10:22:05 nodeb kernel: d-con r0: conn( Unconnected -> WFConnection )
Jul 10 10:22:05 nodeb kernel: d-con r0: Handshake successful: Agreed network protocol version 101
Jul 10 10:22:05 nodeb kernel: d-con r0: conn( WFConnection -> WFReportParams )
Jul 10 10:22:05 nodeb kernel: d-con r0: Starting asender thread (from drbd_r_r0 [28584])
Jul 10 10:22:05 nodeb kernel: block drbd0: max BIO size = 4096
Jul 10 10:22:05 nodeb kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> Connected ) pdsk( DUnknown -> Diskless )
Jul 10 10:22:43 nodeb kernel: block drbd0: receiver updated UUIDs to effective data uuid: 0000000000000004
Jul 10 10:22:43 nodeb kernel: block drbd0: real peer disk state = Inconsistent
Jul 10 10:22:43 nodeb kernel: block drbd0: pdsk( Diskless -> Inconsistent )
Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Diskless -> Attaching )
Jul 10 10:22:59 nodeb kernel: d-con r0: Method to ensure write ordering: flush
Jul 10 10:22:59 nodeb kernel: block drbd0: max BIO size = 286720
Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_bm_resize called with capacity == 4294834616
Jul 10 10:22:59 nodeb kernel: block drbd0: resync bitmap: bits=536854327 words=8388349 pages=16384
Jul 10 10:22:59 nodeb kernel: block drbd0: size = 2048 GB (2147417308 KB)
Jul 10 10:22:59 nodeb kernel: block drbd0: Writing the whole bitmap, size changed
Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap WRITE of 16384 pages took 20 jiffies
Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap READ of 16384 pages took 19 jiffies
Jul 10 10:22:59 nodeb kernel: block drbd0: recounting of set bits took additional 29 jiffies
Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Attaching -> Negotiating )
Jul 10 10:22:59 nodeb kernel: block drbd0: attached to UUIDs 0000000000000004:0000000000000000:0000000000000000:0000000000000000
Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_sync_handshake:
Jul 10 10:22:59 nodeb kernel: block drbd0: self 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:0
Jul 10 10:22:59 nodeb kernel: block drbd0: peer 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:4
Jul 10 10:22:59 nodeb kernel: block drbd0: uuid_compare()=0 by rule 10
Jul 10 10:22:59 nodeb kernel: block drbd0: No resync, but 536854327 bits in bitmap!
Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Negotiating -> Inconsistent )
Jul 10 10:22:59 nodeb kernel: block drbd0: Resync of new storage after online grow
Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0
Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0 exit code 0 (0x0)
Jul 10 10:22:59 nodeb kernel: block drbd0: Implicitly upgraded disk
Jul 10 10:22:59 nodeb kernel: block drbd0: conn( Connected -> SyncSource ) disk( Inconsistent -> UpToDate )
Jul 10 10:22:59 nodeb kernel: block drbd0: Began resync as SyncSource (will sync 2147417308 KB [536854327 bits set]).
Jul 10 10:22:59 nodeb kernel: block drbd0: updated sync UUID 0000000000000004:087F262FA3DE1BDF:0000000000000000:0000000000000000
Regards,
--
Hiroshi Fujishima