Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Thu, Jul 11, 2013 at 05:32:00PM +0900, Hiroshi Fujishima wrote:
> Hi all,
>
> With drbd-8.4.3, I noticed that performing `drbdadm attach r0' with a
> newly-initialized empty disk causes initial full synchronization.
>
> In the following document, it seems it is necessary to perform `drbdadm
> primary --force resource' to start the initial full synchronization.
If you follow the proper sequence (*create-md* first, then attach, then
connect).
>
> http://www.drbd.org/users-guide-emb/s-initial-full-sync.html
>
> However, it starts without performing `drbdadm primary --force
> resource'.
>
> I'd like to know whether this is an expected behavior or not.
Actually yes, given your sequence of events, this is expected.
To DRBD, this lokks like "online grow" (because you started with device
size of 0):
> Jul 10 10:22:59 nodea kernel: block drbd0: Resync of new storage after online grow
A bit ugly, because apparently that way you you may end up with a current
uuid of "just created", while being "uptodate". This later detail should
be fixed.
Anyways: don't do that.
>
> * enviroment
>
> OS: Scientific Linux release 6.2 (Carbon)
> DRBD: drbd-8.4.3
>
> * configuration
>
> # /etc/drbd.conf
> common {
> }
>
> # resource r0 on nodea: not ignored, not stacked
> # defined at /etc/drbd.d/r0.res:1
> resource r0 {
> on nodea {
> volume 0 {
> device /dev/drbd0 minor 0;
> disk /dev/sdc;
> meta-disk internal;
> }
> address ipv4 192.168.0.1:7789;
> }
> on nodeb {
> volume 0 {
> device /dev/drbd0 minor 0;
> disk /dev/sdc;
> meta-disk internal;
> }
> address ipv4 192.168.0.2:7789;
> }
> net {
> protocol C;
> }
> disk {
> resync-rate 2g;
> c-plan-ahead 0;
> }
> }
>
> * How To reproduce
>
> (1) cleanup metadata in /dev/sdc
>
> [root at nodea ~]# MegaCli ldinit start l2 a0
>
> Start Initialization on Virtual Drive 2 (target id: 2) Success.
>
> Exit Code: 0x00
> [root at nodeb ~]# MegaCli ldinit start l2 a0
>
> Start Initialization on Virtual Drive 2 (target id: 2) Success.
>
> Exit Code: 0x00
>
> (2) drbd start
>
> [root at nodea ~]# service drbd start
> Starting DRBD resources: [
> create res: r0
> prepare disk: r0
> adjust disk: r0:failed(apply-al:255)
> adjust net: r0
> ]
> ....
> [root at nodeb ~]# service drbd start
> Starting DRBD resources: [
> create res: r0
> prepare disk: r0
> adjust disk: r0:failed(apply-al:255)
> adjust net: r0
> ]
> .
> [root at nodea ~]# cat /proc/drbd
> version: 8.4.3 (api:1/proto:86-101)
> GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
> 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
> [root at nodeb ~]# cat /proc/drbd
> version: 8.4.3 (api:1/proto:86-101)
> GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
> 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
>
> => Disk states is Diskless
>
> (3) create metadata
>
> [root at nodea ~]# drbdadm create-md r0
> Writing meta data...
> initializing activity log
> NOT initializing bitmap
> New drbd meta data block successfully created.
> success
> [root at nodeb ~]# drbdadm create-md r0
> Writing meta data...
> initializing activity log
> NOT initializing bitmap
> New drbd meta data block successfully created.
> success
>
> (4) attach backing device
>
> [root at nodea ~]# cat /proc/drbd
> version: 8.4.3 (api:1/proto:86-101)
> GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
> 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
> [root at nodea ~]# drbdadm attach r0
> [root at nodea ~]# cat /proc/drbd
> version: 8.4.3 (api:1/proto:86-101)
> GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
> 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Diskless C r-----
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2147417308
> [root at nodeb ~]# cat /proc/drbd
> version: 8.4.3 (api:1/proto:86-101)
> GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
> 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Inconsistent C r-----
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
> [root at nodeb ~]# drbdadm attach r0
> [root at nodeb ~]# cat /proc/drbd
> version: 8.4.3 (api:1/proto:86-101)
> GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
> 0: cs:SyncSource ro:Secondary/Secondary ds:UpToDate/Inconsistent C r---n-
> ns:460616 nr:0 dw:0 dr:468248 al:0 bm:28 lo:6 pe:11 ua:80 ap:0 ep:1 wo:f oos:2146957764
> [>....................] sync'ed: 0.1% (2096636/2097084)M
> finish: 1:17:51 speed: 459,544 (459,544) K/sec
>
> (5) log
>
> [nodea]
> Jul 10 10:22:03 nodea kernel: events: mcg drbd: 2
> Jul 10 10:22:03 nodea kernel: drbd: initialized. Version: 8.4.3 (api:1/proto:86-101)
> Jul 10 10:22:03 nodea kernel: drbd: GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodea, 2013-07-09 17:08:55
> Jul 10 10:22:03 nodea kernel: drbd: registered as block device major 147
> Jul 10 10:22:03 nodea kernel: d-con r0: Starting worker thread (from drbdsetup [32273])
> Jul 10 10:22:03 nodea kernel: d-con r0: conn( StandAlone -> Unconnected )
> Jul 10 10:22:03 nodea kernel: d-con r0: Starting receiver thread (from drbd_w_r0 [32275])
> Jul 10 10:22:03 nodea kernel: d-con r0: receiver (re)started
> Jul 10 10:22:03 nodea kernel: d-con r0: conn( Unconnected -> WFConnection )
> Jul 10 10:22:05 nodea kernel: d-con r0: Handshake successful: Agreed network protocol version 101
> Jul 10 10:22:05 nodea kernel: d-con r0: conn( WFConnection -> WFReportParams )
> Jul 10 10:22:05 nodea kernel: d-con r0: Starting asender thread (from drbd_r_r0 [32276])
> Jul 10 10:22:05 nodea kernel: block drbd0: max BIO size = 4096
> Jul 10 10:22:05 nodea kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> Connected ) pdsk( DUnknown -> Diskless )
> Jul 10 10:22:43 nodea kernel: block drbd0: disk( Diskless -> Attaching )
> Jul 10 10:22:43 nodea kernel: d-con r0: Method to ensure write ordering: flush
> Jul 10 10:22:43 nodea kernel: block drbd0: max BIO size = 286720
> Jul 10 10:22:43 nodea kernel: block drbd0: drbd_bm_resize called with capacity == 4294834616
> Jul 10 10:22:43 nodea kernel: block drbd0: resync bitmap: bits=536854327 words=8388349 pages=16384
> Jul 10 10:22:43 nodea kernel: block drbd0: size = 2048 GB (2147417308 KB)
> Jul 10 10:22:43 nodea kernel: block drbd0: Writing the whole bitmap, size changed
> Jul 10 10:22:43 nodea kernel: block drbd0: bitmap WRITE of 16384 pages took 20 jiffies
> Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
> Jul 10 10:22:43 nodea kernel: block drbd0: bitmap READ of 16384 pages took 18 jiffies
> Jul 10 10:22:43 nodea kernel: block drbd0: recounting of set bits took additional 28 jiffies
> Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
> Jul 10 10:22:43 nodea kernel: block drbd0: disk( Attaching -> Negotiating )
> Jul 10 10:22:43 nodea kernel: block drbd0: attached to UUIDs 0000000000000004:0000000000000000:0000000000000000:0000000000000000
> Jul 10 10:22:43 nodea kernel: block drbd0: disk( Negotiating -> Inconsistent )
> Jul 10 10:22:59 nodea kernel: block drbd0: real peer disk state = Inconsistent
> Jul 10 10:22:59 nodea kernel: block drbd0: drbd_sync_handshake:
> Jul 10 10:22:59 nodea kernel: block drbd0: self 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:0
> Jul 10 10:22:59 nodea kernel: block drbd0: peer 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:4
> Jul 10 10:22:59 nodea kernel: block drbd0: uuid_compare()=0 by rule 10
> Jul 10 10:22:59 nodea kernel: block drbd0: No resync, but 536854327 bits in bitmap!
> Jul 10 10:22:59 nodea kernel: block drbd0: pdsk( Diskless -> Inconsistent )
> Jul 10 10:22:59 nodea kernel: block drbd0: Resync of new storage after online grow
> Jul 10 10:22:59 nodea kernel: block drbd0: conn( Connected -> WFSyncUUID ) pdsk( Inconsistent -> UpToDate )
> Jul 10 10:22:59 nodea kernel: block drbd0: updated sync uuid 087F262FA3DE1BDE:0000000000000000:0000000000000000:0000000000000000
> Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0
> Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 exit code 0 (0x0)
> Jul 10 10:22:59 nodea kernel: block drbd0: conn( WFSyncUUID -> SyncTarget )
> Jul 10 10:22:59 nodea kernel: block drbd0: Began resync as SyncTarget (will sync 2147417308 KB [536854327 bits set]).
>
> [nodeb]
> Jul 10 10:22:05 nodeb kernel: events: mcg drbd: 2
> Jul 10 10:22:05 nodeb kernel: drbd: initialized. Version: 8.4.3 (api:1/proto:86-101)
> Jul 10 10:22:05 nodeb kernel: drbd: GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root at nodeb, 2013-07-09 17:10:11
> Jul 10 10:22:05 nodeb kernel: drbd: registered as block device major 147
> Jul 10 10:22:05 nodeb kernel: d-con r0: Starting worker thread (from drbdsetup [28581])
> Jul 10 10:22:05 nodeb kernel: d-con r0: conn( StandAlone -> Unconnected )
> Jul 10 10:22:05 nodeb kernel: d-con r0: Starting receiver thread (from drbd_w_r0 [28583])
> Jul 10 10:22:05 nodeb kernel: d-con r0: receiver (re)started
> Jul 10 10:22:05 nodeb kernel: d-con r0: conn( Unconnected -> WFConnection )
> Jul 10 10:22:05 nodeb kernel: d-con r0: Handshake successful: Agreed network protocol version 101
> Jul 10 10:22:05 nodeb kernel: d-con r0: conn( WFConnection -> WFReportParams )
> Jul 10 10:22:05 nodeb kernel: d-con r0: Starting asender thread (from drbd_r_r0 [28584])
> Jul 10 10:22:05 nodeb kernel: block drbd0: max BIO size = 4096
> Jul 10 10:22:05 nodeb kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> Connected ) pdsk( DUnknown -> Diskless )
> Jul 10 10:22:43 nodeb kernel: block drbd0: receiver updated UUIDs to effective data uuid: 0000000000000004
> Jul 10 10:22:43 nodeb kernel: block drbd0: real peer disk state = Inconsistent
> Jul 10 10:22:43 nodeb kernel: block drbd0: pdsk( Diskless -> Inconsistent )
> Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Diskless -> Attaching )
> Jul 10 10:22:59 nodeb kernel: d-con r0: Method to ensure write ordering: flush
> Jul 10 10:22:59 nodeb kernel: block drbd0: max BIO size = 286720
> Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_bm_resize called with capacity == 4294834616
> Jul 10 10:22:59 nodeb kernel: block drbd0: resync bitmap: bits=536854327 words=8388349 pages=16384
> Jul 10 10:22:59 nodeb kernel: block drbd0: size = 2048 GB (2147417308 KB)
> Jul 10 10:22:59 nodeb kernel: block drbd0: Writing the whole bitmap, size changed
> Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap WRITE of 16384 pages took 20 jiffies
> Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
> Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap READ of 16384 pages took 19 jiffies
> Jul 10 10:22:59 nodeb kernel: block drbd0: recounting of set bits took additional 29 jiffies
> Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map.
> Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Attaching -> Negotiating )
> Jul 10 10:22:59 nodeb kernel: block drbd0: attached to UUIDs 0000000000000004:0000000000000000:0000000000000000:0000000000000000
> Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_sync_handshake:
> Jul 10 10:22:59 nodeb kernel: block drbd0: self 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:0
> Jul 10 10:22:59 nodeb kernel: block drbd0: peer 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:4
> Jul 10 10:22:59 nodeb kernel: block drbd0: uuid_compare()=0 by rule 10
> Jul 10 10:22:59 nodeb kernel: block drbd0: No resync, but 536854327 bits in bitmap!
> Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Negotiating -> Inconsistent )
> Jul 10 10:22:59 nodeb kernel: block drbd0: Resync of new storage after online grow
> Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0
> Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0 exit code 0 (0x0)
> Jul 10 10:22:59 nodeb kernel: block drbd0: Implicitly upgraded disk
> Jul 10 10:22:59 nodeb kernel: block drbd0: conn( Connected -> SyncSource ) disk( Inconsistent -> UpToDate )
> Jul 10 10:22:59 nodeb kernel: block drbd0: Began resync as SyncSource (will sync 2147417308 KB [536854327 bits set]).
> Jul 10 10:22:59 nodeb kernel: block drbd0: updated sync UUID 0000000000000004:087F262FA3DE1BDF:0000000000000000:0000000000000000
>
> Regards,
>
> --
> Hiroshi Fujishima
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
__
please don't Cc me, but send to list -- I'm subscribed