[DRBD-user] Data corrupted after relocation of DRBD-resources
Den
drbdsys at made.net
Thu Nov 21 16:22:43 CET 2019
Hello,
this sounds no good...
Did you move your primary resource too during the relocation?
Il 19/11/19 22:45, kvaps ha scritto:
> Hi, today we faced with the weird issue during reloaction some of
> drbd-resources. Everywhere steps were the same: each resources had two
> replicase, one replica was removed, then another one was created on
> another node.
>
> Almost all of them started having data coruption. Hex dump of two
> similar drives shows zeroes on place where should be data.
>
> We're using LVM_THIN driver with LINSTOR.
>
> Kernel version: 4.15.18-18-pve
>
> DRBD module version:
>
> version: 9.0.19-1 (api:2/proto:86-115)
> GIT-hash: 6f5fa5d348a99e5eeb09d83c49853d72e614fd07 build by
> @eb2143fc2fef, 2019-07-26 11:40:44
> Transports (api:16): tcp (9.0.19-1)
>
> I inspected one of these resources, this resource is not used by
> anyone, usually it is used only in readonly mode just for copying data
> from it.
>
> What was done:
>
> We had a resource on m13c20, m13c14 nodes, then the next actions was done:
>
> 1) Resource was removed form m13c14 node
>
> 2) Resource was created on m13c15 node
>
> 3) Resource was removed from m13c15 node
>
> 4) Resource was created on m14c14 node
>
> No errors was reported (kernel logs and executed commands look further)
>
> Currently resource is looking like:
>
> +----------------------------------------------------+
> | ResourceName | Node | Port | Usage | State |
> |====================================================|
> | one-image-1901 | m13c20 | 7355 | Unused | UpToDate |
> | one-image-1901 | m14c14 | 7355 | Unused | UpToDate |
> +----------------------------------------------------+
>
> Commands and kernel logs:
>
> 1) linstor r d m13c14 one-image-1901
>
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901: State
> change 4153216007: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15:
> conn( Connected -> Disconnecting ) peer( Secondary -> Unknown )
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901/0
> drbd1355 m13c15: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15:
> Terminating ack_recv thread
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901:
> Preparing cluster-wide state change 4153216007 (0->1 496/16)
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15:
> Cluster is now split
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901:
> Committing cluster-wide state change 4153216007 (0ms)
> November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15:
> ack_receiver terminated
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901/0
> drbd1355: disk( UpToDate -> Detaching )
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
> Terminating receiver thread
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901/0
> drbd1355: drbd_bm_resize called with capacity == 0
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
> Connection closed
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
> Restarting sender thread
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
> Terminating sender thread
> November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901/0
> drbd1355: disk( Detaching -> Diskless )
> November 18th 2019, 11:42:58.006 |m13c14| drbd one-image-1901:
> Terminating worker thread
>
> November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
> Committing remote state change 4153216007 (primary_nodes=0)
> November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
> conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901/0
> drbd1355 m13c14: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
> conn( TearDown -> Disconnecting )
> November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
> Preparing remote state change 4153216007
> November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
> Terminating ack_recv thread
> November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
> ack_receiver terminated
> November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14:
> Terminating receiver thread
> November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14:
> Restarting sender thread
> November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14:
> Connection closed
> November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:42:57.994 |m13c15| drbd one-image-1901 m13c14:
> Terminating sender thread
> November 18th 2019, 11:42:58.019 |m13c15| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
>
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903: State
> change 3926620536: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2:
> ack_receiver terminated
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2:
> conn( Connected -> Disconnecting ) peer( Secondary -> Unknown )
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903:
> Committing cluster-wide state change 3926620536 (0ms)
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2:
> Cluster is now split
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903:
> Preparing cluster-wide state change 3926620536 (1->0 496/16)
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903/0
> drbd1358 m13c2: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2:
> Terminating ack_recv thread
> November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
> Terminating sender thread
> November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
> Connection closed
> November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
> Terminating receiver thread
> November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
> Restarting sender thread
> November 18th 2019, 11:42:58.954 |m13c20| drbd one-image-1903/0
> drbd1358: rs_discard_granularity changed to 524288
>
>
> 2) linstor r c one-image-1901 --auto-place 2 --replicas-on-same
> opennebula-1=true --replicas-on-different moonshot -s thindata
> (creates resource on m13c15)
>
> November 18th 2019, 11:46:07.159 |m13c15| drbd one-image-1901 m13c20:
> Starting sender thread (from drbdsetup [7079])
> November 18th 2019, 11:46:07.191 |m13c15| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 18th 2019, 11:46:07.219 |m13c15| drbd one-image-1901 m13c20:
> Starting receiver thread (from drbd_w_one-imag [29649])
> November 18th 2019, 11:46:07.219 |m13c15| drbd one-image-1901 m13c20:
> conn( Unconnected -> Connecting )
> November 18th 2019, 11:46:07.219 |m13c15| drbd one-image-1901 m13c20:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20:
> Starting ack_recv thread (from drbd_r_one-imag [7085])
> November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20:
> Handshake to peer 0 successful: Agreed network protocol version 115
> November 18th 2019, 11:46:07.943 |m13c15| drbd one-image-1901 m13c20:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 11:46:07.943 |m13c15| drbd one-image-1901 m13c20:
> Preparing remote state change 98375023
> November 18th 2019, 11:46:07.943 |m13c15| drbd one-image-1901 m13c20:
> Committing remote state change 98375023 (primary_nodes=0)
> November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: drbd_sync_handshake:
> November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: self
> D7ECA5851EFE55DE:4B6493096F3F93A6:0000000000000000:0000000000000000
> bits:64512 flags:20
> November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: peer
> 4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
> November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: uuid_compare()=2 by rule 70
> November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 25(1), total 25; compression: 99.9%
> November 18th 2019, 11:46:07.975 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: pdsk( Consistent -> Outdated )
> November 18th 2019, 11:46:07.975 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 25(1), total 25; compression: 99.9%
> November 18th 2019, 11:46:07.975 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: helper command: /sbin/drbdadm before-resync-source
> November 18th 2019, 11:46:07.976 |m13c15| Don't know which config file
> belongs to resource one-image-1901, trying default ones...
> November 18th 2019, 11:46:07.987 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: helper command: /sbin/drbdadm before-resync-source
> exit code 0 (0x0)
> November 18th 2019, 11:46:07.987 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: Began resync as SyncSource (will sync 258048 KB
> [64512 bits set]).
> November 18th 2019, 11:46:07.987 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS ->
> SyncSource )
> November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: updated UUIDs
> D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
> November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: Resync done (total 1 sec; paused 0 sec; 258048 K/sec)
> November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: 44 % had equal checksums, eliminated: 115436K;
> transferred 142612K total 258048K
> November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: pdsk( Inconsistent -> UpToDate ) repl( SyncSource ->
> Established )
>
> November 18th 2019, 11:46:01.753 |m13c20| drbd one-image-1883 m13c11:
> Starting sender thread (from drbdsetup [31258])
> November 18th 2019, 11:46:01.753 |m13c20| drbd one-image-1883:
> Starting worker thread (from drbdsetup [31253])
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0
> drbd1353: resync bitmap: bits=525190 words=57449 pages=113
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0
> drbd1353: rs_discard_granularity changed to 524288
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0
> drbd1353: drbd_bm_resize called with capacity == 4201520
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0
> drbd1353: size = 2052 MB (2100760 KB)
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0
> drbd1353: disk( Diskless -> Attaching )
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0
> drbd1353: meta-data IO uses: blk-bio
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0
> drbd1353: Maximum number of peer devices = 7
> November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883: Method
> to ensure write ordering: flush
> November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0
> drbd1353: size = 2052 MB (2100760 KB)
> November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0
> drbd1353: recounting of set bits took additional 0ms
> November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0
> drbd1353: disk( Attaching -> UpToDate )
> November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0
> drbd1353: attached to current UUID: 3454322884DFC87C
> November 18th 2019, 11:46:01.802 |m13c20| drbd one-image-1883 m13c11:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 11:46:01.803 |m13c20| drbd one-image-1883 m13c11:
> Starting receiver thread (from drbd_w_one-imag [31254])
> November 18th 2019, 11:46:01.803 |m13c20| drbd one-image-1883 m13c11:
> conn( Unconnected -> Connecting )
> November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11:
> Starting ack_recv thread (from drbd_r_one-imag [31275])
> November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 11:46:02.438 |m13c20| drbd one-image-1883:
> Preparing cluster-wide state change 3727902976 (0->1 499/146)
> November 18th 2019, 11:46:02.438 |m13c20| drbd one-image-1883: State
> change 3727902976: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:46:02.439 |m13c20| drbd one-image-1883:
> Committing cluster-wide state change 3727902976 (0ms)
> November 18th 2019, 11:46:02.439 |m13c20| drbd one-image-1883 m13c11:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 11:46:02.458 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: self
> 3454322884DFC87C:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 11:46:02.458 |m13c20| drbd one-image-1883/0
> drbd1353: disk( UpToDate -> Outdated )
> November 18th 2019, 11:46:02.458 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: drbd_sync_handshake:
> November 18th 2019, 11:46:02.459 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: uuid_compare()=-2 by rule 50
> November 18th 2019, 11:46:02.459 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 11:46:02.459 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: peer
> FBE6531F94A3E69A:3454322884DFC87C:0000000000000000:0000000000000000
> bits:262624 flags:20
> November 18th 2019, 11:46:02.477 |m13c20| Don't know which config file
> belongs to resource one-image-1883, trying default ones...
> November 18th 2019, 11:46:02.478 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 11:46:02.478 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:02.478 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: Began resync as SyncTarget (will sync 1050496 KB
> [262624 bits set]).
> November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0
> drbd1353: disk( Outdated -> Inconsistent )
> November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: 6 % had equal checksums, eliminated: 70996K;
> transferred 979500K total 1050496K
> November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: Resync done (total 1 sec; paused 0 sec; 1050496 K/sec)
> November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0
> drbd1353: disk( Inconsistent -> UpToDate )
> November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: repl( SyncTarget -> Established )
> November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: updated UUIDs
> FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
> November 18th 2019, 11:46:04.145 |m13c20| Don't know which config file
> belongs to resource one-image-1883, trying default ones...
> November 18th 2019, 11:46:04.146 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 11:46:04.150 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:07.330 |m13c20| drbd one-image-1901 m13c15:
> Starting sender thread (from drbdsetup [31507])
> November 18th 2019, 11:46:07.330 |m13c20| drbd one-image-1901:
> Starting worker thread (from drbdsetup [31501])
> November 18th 2019, 11:46:07.366 |m13c20| drbd one-image-1901/0
> drbd1355: meta-data IO uses: blk-bio
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: size = 504 MB (515948 KB)
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: recounting of set bits took additional 0ms
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: disk( Attaching -> UpToDate )
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: resync bitmap: bits=128987 words=14112 pages=28
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: drbd_bm_resize called with capacity == 1031896
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: size = 504 MB (515948 KB)
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: attached to current UUID: 4B6493096F3F93A6
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: disk( Diskless -> Attaching )
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901: Method
> to ensure write ordering: flush
> November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0
> drbd1355: Maximum number of peer devices = 7
> November 18th 2019, 11:46:07.370 |m13c20| drbd one-image-1901 m13c15:
> Starting receiver thread (from drbd_w_one-imag [31502])
> November 18th 2019, 11:46:07.370 |m13c20| drbd one-image-1901 m13c15:
> conn( Unconnected -> Connecting )
> November 18th 2019, 11:46:07.370 |m13c20| drbd one-image-1901 m13c15:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15:
> Starting ack_recv thread (from drbd_r_one-imag [31523])
> November 18th 2019, 11:46:07.946 |m13c20| drbd one-image-1901:
> Preparing cluster-wide state change 98375023 (0->1 499/146)
> November 18th 2019, 11:46:07.946 |m13c20| drbd one-image-1901: State
> change 98375023: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:46:07.946 |m13c20| drbd one-image-1901:
> Committing cluster-wide state change 98375023 (0ms)
> November 18th 2019, 11:46:07.947 |m13c20| drbd one-image-1901 m13c15:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 11:46:07.970 |m13c20| drbd one-image-1901/0
> drbd1355: disk( UpToDate -> Outdated )
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: peer
> D7ECA5851EFE55DE:4B6493096F3F93A6:0000000000000000:0000000000000000
> bits:64512 flags:20
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 25(1), total 25; compression: 99.9%
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: drbd_sync_handshake:
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: self
> 4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: uuid_compare()=-2 by rule 50
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 25(1), total 25; compression: 99.9%
> November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 11:46:07.974 |m13c20| Don't know which config file
> belongs to resource one-image-1901, trying default ones...
> November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: Began resync as SyncTarget (will sync 258048 KB
> [64512 bits set]).
> November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0
> drbd1355: disk( Outdated -> Inconsistent )
> November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 11:46:08.340 |m13c20| Don't know which config file
> belongs to resource one-image-1901, trying default ones...
> November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: Resync done (total 1 sec; paused 0 sec; 258048 K/sec)
> November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0
> drbd1355: disk( Inconsistent -> UpToDate )
> November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: repl( SyncTarget -> Established )
> November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: updated UUIDs
> D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
> November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: 44 % had equal checksums, eliminated: 115436K;
> transferred 142612K total 258048K
> November 18th 2019, 11:46:08.345 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:08.550 |m13c20| drbd one-image-1902:
> Starting worker thread (from drbdsetup [31618])
> November 18th 2019, 11:46:08.555 |m13c20| drbd one-image-1902 m13c17:
> Starting sender thread (from drbdsetup [31623])
> November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0
> drbd1356: meta-data IO uses: blk-bio
> November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0
> drbd1356: Maximum number of peer devices = 7
> November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0
> drbd1356: disk( Diskless -> Attaching )
> November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0
> drbd1356: rs_discard_granularity changed to 524288
> November 18th 2019, 11:46:08.599 |m13c20| drbd one-image-1902: Method
> to ensure write ordering: flush
> November 18th 2019, 11:46:08.599 |m13c20| drbd one-image-1902/0
> drbd1356: drbd_bm_resize called with capacity == 4201520
> November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0
> drbd1356: attached to current UUID: E55EC4875988BFDC
> November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0
> drbd1356: size = 2052 MB (2100760 KB)
> November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0
> drbd1356: recounting of set bits took additional 0ms
> November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0
> drbd1356: disk( Attaching -> UpToDate )
> November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0
> drbd1356: resync bitmap: bits=525190 words=57449 pages=113
> November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0
> drbd1356: size = 2052 MB (2100760 KB)
> November 18th 2019, 11:46:08.605 |m13c20| drbd one-image-1902 m13c17:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 11:46:08.612 |m13c20| drbd one-image-1902 m13c17:
> Starting receiver thread (from drbd_w_one-imag [31619])
> November 18th 2019, 11:46:08.612 |m13c20| drbd one-image-1902 m13c17:
> conn( Unconnected -> Connecting )
> November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17:
> Starting ack_recv thread (from drbd_r_one-imag [31645])
> November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 11:46:09.149 |m13c20| drbd one-image-1902:
> Preparing cluster-wide state change 2478793358 (0->1 499/146)
> November 18th 2019, 11:46:09.150 |m13c20| drbd one-image-1902 m13c17:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 11:46:09.150 |m13c20| drbd one-image-1902:
> Committing cluster-wide state change 2478793358 (4ms)
> November 18th 2019, 11:46:09.150 |m13c20| drbd one-image-1902: State
> change 2478793358: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:46:09.186 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: drbd_sync_handshake:
> November 18th 2019, 11:46:09.186 |m13c20| drbd one-image-1902/0
> drbd1356: disk( UpToDate -> Outdated )
> November 18th 2019, 11:46:09.186 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: self
> E55EC4875988BFDC:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: uuid_compare()=-2 by rule 50
> November 18th 2019, 11:46:09.187 |m13c20| Don't know which config file
> belongs to resource one-image-1902, trying default ones...
> November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: peer
> B298F86F65888842:E55EC4875988BFDC:0000000000000000:0000000000000000
> bits:262624 flags:20
> November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: Began resync as SyncTarget (will sync 1050496 KB
> [262624 bits set]).
> November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0
> drbd1356: disk( Outdated -> Inconsistent )
> November 18th 2019, 11:46:09.762 |m13c20| drbd one-image-1903 m13c41:
> Starting sender thread (from drbdsetup [31684])
> November 18th 2019, 11:46:09.794 |m13c20| drbd one-image-1903/0
> drbd1358: rs_discard_granularity changed to 524288
> November 18th 2019, 11:46:09.820 |m13c20| drbd one-image-1903 m13c41:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 11:46:09.866 |m13c20| drbd one-image-1903 m13c41:
> conn( Unconnected -> Connecting )
> November 18th 2019, 11:46:09.866 |m13c20| drbd one-image-1903 m13c41:
> Starting receiver thread (from drbd_w_one-imag [3407])
> November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41:
> Starting ack_recv thread (from drbd_r_one-imag [31696])
> November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41:
> Handshake to peer 0 successful: Agreed network protocol version 115
> November 18th 2019, 11:46:10.402 |m13c20| drbd one-image-1903 m13c41:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 11:46:10.402 |m13c20| drbd one-image-1903 m13c41:
> Committing remote state change 197112278 (primary_nodes=0)
> November 18th 2019, 11:46:10.402 |m13c20| drbd one-image-1903 m13c41:
> Preparing remote state change 197112278
> November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: uuid_compare()=2 by rule 70
> November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: drbd_sync_handshake:
> November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: peer
> 399F1FC3E453DEAC:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
> November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: self
> B584A09F5803C860:399F1FC3E453DEAC:0000000000000000:0000000000000000
> bits:262624 flags:20
> November 18th 2019, 11:46:10.492 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: pdsk( Consistent -> Outdated )
> November 18th 2019, 11:46:10.498 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: helper command: /sbin/drbdadm before-resync-source
> November 18th 2019, 11:46:10.498 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:10.499 |m13c20| Don't know which config file
> belongs to resource one-image-1903, trying default ones...
> November 18th 2019, 11:46:10.506 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: helper command: /sbin/drbdadm before-resync-source
> exit code 0 (0x0)
> November 18th 2019, 11:46:10.506 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: Began resync as SyncSource (will sync 1050496 KB
> [262624 bits set]).
> November 18th 2019, 11:46:10.506 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS ->
> SyncSource )
> November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: 14 % had equal checksums, eliminated: 151004K;
> transferred 899492K total 1050496K
> November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: updated UUIDs
> B298F86F65888842:0000000000000000:E55EC4875988BFDC:0000000000000000
> November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0
> drbd1356: disk( Inconsistent -> UpToDate )
> November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: Resync done (total 2 sec; paused 0 sec; 525248 K/sec)
> November 18th 2019, 11:46:11.351 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: repl( SyncTarget -> Established )
> November 18th 2019, 11:46:11.366 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 11:46:11.367 |m13c20| Don't know which config file
> belongs to resource one-image-1902, trying default ones...
> November 18th 2019, 11:46:11.378 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:13.878 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: updated UUIDs
> B584A09F5803C860:0000000000000000:399F1FC3E453DEAC:0000000000000000
> November 18th 2019, 11:46:13.882 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: 10 % had equal checksums, eliminated: 110592K;
> transferred 939904K total 1050496K
> November 18th 2019, 11:46:13.882 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: pdsk( Inconsistent -> UpToDate ) repl( SyncSource ->
> Established )
> November 18th 2019, 11:46:13.882 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: Resync done (total 3 sec; paused 0 sec; 350164 K/sec)
> November 18th 2019, 11:46:16.992 |m13c20| drbd one-image-1907:
> Starting worker thread (from drbdsetup [31845])
> November 18th 2019, 11:46:16.993 |m13c20| drbd one-image-1907 m13c22:
> Starting sender thread (from drbdsetup [31850])
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: meta-data IO uses: blk-bio
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: disk( Diskless -> Attaching )
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: resync bitmap: bits=525190 words=57449 pages=113
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: recounting of set bits took additional 0ms
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: rs_discard_granularity changed to 524288
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: Maximum number of peer devices = 7
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: size = 2052 MB (2100760 KB)
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907: Method
> to ensure write ordering: flush
> November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0
> drbd1359: drbd_bm_resize called with capacity == 4201520
> November 18th 2019, 11:46:17.019 |m13c20| drbd one-image-1907/0
> drbd1359: size = 2052 MB (2100760 KB)
> November 18th 2019, 11:46:17.019 |m13c20| drbd one-image-1907/0
> drbd1359: disk( Attaching -> UpToDate )
> November 18th 2019, 11:46:17.019 |m13c20| drbd one-image-1907/0
> drbd1359: attached to current UUID: 93AC1857F962278E
> November 18th 2019, 11:46:17.020 |m13c20| drbd one-image-1907 m13c22:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 11:46:17.020 |m13c20| drbd one-image-1907 m13c22:
> conn( Unconnected -> Connecting )
> November 18th 2019, 11:46:17.020 |m13c20| drbd one-image-1907 m13c22:
> Starting receiver thread (from drbd_w_one-imag [31846])
> November 18th 2019, 11:46:17.538 |m13c20| drbd one-image-1907 m13c22:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 11:46:17.538 |m13c20| drbd one-image-1907 m13c22:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 11:46:17.539 |m13c20| drbd one-image-1907 m13c22:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 11:46:17.539 |m13c20| drbd one-image-1907 m13c22:
> Starting ack_recv thread (from drbd_r_one-imag [31869])
> November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907 m13c22:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907: State
> change 1214514917: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907:
> Committing cluster-wide state change 1214514917 (0ms)
> November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907:
> Preparing cluster-wide state change 1214514917 (0->1 499/146)
> November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0
> drbd1359: disk( UpToDate -> Outdated )
> November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: self
> 93AC1857F962278E:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: uuid_compare()=-2 by rule 50
> November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: drbd_sync_handshake:
> November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: peer
> 92983D105D747C54:93AC1857F962278E:0000000000000000:0000000000000000
> bits:262624 flags:20
> November 18th 2019, 11:46:17.604 |m13c20| Don't know which config file
> belongs to resource one-image-1907, trying default ones...
> November 18th 2019, 11:46:17.608 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 11:46:17.608 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:17.608 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 25(1), total 25; compression: 100.0%
> November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0
> drbd1359: disk( Outdated -> Inconsistent )
> November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: Began resync as SyncTarget (will sync 1050496 KB
> [262624 bits set]).
> November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: 10 % had equal checksums, eliminated: 110784K;
> transferred 939712K total 1050496K
> November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: updated UUIDs
> 92983D105D747C54:0000000000000000:93AC1857F962278E:0000000000000000
> November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: repl( SyncTarget -> Established )
> November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: Resync done (total 1 sec; paused 0 sec; 1050496 K/sec)
> November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0
> drbd1359: disk( Inconsistent -> UpToDate )
> November 18th 2019, 11:46:19.357 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 11:46:19.357 |m13c20| Don't know which config file
> belongs to resource one-image-1907, trying default ones...
> November 18th 2019, 11:46:19.362 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:23.686 |m13c20| drbd one-image-1915:
> Starting worker thread (from drbdsetup [32126])
> November 18th 2019, 11:46:23.688 |m13c20| drbd one-image-1915 m13c37:
> Starting sender thread (from drbdsetup [32131])
> November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0
> drbd1370: meta-data IO uses: blk-bio
> November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0
> drbd1370: Maximum number of peer devices = 7
> November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0
> drbd1370: disk( Diskless -> Attaching )
> November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0
> drbd1370: drbd_bm_resize called with capacity == 41950384
> November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0
> drbd1370: rs_discard_granularity changed to 524288
> November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915: Method
> to ensure write ordering: flush
> November 18th 2019, 11:46:23.725 |m13c20| drbd one-image-1915/0
> drbd1370: resync bitmap: bits=5243798 words=573545 pages=1121
> November 18th 2019, 11:46:23.725 |m13c20| drbd one-image-1915/0
> drbd1370: size = 20 GB (20975192 KB)
> November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0
> drbd1370: disk( Attaching -> UpToDate )
> November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0
> drbd1370: attached to current UUID: 9217094A747642A4
> November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0
> drbd1370: recounting of set bits took additional 4ms
> November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0
> drbd1370: size = 20 GB (20975192 KB)
> November 18th 2019, 11:46:23.750 |m13c20| drbd one-image-1915 m13c37:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 11:46:23.754 |m13c20| drbd one-image-1915 m13c37:
> Starting receiver thread (from drbd_w_one-imag [32127])
> November 18th 2019, 11:46:23.754 |m13c20| drbd one-image-1915 m13c37:
> conn( Unconnected -> Connecting )
> November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37:
> Starting ack_recv thread (from drbd_r_one-imag [32148])
> November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915:
> Committing cluster-wide state change 3354442230 (0ms)
> November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915:
> Preparing cluster-wide state change 3354442230 (0->1 499/146)
> November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915 m13c37:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915: State
> change 3354442230: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:46:24.352 |m13c20| drbd one-image-1915/0
> drbd1370: disk( UpToDate -> Outdated )
> November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: self
> 9217094A747642A4:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: peer
> F80052D0186111B8:9217094A747642A4:0000000000000000:0000000000000000
> bits:2621920 flags:20
> November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: drbd_sync_handshake:
> November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: uuid_compare()=-2 by rule 50
> November 18th 2019, 11:46:24.365 |m13c20| Don't know which config file
> belongs to resource one-image-1915, trying default ones...
> November 18th 2019, 11:46:24.366 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 28(1), total 28; compression: 100.0%
> November 18th 2019, 11:46:24.366 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 11:46:24.366 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 28(1), total 28; compression: 100.0%
> November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: Began resync as SyncTarget (will sync 10487680 KB
> [2621920 bits set]).
> November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0
> drbd1370: disk( Outdated -> Inconsistent )
> November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 11:46:41.420 |m13c20| Don't know which config file
> belongs to resource one-image-1915, trying default ones...
> November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0
> drbd1370: disk( Inconsistent -> UpToDate )
> November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: 75 % had equal checksums, eliminated: 7930080K;
> transferred 2557600K total 10487680K
> November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: Resync done (total 17 sec; paused 0 sec; 616920 K/sec)
> November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: repl( SyncTarget -> Established )
> November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: updated UUIDs
> F80052D0186111B8:0000000000000000:9217094A747642A4:0000000000000000
> November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 11:46:41.426 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
>
>
> 3) linstor r d m13c15 one-image-1901
>
>
> November 18th 2019, 11:48:27.268 |m13c15| drbd one-image-1901 m13c20:
> Preparing remote state change 364736091
> November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20:
> Committing remote state change 364736091 (primary_nodes=0)
> November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20:
> conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901/0
> drbd1355 m13c20: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20:
> ack_receiver terminated
> November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20:
> Terminating ack_recv thread
> November 18th 2019, 11:48:27.271 |m13c15| drbd one-image-1901 m13c20:
> conn( TearDown -> Disconnecting )
> November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20:
> Connection closed
> November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20:
> Terminating receiver thread
> November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20:
> Restarting sender thread
> November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901 m13c20:
> Terminating sender thread
> November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901/0
> drbd1355: drbd_bm_resize called with capacity == 0
> November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901/0
> drbd1355: disk( Detaching -> Diskless )
> November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901/0
> drbd1355: disk( UpToDate -> Detaching )
> November 18th 2019, 11:48:27.351 |m13c15| drbd one-image-1901:
> Terminating worker thread
>
> November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11:
> Preparing remote state change 1038193810
> November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11:
> conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883/0
> drbd1353 m13c11: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11:
> ack_receiver terminated
> November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11:
> Committing remote state change 1038193810 (primary_nodes=0)
> November 18th 2019, 11:48:26.763 |m13c20| drbd one-image-1883 m13c11:
> Terminating ack_recv thread
> November 18th 2019, 11:48:26.782 |m13c20| drbd one-image-1883 m13c11:
> conn( TearDown -> Disconnecting )
> November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
> Connection closed
> November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
> Restarting sender thread
> November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
> Terminating receiver thread
> November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
> Terminating sender thread
> November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:48:26.838 |m13c20| drbd one-image-1883/0
> drbd1353: rs_discard_granularity changed to 524288
> November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901:
> Committing cluster-wide state change 364736091 (0ms)
> November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901 m13c15:
> conn( Connected -> Disconnecting ) peer( Secondary -> Unknown )
> November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901/0
> drbd1355 m13c15: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901:
> Preparing cluster-wide state change 364736091 (0->1 496/16)
> November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901 m13c15:
> Cluster is now split
> November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901: State
> change 364736091: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:48:27.271 |m13c20| drbd one-image-1901 m13c15:
> ack_receiver terminated
> November 18th 2019, 11:48:27.271 |m13c20| drbd one-image-1901 m13c15:
> Terminating ack_recv thread
> November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
> Terminating receiver thread
> November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
> Terminating sender thread
> November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
> Connection closed
> November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
> Restarting sender thread
> November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:48:27.346 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
> ack_receiver terminated
> November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902/0
> drbd1356 m13c17: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
> Terminating ack_recv thread
> November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
> Committing remote state change 4188838677 (primary_nodes=0)
> November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
> Preparing remote state change 4188838677
> November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
> conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 11:48:27.782 |m13c20| drbd one-image-1902 m13c17:
> conn( TearDown -> Disconnecting )
> November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
> Connection closed
> November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
> Terminating sender thread
> November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
> Terminating receiver thread
> November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
> Restarting sender thread
> November 18th 2019, 11:48:27.830 |m13c20| drbd one-image-1902/0
> drbd1356: rs_discard_granularity changed to 524288
> November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
> conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
> Terminating ack_recv thread
> November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
> Preparing remote state change 1248155320
> November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
> Committing remote state change 1248155320 (primary_nodes=0)
> November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
> ack_receiver terminated
> November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903/0
> drbd1358 m13c41: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:48:28.302 |m13c20| drbd one-image-1903 m13c41:
> conn( TearDown -> Disconnecting )
> November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
> Terminating sender thread
> November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
> Restarting sender thread
> November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
> Connection closed
> November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903/0
> drbd1358: disk( UpToDate -> Detaching )
> November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
> Terminating receiver thread
> November 18th 2019, 11:48:28.352 |m13c20| drbd one-image-1903/0
> drbd1358: disk( Detaching -> Diskless )
> November 18th 2019, 11:48:28.352 |m13c20| drbd one-image-1903/0
> drbd1358: drbd_bm_resize called with capacity == 0
> November 18th 2019, 11:48:28.382 |m13c20| drbd one-image-1903:
> Terminating worker thread
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22:
> Terminating ack_recv thread
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907:
> Committing cluster-wide state change 3182775954 (0ms)
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907:
> Preparing cluster-wide state change 3182775954 (0->1 496/16)
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907: State
> change 3182775954: primary_nodes=0, weak_nodes=0
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22:
> Cluster is now split
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22:
> ack_receiver terminated
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22:
> conn( Connected -> Disconnecting ) peer( Secondary -> Unknown )
> November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907/0
> drbd1359 m13c22: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
> Restarting sender thread
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907/0
> drbd1359: disk( UpToDate -> Detaching )
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
> Connection closed
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907/0
> drbd1359: drbd_bm_resize called with capacity == 0
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907/0
> drbd1359: disk( Detaching -> Diskless )
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
> Terminating receiver thread
> November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
> Terminating sender thread
> November 18th 2019, 11:48:30.438 |m13c20| drbd one-image-1907:
> Terminating worker thread
> November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
> ack_receiver terminated
> November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
> Preparing remote state change 2963804021
> November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
> Terminating ack_recv thread
> November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
> conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
> Committing remote state change 2963804021 (primary_nodes=0)
> November 18th 2019, 11:48:33.198 |m13c20| drbd one-image-1915 m13c37:
> conn( TearDown -> Disconnecting )
> November 18th 2019, 11:48:33.243 |m13c20| drbd one-image-1915 m13c37:
> Connection closed
> November 18th 2019, 11:48:33.243 |m13c20| drbd one-image-1915 m13c37:
> Restarting sender thread
> November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915 m13c37:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915 m13c37:
> Terminating sender thread
> November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915 m13c37:
> Terminating receiver thread
> November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915/0
> drbd1370: disk( Detaching -> Diskless )
> November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915/0
> drbd1370: drbd_bm_resize called with capacity == 0
> November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915/0
> drbd1370: disk( UpToDate -> Detaching )
> November 18th 2019, 11:48:33.282 |m13c20| drbd one-image-1915:
> Terminating worker thread
>
>
> 4) linstor r c one-image-1901 --auto-place 2 --replicas-on-same
> opennebula-1=true moonshot=14 -s thindata
> (creates resource on m14c14)
>
>
> November 18th 2019, 12:06:57.002 |m14c14| drbd one-image-1883:
> Starting worker thread (from drbdsetup [2120])
> November 18th 2019, 12:06:57.006 |m14c14| drbd one-image-1883 m13c20:
> Starting sender thread (from drbdsetup [2125])
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0
> drbd1353: meta-data IO uses: blk-bio
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0
> drbd1353: resync bitmap: bits=525190 words=57449 pages=113
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883: Method
> to ensure write ordering: flush
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0
> drbd1353: size = 2052 MB (2100760 KB)
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0
> drbd1353: disk( Diskless -> Attaching )
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0
> drbd1353: drbd_bm_resize called with capacity == 4201520
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0
> drbd1353: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0
> drbd1353: Maximum number of peer devices = 7
> November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0
> drbd1353: attached to current UUID: 3454322884DFC87C
> November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0
> drbd1353: recounting of set bits took additional 0ms
> November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0
> drbd1353: size = 2052 MB (2100760 KB)
> November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0
> drbd1353: disk( Attaching -> UpToDate )
> November 18th 2019, 12:06:57.038 |m14c14| drbd one-image-1883 m13c20:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 12:06:57.038 |m14c14| drbd one-image-1883 m13c20:
> conn( Unconnected -> Connecting )
> November 18th 2019, 12:06:57.038 |m14c14| drbd one-image-1883 m13c20:
> Starting receiver thread (from drbd_w_one-imag [2121])
> November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20:
> Starting ack_recv thread (from drbd_r_one-imag [2140])
> November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20:
> Handshake to peer 0 successful: Agreed network protocol version 115
> November 18th 2019, 12:06:57.618 |m14c14| drbd one-image-1883 m13c20:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:06:57.618 |m14c14| drbd one-image-1883 m13c20:
> Committing remote state change 2284081876 (primary_nodes=0)
> November 18th 2019, 12:06:57.618 |m14c14| drbd one-image-1883 m13c20:
> Preparing remote state change 2284081876
> November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: peer
> FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
> bits:244875 flags:20
> November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: uuid_compare()=-3 by rule 60
> November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: drbd_sync_handshake:
> November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: self
> 3454322884DFC87C:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0
> drbd1353: disk( UpToDate -> Outdated )
> November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: Writing the whole bitmap, full sync required after
> drbd_sync_handshake.
> November 18th 2019, 12:06:57.642 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 12:06:57.642 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:06:57.645 |m14c14| Don't know which config file
> belongs to resource one-image-1883, trying default ones...
> November 18th 2019, 12:06:57.646 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 12:06:57.646 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0
> drbd1353: disk( Outdated -> Inconsistent )
> November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: Began resync as SyncTarget (will sync 2100760 KB
> [525190 bits set]).
> November 18th 2019, 12:06:58.554 |m14c14| drbd one-image-1901:
> Starting worker thread (from drbdsetup [2319])
> November 18th 2019, 12:06:58.558 |m14c14| drbd one-image-1901 m13c20:
> Starting sender thread (from drbdsetup [2324])
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: Maximum number of peer devices = 7
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: resync bitmap: bits=128987 words=14112 pages=28
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901: Method
> to ensure write ordering: flush
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: disk( Diskless -> Attaching )
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: drbd_bm_resize called with capacity == 1031896
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: size = 504 MB (515948 KB)
> November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: meta-data IO uses: blk-bio
> November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0
> drbd1355: disk( Attaching -> UpToDate )
> November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0
> drbd1355: recounting of set bits took additional 0ms
> November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0
> drbd1355: attached to current UUID: 4B6493096F3F93A6
> November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0
> drbd1355: size = 504 MB (515948 KB)
> November 18th 2019, 12:06:58.638 |m14c14| drbd one-image-1901 m13c20:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 12:06:58.709 |m14c14| drbd one-image-1901 m13c20:
> Starting receiver thread (from drbd_w_one-imag [2320])
> November 18th 2019, 12:06:58.709 |m14c14| drbd one-image-1901 m13c20:
> conn( Unconnected -> Connecting )
> November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20:
> Starting ack_recv thread (from drbd_r_one-imag [2350])
> November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20:
> Handshake to peer 0 successful: Agreed network protocol version 115
> November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:06:59.334 |m14c14| drbd one-image-1901 m13c20:
> Preparing remote state change 2709062739
> November 18th 2019, 12:06:59.334 |m14c14| drbd one-image-1901 m13c20:
> Committing remote state change 2709062739 (primary_nodes=0)
> November 18th 2019, 12:06:59.334 |m14c14| drbd one-image-1901 m13c20:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:06:59.358 |m14c14| drbd one-image-1901/0
> drbd1355: disk( UpToDate -> Outdated )
> November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: peer
> D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
> bits:35653 flags:20
> November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: uuid_compare()=-3 by rule 60
> November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: Writing the whole bitmap, full sync required after
> drbd_sync_handshake.
> November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: drbd_sync_handshake:
> November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: self
> 4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:06:59.446 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 12:06:59.448 |m14c14| Don't know which config file
> belongs to resource one-image-1901, trying default ones...
> November 18th 2019, 12:06:59.450 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 21(1), total 21; compression: 99.9%
> November 18th 2019, 12:06:59.450 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 21(1), total 21; compression: 99.9%
> November 18th 2019, 12:06:59.450 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0
> drbd1355: disk( Outdated -> Inconsistent )
> November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: Began resync as SyncTarget (will sync 515948 KB
> [128987 bits set]).
> November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: Resync done (total 1 sec; paused 0 sec; 515948 K/sec)
> November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: updated UUIDs
> D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
> November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0
> drbd1355: disk( Inconsistent -> UpToDate )
> November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: repl( SyncTarget -> Established )
> November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: 72 % had equal checksums, eliminated: 372660K;
> transferred 143288K total 515948K
> November 18th 2019, 12:07:00.331 |m14c14| Don't know which config file
> belongs to resource one-image-1901, trying default ones...
> November 18th 2019, 12:07:00.334 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 12:07:00.338 |m14c14| drbd one-image-1901/0
> drbd1355 m13c20: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: updated UUIDs
> FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
> November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: repl( SyncTarget -> Established )
> November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: 53 % had equal checksums, eliminated: 1120792K;
> transferred 979968K total 2100760K
> November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0
> drbd1353: disk( Inconsistent -> UpToDate )
> November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
> November 18th 2019, 12:07:01.113 |m14c14| Don't know which config file
> belongs to resource one-image-1883, trying default ones...
> November 18th 2019, 12:07:01.114 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 12:07:01.116 |m14c14| drbd one-image-1883/0
> drbd1353 m13c20: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:07:04.574 |m14c14| drbd one-image-1905:
> Starting worker thread (from drbdsetup [2722])
> November 18th 2019, 12:07:04.575 |m14c14| drbd one-image-1905 m13c44:
> Starting sender thread (from drbdsetup [2728])
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0
> drbd1360: rs_discard_granularity changed to 524288
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0
> drbd1360: meta-data IO uses: blk-bio
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905: Method
> to ensure write ordering: flush
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0
> drbd1360: size = 2052 MB (2100760 KB)
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0
> drbd1360: Maximum number of peer devices = 7
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0
> drbd1360: resync bitmap: bits=525190 words=57449 pages=113
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0
> drbd1360: disk( Diskless -> Attaching )
> November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0
> drbd1360: drbd_bm_resize called with capacity == 4201520
> November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905 m13c44:
> conn( Unconnected -> Connecting )
> November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0
> drbd1360: disk( Attaching -> UpToDate )
> November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0
> drbd1360: attached to current UUID: 0832983AA7F2AF02
> November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0
> drbd1360: recounting of set bits took additional 0ms
> November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0
> drbd1360: size = 2052 MB (2100760 KB)
> November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905 m13c44:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905 m13c44:
> Starting receiver thread (from drbd_w_one-imag [2723])
> November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44:
> Handshake to peer 0 successful: Agreed network protocol version 115
> November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44:
> Starting ack_recv thread (from drbd_r_one-imag [2743])
> November 18th 2019, 12:07:05.170 |m14c14| drbd one-image-1905 m13c44:
> Preparing remote state change 3787063659
> November 18th 2019, 12:07:05.170 |m14c14| drbd one-image-1905 m13c44:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:07:05.170 |m14c14| drbd one-image-1905 m13c44:
> Committing remote state change 3787063659 (primary_nodes=0)
> November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0
> drbd1360: disk( UpToDate -> Outdated )
> November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: drbd_sync_handshake:
> November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: uuid_compare()=-3 by rule 60
> November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: Writing the whole bitmap, full sync required after
> drbd_sync_handshake.
> November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: peer
> C8AAB1C5FAA5A5B0:0000000000000000:0832983AA7F2AF02:0000000000000000
> bits:236512 flags:20
> November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: self
> 0832983AA7F2AF02:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:07:05.206 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:07:05.206 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 12:07:05.207 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 12:07:05.207 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:07:05.209 |m14c14| Don't know which config file
> belongs to resource one-image-1905, trying default ones...
> November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: Began resync as SyncTarget (will sync 2100760 KB
> [525190 bits set]).
> November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0
> drbd1360: disk( Outdated -> Inconsistent )
> November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:07:06.082 |m14c14| drbd one-image-1906:
> Starting worker thread (from drbdsetup [3020])
> November 18th 2019, 12:07:06.086 |m14c14| drbd one-image-1906 m13c44:
> Starting sender thread (from drbdsetup [3026])
> November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0
> drbd1362: rs_discard_granularity changed to 524288
> November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0
> drbd1362: disk( Diskless -> Attaching )
> November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0
> drbd1362: meta-data IO uses: blk-bio
> November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0
> drbd1362: Maximum number of peer devices = 7
> November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906/0
> drbd1362: drbd_bm_resize called with capacity == 4201520
> November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906/0
> drbd1362: resync bitmap: bits=525190 words=57449 pages=113
> November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906/0
> drbd1362: size = 2052 MB (2100760 KB)
> November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906: Method
> to ensure write ordering: flush
> November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0
> drbd1362: disk( Attaching -> UpToDate )
> November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0
> drbd1362: attached to current UUID: 4EBBBE865C77D32C
> November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0
> drbd1362: size = 2052 MB (2100760 KB)
> November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0
> drbd1362: recounting of set bits took additional 0ms
> November 18th 2019, 12:07:06.186 |m14c14| drbd one-image-1906 m13c44:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 12:07:06.194 |m14c14| drbd one-image-1906 m13c44:
> Starting receiver thread (from drbd_w_one-imag [3021])
> November 18th 2019, 12:07:06.194 |m14c14| drbd one-image-1906 m13c44:
> conn( Unconnected -> Connecting )
> November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44:
> Handshake to peer 0 successful: Agreed network protocol version 115
> November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44:
> Starting ack_recv thread (from drbd_r_one-imag [3043])
> November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:07:06.758 |m14c14| drbd one-image-1906 m13c44:
> Committing remote state change 2466514477 (primary_nodes=0)
> November 18th 2019, 12:07:06.758 |m14c14| drbd one-image-1906 m13c44:
> Preparing remote state change 2466514477
> November 18th 2019, 12:07:06.758 |m14c14| drbd one-image-1906 m13c44:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:07:06.782 |m14c14| drbd one-image-1906/0
> drbd1362: disk( UpToDate -> Outdated )
> November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: Writing the whole bitmap, full sync required after
> drbd_sync_handshake.
> November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: self
> 4EBBBE865C77D32C:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: drbd_sync_handshake:
> November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: peer
> 86A2611E5BFC6FB2:0000000000000000:4EBBBE865C77D32C:0000000000000000
> bits:235872 flags:20
> November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: uuid_compare()=-3 by rule 60
> November 18th 2019, 12:07:06.850 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
> November 18th 2019, 12:07:06.913 |m14c14| Don't know which config file
> belongs to resource one-image-1906, trying default ones...
> November 18th 2019, 12:07:06.914 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: helper command: /sbin/drbdadm before-resync-target
> November 18th 2019, 12:07:06.914 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:07:06.914 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: repl( WFBitMapT -> SyncTarget )
> November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: helper command: /sbin/drbdadm before-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0
> drbd1362: disk( Outdated -> Inconsistent )
> November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: Began resync as SyncTarget (will sync 2100760 KB
> [525190 bits set]).
> November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: 55 % had equal checksums, eliminated: 1155240K;
> transferred 945520K total 2100760K
> November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: repl( SyncTarget -> Established )
> November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: updated UUIDs
> C8AAB1C5FAA5A5B0:0000000000000000:0832983AA7F2AF02:0000000000000000
> November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0
> drbd1360: disk( Inconsistent -> UpToDate )
> November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
> November 18th 2019, 12:07:09.194 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 12:07:09.195 |m14c14| Don't know which config file
> belongs to resource one-image-1905, trying default ones...
> November 18th 2019, 12:07:09.214 |m14c14| drbd one-image-1905/0
> drbd1360 m13c44: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:07:10.640 |m14c14| Don't know which config file
> belongs to resource one-image-1906, trying default ones...
> November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
> November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: 55 % had equal checksums, eliminated: 1157144K;
> transferred 943616K total 2100760K
> November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: updated UUIDs
> 86A2611E5BFC6FB2:0000000000000000:4EBBBE865C77D32C:0000000000000000
> November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: helper command: /sbin/drbdadm after-resync-target
> November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: repl( SyncTarget -> Established )
> November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0
> drbd1362: disk( Inconsistent -> UpToDate )
> November 18th 2019, 12:07:10.646 |m14c14| drbd one-image-1906/0
> drbd1362 m13c44: helper command: /sbin/drbdadm after-resync-target
> exit code 0 (0x0)
> November 18th 2019, 12:09:25.542 |m14c14| drbd one-vm-5440-disk-0:
> Starting worker thread (from drbdsetup [5177])
> November 18th 2019, 12:09:25.550 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Starting sender thread (from drbdsetup [5182])
> November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: drbd_bm_resize called with capacity == 125834928
> November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0:
> Method to ensure write ordering: flush
> November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: meta-data IO uses: blk-bio
> November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: disk( Diskless -> Attaching )
> November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: Maximum number of peer devices = 7
> November 18th 2019, 12:09:25.622 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: size = 60 GB (62917464 KB)
> November 18th 2019, 12:09:25.622 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: resync bitmap: bits=15729366 words=1720404 pages=3361
> November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: recounting of set bits took additional 12ms
> November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: size = 60 GB (62917464 KB)
> November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: attached to current UUID: 0C79F47BDA4C7504
> November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: disk( Attaching -> UpToDate )
> November 18th 2019, 12:09:25.666 |m14c14| drbd one-vm-5440-disk-0
> m13c37: conn( StandAlone -> Unconnected )
> November 18th 2019, 12:09:25.666 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Starting receiver thread (from drbd_w_one-vm-5 [5178])
> November 18th 2019, 12:09:25.666 |m14c14| drbd one-vm-5440-disk-0
> m13c37: conn( Unconnected -> Connecting )
> November 18th 2019, 12:09:25.842 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Starting ack_recv thread (from drbd_r_one-vm-5 [5210])
> November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Handshake to peer 0 successful: Agreed network protocol
> version 115
> November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:09:26.270 |m14c14| drbd one-vm-5440-disk-0
> m13c37: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:09:26.270 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Committing remote state change 3586394475 (primary_nodes=0)
> November 18th 2019, 12:09:26.270 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Preparing remote state change 3586394475
> November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m13c37: self
> 0C79F47BDA4C7504:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m13c37: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
> November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m13c37: uuid_compare()=0 by rule 40
> November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m13c37: drbd_sync_handshake:
> November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m13c37: peer
> 0C79F47BDA4C7504:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:09:26.562 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:09:26.746 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:09:26.962 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:09:27.458 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Starting sender thread (from drbdsetup [5391])
> November 18th 2019, 12:09:27.482 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:09:27.534 |m14c14| drbd one-vm-5440-disk-0
> m5c14: conn( StandAlone -> Unconnected )
> November 18th 2019, 12:09:27.534 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Starting receiver thread (from drbd_w_one-vm-5 [5178])
> November 18th 2019, 12:09:27.534 |m14c14| drbd one-vm-5440-disk-0
> m5c14: conn( Unconnected -> Connecting )
> November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Starting ack_recv thread (from drbd_r_one-vm-5 [5397])
> November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Handshake to peer 2 successful: Agreed network protocol version 114
> November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:09:28.138 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Preparing remote state change 576618137
> November 18th 2019, 12:09:28.138 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Committing remote state change 576618137 (primary_nodes=0)
> November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0:
> Preparing cluster-wide state change 449952009 (1->2 499/146)
> November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0:
> State change 449952009: primary_nodes=0, weak_nodes=0
> November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0
> m5c14: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0:
> Committing cluster-wide state change 449952009 (0ms)
> November 18th 2019, 12:09:28.274 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m5c14: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
> November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0
> m5c14: peer( Secondary -> Primary )
> November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Preparing remote state change 2365784547
> November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m5c14: received new current UUID: E8E1C74482F2EB05
> weak_nodes=FFFFFFFFFFFFFFF8
> November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Committing remote state change 2365784547 (primary_nodes=4)
> November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0
> m5c14: peer( Primary -> Secondary )
> November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Preparing remote state change 2409146247
> November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0
> m5c14: peer( Secondary -> Primary )
> November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Committing remote state change 2409146247 (primary_nodes=4)
> November 18th 2019, 12:09:33.658 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m5c14: received new current UUID: 6C91FAC1689A1645
> weak_nodes=FFFFFFFFFFFFFFF8
> November 18th 2019, 12:11:05.006 |m14c14| WARNING: Thin pool
> data-thindata-tpool data is now 80.07% full.
> November 18th 2019, 12:21:45.015 |m14c14| WARNING: Thin pool
> data-thindata-tpool data is now 85.05% full.
> November 18th 2019, 12:23:54.646 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Starting sender thread (from drbdsetup [19016])
> November 18th 2019, 12:23:54.694 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:23:54.746 |m14c14| drbd one-vm-5440-disk-0
> m10c3: conn( StandAlone -> Unconnected )
> November 18th 2019, 12:23:54.746 |m14c14| drbd one-vm-5440-disk-0
> m10c3: conn( Unconnected -> Connecting )
> November 18th 2019, 12:23:54.746 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Starting receiver thread (from drbd_w_one-vm-5 [5178])
> November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Starting ack_recv thread (from drbd_r_one-vm-5 [19023])
> November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Handshake to peer 3 successful: Agreed network protocol version 115
> November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:23:55.330 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Committing remote state change 2626253254 (primary_nodes=4)
> November 18th 2019, 12:23:55.330 |m14c14| drbd one-vm-5440-disk-0
> m13c37: Preparing remote state change 2626253254
> November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0:
> State change 3131838755: primary_nodes=4, weak_nodes=FFFFFFFFFFFFFFF8
> November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0:
> Preparing cluster-wide state change 3131838755 (1->3 499/146)
> November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0:
> Committing cluster-wide state change 3131838755 (0ms)
> November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0
> m10c3: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:23:55.398 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m10c3: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
> November 18th 2019, 12:23:55.698 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:23:57.038 |m14c14| drbd one-vm-5440-disk-0
> m10c3: peer( Secondary -> Primary )
> November 18th 2019, 12:23:57.038 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Preparing remote state change 1644389047
> November 18th 2019, 12:23:57.038 |m14c14| drbd one-vm-5440-disk-0
> m10c3: Committing remote state change 1644389047 (primary_nodes=C)
> November 18th 2019, 12:23:57.058 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m10c3: received new current UUID: BF79E942E02DA943
> weak_nodes=FFFFFFFFFFFFFFF4
> November 18th 2019, 12:24:04.138 |m14c14| drbd one-vm-5440-disk-0
> m5c14: peer( Primary -> Secondary )
> November 18th 2019, 12:24:04.146 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: bitmap WRITE of 252 pages took 8 ms
> November 18th 2019, 12:24:06.962 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:24:07.374 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Committing remote state change 396495173 (primary_nodes=8)
> November 18th 2019, 12:24:07.374 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Preparing remote state change 396495173
> November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Preparing remote state change 2772750783
> November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Terminating ack_recv thread
> November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0
> m5c14: conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Committing remote state change 2772750783 (primary_nodes=8)
> November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372 m5c14: pdsk( Diskless -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0
> m5c14: ack_receiver terminated
> November 18th 2019, 12:24:07.438 |m14c14| drbd one-vm-5440-disk-0
> m5c14: conn( TearDown -> Disconnecting )
> November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Restarting sender thread
> November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0/0
> drbd2372: rs_discard_granularity changed to 524288
> November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Terminating sender thread
> November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Connection closed
> November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0
> m5c14: conn( Disconnecting -> StandAlone )
> November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0
> m5c14: Terminating receiver thread
>
> November 18th 2019, 12:03:13.710 |m13c20| drbd one-vm-5444-disk-0:
> Starting worker thread (from drbdsetup [17328])
> November 18th 2019, 12:03:13.714 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Starting sender thread (from drbdsetup [17333])
> November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: meta-data IO uses: blk-bio
> November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: Maximum number of peer devices = 7
> November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: disk( Diskless -> Attaching )
> November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: size = 15 GB (15729336 KB)
> November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0:
> Method to ensure write ordering: flush
> November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: drbd_bm_resize called with capacity == 31458672
> November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: resync bitmap: bits=3932334 words=430101 pages=841
> November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: recounting of set bits took additional 0ms
> November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: attached to current UUID: CDCBD9A0E058BAC4
> November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: disk( Attaching -> UpToDate )
> November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: size = 15 GB (15729336 KB)
> November 18th 2019, 12:03:13.790 |m13c20| drbd one-vm-5444-disk-0
> m14c16: conn( StandAlone -> Unconnected )
> November 18th 2019, 12:03:13.790 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Starting receiver thread (from drbd_w_one-vm-5 [17329])
> November 18th 2019, 12:03:13.790 |m13c20| drbd one-vm-5444-disk-0
> m14c16: conn( Unconnected -> Connecting )
> November 18th 2019, 12:03:13.982 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:03:14.317 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Handshake to peer 1 successful: Agreed network protocol
> version 115
> November 18th 2019, 12:03:14.317 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:03:14.318 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:03:14.318 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Starting ack_recv thread (from drbd_r_one-vm-5 [17349])
> November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0
> m14c16: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0:
> Committing cluster-wide state change 1480358298 (0ms)
> November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0:
> Preparing cluster-wide state change 1480358298 (0->1 499/146)
> November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0:
> State change 1480358298: primary_nodes=0, weak_nodes=0
> November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m14c16: self
> CDCBD9A0E058BAC4:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m14c16: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
> November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m14c16: uuid_compare()=0 by rule 40
> November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m14c16: drbd_sync_handshake:
> November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m14c16: peer
> CDCBD9A0E058BAC4:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:03:14.694 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:03:14.914 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:03:15.126 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:03:15.622 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Starting sender thread (from drbdsetup [17513])
> November 18th 2019, 12:03:15.646 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:03:15.687 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( StandAlone -> Unconnected )
> November 18th 2019, 12:03:15.687 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Starting receiver thread (from drbd_w_one-vm-5 [17329])
> November 18th 2019, 12:03:15.688 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( Unconnected -> Connecting )
> November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Starting ack_recv thread (from drbd_r_one-vm-5 [17519])
> November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Handshake to peer 2 successful: Agreed network protocol version 114
> November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:03:16.230 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Preparing remote state change 1510321921
> November 18th 2019, 12:03:16.230 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Committing remote state change 1510321921 (primary_nodes=0)
> November 18th 2019, 12:03:16.410 |m13c20| drbd one-vm-5444-disk-0:
> Preparing cluster-wide state change 1326202137 (0->2 499/146)
> November 18th 2019, 12:03:16.411 |m13c20| drbd one-vm-5444-disk-0:
> Committing cluster-wide state change 1326202137 (0ms)
> November 18th 2019, 12:03:16.411 |m13c20| drbd one-vm-5444-disk-0:
> State change 1326202137: primary_nodes=0, weak_nodes=0
> November 18th 2019, 12:03:16.411 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:03:16.446 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m5c5: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
> November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Preparing remote state change 3457834078
> November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Committing remote state change 3457834078 (primary_nodes=4)
> November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0
> m5c5: peer( Secondary -> Primary )
> November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m5c5: received new current UUID: 48346FFFC6B44693
> weak_nodes=FFFFFFFFFFFFFFF8
> November 18th 2019, 12:03:16.690 |m13c20| drbd one-vm-5444-disk-0
> m5c5: peer( Primary -> Secondary )
> November 18th 2019, 12:03:16.698 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Preparing remote state change 422036201
> November 18th 2019, 12:03:16.698 |m13c20| drbd one-vm-5444-disk-0
> m5c5: peer( Secondary -> Primary )
> November 18th 2019, 12:03:16.698 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Committing remote state change 422036201 (primary_nodes=4)
> November 18th 2019, 12:03:21.850 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m5c5: received new current UUID: BE4FF7FC641F0E17
> weak_nodes=FFFFFFFFFFFFFFF8
> November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0
> m15c41: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown )
> November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0
> m15c41: PingAck did not arrive in time.
> November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0
> m15c41: ack_receiver terminated
> November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Terminating ack_recv thread
> November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0/0
> drbd2340 m15c41: pdsk( Diskless -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 12:05:14.366 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Restarting sender thread
> November 18th 2019, 12:05:14.405 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Restarting receiver thread
> November 18th 2019, 12:05:14.405 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Connection closed
> November 18th 2019, 12:05:14.405 |m13c20| drbd one-vm-5661-disk-0
> m15c41: conn( NetworkFailure -> Unconnected )
> November 18th 2019, 12:05:14.406 |m13c20| drbd one-vm-5661-disk-0
> m15c41: conn( Unconnected -> Connecting )
> November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
> conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915/0
> drbd1370 m13c37: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
> Preparing remote state change 3519185948
> November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
> ack_receiver terminated
> November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
> Committing remote state change 3519185948 (primary_nodes=0)
> November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
> Terminating ack_recv thread
> November 18th 2019, 12:05:41.727 |m13c20| drbd one-image-1915 m13c37:
> Restarting sender thread
> November 18th 2019, 12:05:41.734 |m13c20| drbd one-image-1915 m13c37:
> conn( TearDown -> Disconnecting )
> November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37:
> Connection closed
> November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915/0
> drbd1370: disk( UpToDate -> Detaching )
> November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915/0
> drbd1370: drbd_bm_resize called with capacity == 0
> November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915/0
> drbd1370: disk( Detaching -> Diskless )
> November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37:
> Terminating sender thread
> November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37:
> conn( Disconnecting -> StandAlone )
> November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37:
> Terminating receiver thread
> November 18th 2019, 12:05:41.785 |m13c20| drbd one-image-1915:
> Terminating worker thread
> November 18th 2019, 12:06:30.586 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Starting sender thread (from drbdsetup [21166])
> November 18th 2019, 12:06:30.650 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:30.680 |m13c20| drbd one-vm-5444-disk-0
> m15c13: conn( StandAlone -> Unconnected )
> November 18th 2019, 12:06:30.680 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Starting receiver thread (from drbd_w_one-vm-5 [17329])
> November 18th 2019, 12:06:30.680 |m13c20| drbd one-vm-5444-disk-0
> m15c13: conn( Unconnected -> Connecting )
> November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Starting ack_recv thread (from drbd_r_one-vm-5 [21173])
> November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Handshake to peer 3 successful: Agreed network protocol
> version 115
> November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:06:31.398 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Preparing remote state change 2799790767
> November 18th 2019, 12:06:31.398 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Committing remote state change 2799790767 (primary_nodes=4)
> November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0:
> Preparing cluster-wide state change 3271298677 (0->3 499/146)
> November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0:
> State change 3271298677: primary_nodes=4, weak_nodes=FFFFFFFFFFFFFFF8
> November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0:
> Committing cluster-wide state change 3271298677 (0ms)
> November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0
> m15c13: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:06:31.548 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m15c13: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
> November 18th 2019, 12:06:31.834 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:33.414 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Committing remote state change 3067680197 (primary_nodes=C)
> November 18th 2019, 12:06:33.414 |m13c20| drbd one-vm-5444-disk-0
> m15c13: Preparing remote state change 3067680197
> November 18th 2019, 12:06:33.415 |m13c20| drbd one-vm-5444-disk-0
> m15c13: peer( Secondary -> Primary )
> November 18th 2019, 12:06:33.430 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m15c13: received new current UUID: D5AF2A094556BE75
> weak_nodes=FFFFFFFFFFFFFFF4
> November 18th 2019, 12:06:36.861 |m13c20| drbd one-vm-5444-disk-0
> m5c5: peer( Primary -> Secondary )
> November 18th 2019, 12:06:39.454 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0
> m5c5: ack_receiver terminated
> November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Preparing remote state change 3180102972
> November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Committing remote state change 3180102972 (primary_nodes=8)
> November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369 m5c5: pdsk( Diskless -> DUnknown ) repl( Established -> Off )
> November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( Connected -> TearDown ) peer( Secondary -> Unknown )
> November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Terminating ack_recv thread
> November 18th 2019, 12:06:39.878 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Preparing remote state change 1753470888
> November 18th 2019, 12:06:39.878 |m13c20| drbd one-vm-5444-disk-0
> m14c16: Committing remote state change 1753470888 (primary_nodes=8)
> November 18th 2019, 12:06:39.886 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Restarting sender thread
> November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Restarting receiver thread
> November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( TearDown -> Unconnected )
> November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( Unconnected -> Connecting )
> November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Connection closed
> November 18th 2019, 12:06:39.905 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Restarting sender thread
> November 18th 2019, 12:06:39.905 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( Connecting -> Disconnecting )
> November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Connection closed
> November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0
> m5c5: conn( Disconnecting -> StandAlone )
> November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Terminating sender thread
> November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0
> m5c5: Terminating receiver thread
> November 18th 2019, 12:06:39.908 |m13c20| drbd one-vm-5444-disk-0/0
> drbd2369: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:56.878 |m13c20| drbd one-image-1883 m14c14:
> Starting sender thread (from drbdsetup [21856])
> November 18th 2019, 12:06:56.918 |m13c20| drbd one-image-1883/0
> drbd1353: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:56.939 |m13c20| drbd one-image-1883 m14c14:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 12:06:56.942 |m13c20| drbd one-image-1883 m14c14:
> Starting receiver thread (from drbd_w_one-imag [31254])
> November 18th 2019, 12:06:56.942 |m13c20| drbd one-image-1883 m14c14:
> conn( Unconnected -> Connecting )
> November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14:
> Starting ack_recv thread (from drbd_r_one-imag [21863])
> November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883:
> Committing cluster-wide state change 2284081876 (0ms)
> November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883:
> Preparing cluster-wide state change 2284081876 (0->1 499/146)
> November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883: State
> change 2284081876: primary_nodes=0, weak_nodes=0
> November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883 m14c14:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: drbd_sync_handshake:
> November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: Writing the whole bitmap, full sync required after
> drbd_sync_handshake.
> November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: peer
> 3454322884DFC87C:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: self
> FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
> bits:244875 flags:20
> November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: uuid_compare()=3 by rule 80
> November 18th 2019, 12:06:57.638 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
> November 18th 2019, 12:06:57.638 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:06:57.642 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: pdsk( Consistent -> Outdated )
> November 18th 2019, 12:06:57.642 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: helper command: /sbin/drbdadm before-resync-source
> November 18th 2019, 12:06:57.642 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:06:57.643 |m13c20| Don't know which config file
> belongs to resource one-image-1883, trying default ones...
> November 18th 2019, 12:06:57.650 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: helper command: /sbin/drbdadm before-resync-source
> exit code 0 (0x0)
> November 18th 2019, 12:06:57.650 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: Began resync as SyncSource (will sync 2100760 KB
> [525190 bits set]).
> November 18th 2019, 12:06:57.650 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS ->
> SyncSource )
> November 18th 2019, 12:06:58.026 |m13c20| drbd one-image-1901 m14c14:
> Starting sender thread (from drbdsetup [21903])
> November 18th 2019, 12:06:58.061 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:58.092 |m13c20| drbd one-image-1901 m14c14:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 12:06:58.093 |m13c20| drbd one-image-1901 m14c14:
> Starting receiver thread (from drbd_w_one-imag [31502])
> November 18th 2019, 12:06:58.093 |m13c20| drbd one-image-1901 m14c14:
> conn( Unconnected -> Connecting )
> November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14:
> Starting ack_recv thread (from drbd_r_one-imag [21910])
> November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901:
> Preparing cluster-wide state change 2709062739 (0->1 499/146)
> November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901: State
> change 2709062739: primary_nodes=0, weak_nodes=0
> November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901 m14c14:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901:
> Committing cluster-wide state change 2709062739 (0ms)
> November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: drbd_sync_handshake:
> November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: self
> D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
> bits:35653 flags:20
> November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: uuid_compare()=3 by rule 80
> November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: Writing the whole bitmap, full sync required after
> drbd_sync_handshake.
> November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: peer
> 4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:06:59.378 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
> November 18th 2019, 12:06:59.378 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 21(1), total 21; compression: 99.9%
> November 18th 2019, 12:06:59.445 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: pdsk( Consistent -> Outdated )
> November 18th 2019, 12:06:59.447 |m13c20| Don't know which config file
> belongs to resource one-image-1901, trying default ones...
> November 18th 2019, 12:06:59.450 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: helper command: /sbin/drbdadm before-resync-source
> November 18th 2019, 12:06:59.450 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 21(1), total 21; compression: 99.9%
> November 18th 2019, 12:06:59.457 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS ->
> SyncSource )
> November 18th 2019, 12:06:59.457 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: helper command: /sbin/drbdadm before-resync-source
> exit code 0 (0x0)
> November 18th 2019, 12:06:59.457 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: Began resync as SyncSource (will sync 515948 KB
> [128987 bits set]).
> November 18th 2019, 12:06:59.950 |m13c20| drbd one-image-1902 m14c22:
> Starting sender thread (from drbdsetup [21950])
> November 18th 2019, 12:06:59.974 |m13c20| drbd one-image-1902/0
> drbd1356: rs_discard_granularity changed to 524288
> November 18th 2019, 12:06:59.997 |m13c20| drbd one-image-1902 m14c22:
> conn( StandAlone -> Unconnected )
> November 18th 2019, 12:07:00.000 |m13c20| drbd one-image-1902 m14c22:
> Starting receiver thread (from drbd_w_one-imag [31619])
> November 18th 2019, 12:07:00.003 |m13c20| drbd one-image-1902 m14c22:
> conn( Unconnected -> Connecting )
> November 18th 2019, 12:07:00.336 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: Resync done (total 1 sec; paused 0 sec; 515948 K/sec)
> November 18th 2019, 12:07:00.336 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: 72 % had equal checksums, eliminated: 372660K;
> transferred 143288K total 515948K
> November 18th 2019, 12:07:00.336 |m13c20| drbd one-image-1901/0
> drbd1355 m14c14: pdsk( Inconsistent -> UpToDate ) repl( SyncSource ->
> Established )
> November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22:
> Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22:
> Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22:
> Handshake to peer 1 successful: Agreed network protocol version 115
> November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22:
> Starting ack_recv thread (from drbd_r_one-imag [21959])
> November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902: State
> change 306230147: primary_nodes=0, weak_nodes=0
> November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902:
> Committing cluster-wide state change 306230147 (0ms)
> November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902 m14c22:
> conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902:
> Preparing cluster-wide state change 306230147 (0->1 499/146)
> November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: Writing the whole bitmap, full sync required after
> drbd_sync_handshake.
> November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: peer
> E55EC4875988BFDC:0000000000000000:0000000000000000:0000000000000000
> bits:0 flags:20
> November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: self
> B298F86F65888842:0000000000000000:E55EC4875988BFDC:0000000000000000
> bits:224873 flags:20
> November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: uuid_compare()=3 by rule 80
> November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: drbd_sync_handshake:
> November 18th 2019, 12:07:00.774 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: send bitmap stats [Bytes(packets)]: plain 0(0), RLE
> 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:07:00.774 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
> November 18th 2019, 12:07:00.784 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: pdsk( Consistent -> Outdated )
> November 18th 2019, 12:07:00.786 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: helper command: /sbin/drbdadm before-resync-source
> November 18th 2019, 12:07:00.786 |m13c20| Don't know which config file
> belongs to resource one-image-1902, trying default ones...
> November 18th 2019, 12:07:00.786 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: receive bitmap stats [Bytes(packets)]: plain 0(0),
> RLE 21(1), total 21; compression: 100.0%
> November 18th 2019, 12:07:00.792 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: Began resync as SyncSource (will sync 2100760 KB
> [525190 bits set]).
> November 18th 2019, 12:07:00.792 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: helper command: /sbin/drbdadm before-resync-source
> exit code 0 (0x0)
> November 18th 2019, 12:07:00.792 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS ->
> SyncSource )
> November 18th 2019, 12:07:01.118 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
> November 18th 2019, 12:07:01.118 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: 53 % had equal checksums, eliminated: 1120792K;
> transferred 979968K total 2100760K
> November 18th 2019, 12:07:01.118 |m13c20| drbd one-image-1883/0
> drbd1353 m14c14: pdsk( Inconsistent -> UpToDate ) repl( SyncSource ->
> Established )
> November 18th 2019, 12:07:03.894 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
> November 18th 2019, 12:07:03.894 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: 57 % had equal checksums, eliminated: 1201364K;
> transferred 899396K total 2100760K
> November 18th 2019, 12:07:03.894 |m13c20| drbd one-image-1902/0
> drbd1356 m14c22: pdsk( Inconsistent -> UpToDate ) repl( SyncSource ->
> Established )
> November 18th 2019, 12:08:04.738 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Handshake to peer 3 successful: Agreed network protocol
> version 115
> November 18th 2019, 12:08:04.739 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Peer authenticated using 20 bytes HMAC
> November 18th 2019, 12:08:04.739 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC
> WRITE_SAME WRITE_ZEROES.
> November 18th 2019, 12:08:04.739 |m13c20| drbd one-vm-5661-disk-0
> m15c41: Starting ack_recv thread (from drbd_r_one-vm-5 [12782])
> November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0:
> Preparing cluster-wide state change 2061918045 (0->3 499/146)
> November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0:
> Committing cluster-wide state change 2061918045 (0ms)
> November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0:
> State change 2061918045: primary_nodes=0, weak_nodes=0
> November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0
> m15c41: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> November 18th 2019, 12:08:04.798 |m13c20| drbd one-vm-5661-disk-0
> m14c31: Preparing remote state change 4147999259
> November 18th 2019, 12:08:04.798 |m13c20| drbd one-vm-5661-disk-0
> m14c31: Committing remote state change 4147999259 (primary_nodes=0)
> November 18th 2019, 12:08:04.818 |m13c20| drbd one-vm-5661-disk-0/0
> drbd2340 m15c41: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
>
> 5) Also we had these weird mesages after a while:
>
> November 19th 2019, 01:15:35.522 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:15:42.158 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:18:57.694 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:19:04.018 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:26:58.610 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:27:04.486 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:27:14.310 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:27:20.074 |m14c14| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
>
> November 19th 2019, 01:15:35.222 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:15:42.110 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:18:57.354 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:19:04.406 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:26:58.322 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:27:04.458 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:27:13.678 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
> November 19th 2019, 01:27:20.034 |m13c20| drbd one-image-1901/0
> drbd1355: rs_discard_granularity changed to 524288
>
> - kvaps
>
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT
> drbd-user mailing list
> drbd-user at lists.linbit.com
> https://lists.linbit.com/mailman/listinfo/drbd-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20191121/ef5816e3/attachment-0001.htm>
More information about the drbd-user
mailing list