[DRBD-user] Data corrupted after relocation of DRBD-resources

[kvaps]

Hi, today we faced with the weird issue during reloaction some of
drbd-resources. Everywhere steps were the same: each resources had two
replicase, one replica was removed, then another one was created on another
node.

Almost all of them started having data coruption. Hex dump of two similar
drives shows zeroes on place where should be data.

We're using LVM_THIN driver with LINSTOR.

Kernel version: 4.15.18-18-pve

DRBD module version:

version: 9.0.19-1 (api:2/proto:86-115)
GIT-hash: 6f5fa5d348a99e5eeb09d83c49853d72e614fd07 build by @eb2143fc2fef,
2019-07-26 11:40:44
Transports (api:16): tcp (9.0.19-1)

I inspected one of these resources, this resource is not used by anyone,
usually it is used only in readonly mode just for copying data from it.

What was done:

We had a resource on m13c20, m13c14 nodes, then the next actions was done:

1) Resource was removed form m13c14 node

2) Resource was created on m13c15 node

3) Resource was removed from m13c15 node

4) Resource was created on m14c14 node

No errors was reported (kernel logs and executed commands look further)

Currently resource is looking like:

+----------------------------------------------------+
| ResourceName   | Node   | Port | Usage  |    State |
|====================================================|
| one-image-1901 | m13c20 | 7355 | Unused | UpToDate |
| one-image-1901 | m14c14 | 7355 | Unused | UpToDate |
+----------------------------------------------------+

Commands and kernel logs:

1) linstor r d m13c14 one-image-1901

November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901: State change
4153216007: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15: conn(
Connected -> Disconnecting ) peer( Secondary -> Unknown )
November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901/0 drbd1355
m13c15: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15:
Terminating ack_recv thread
November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901: Preparing
cluster-wide state change 4153216007 (0->1 496/16)
November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15:
Cluster is now split
November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901: Committing
cluster-wide state change 4153216007 (0ms)
November 18th 2019, 11:42:57.934 |m13c14| drbd one-image-1901 m13c15:
ack_receiver terminated
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901/0 drbd1355:
disk( UpToDate -> Detaching )
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
Terminating receiver thread
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901/0 drbd1355:
drbd_bm_resize called with capacity == 0
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
Connection closed
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
Restarting sender thread
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901 m13c15:
Terminating sender thread
November 18th 2019, 11:42:57.975 |m13c14| drbd one-image-1901/0 drbd1355:
disk( Detaching -> Diskless )
November 18th 2019, 11:42:58.006 |m13c14| drbd one-image-1901: Terminating
worker thread

November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
Committing remote state change 4153216007 (primary_nodes=0)
November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14: conn(
Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901/0 drbd1355
m13c14: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14: conn(
TearDown -> Disconnecting )
November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
Preparing remote state change 4153216007
November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
Terminating ack_recv thread
November 18th 2019, 11:42:57.935 |m13c15| drbd one-image-1901 m13c14:
ack_receiver terminated
November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14:
Terminating receiver thread
November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14:
Restarting sender thread
November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14:
Connection closed
November 18th 2019, 11:42:57.993 |m13c15| drbd one-image-1901 m13c14: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:42:57.994 |m13c15| drbd one-image-1901 m13c14:
Terminating sender thread
November 18th 2019, 11:42:58.019 |m13c15| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288

November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903: State change
3926620536: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2:
ack_receiver terminated
November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2: conn(
Connected -> Disconnecting ) peer( Secondary -> Unknown )
November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903: Committing
cluster-wide state change 3926620536 (0ms)
November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2:
Cluster is now split
November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903: Preparing
cluster-wide state change 3926620536 (1->0 496/16)
November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903/0 drbd1358
m13c2: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:42:58.889 |m13c20| drbd one-image-1903 m13c2:
Terminating ack_recv thread
November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
Terminating sender thread
November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
Connection closed
November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
Terminating receiver thread
November 18th 2019, 11:42:58.940 |m13c20| drbd one-image-1903 m13c2:
Restarting sender thread
November 18th 2019, 11:42:58.954 |m13c20| drbd one-image-1903/0 drbd1358:
rs_discard_granularity changed to 524288


2) linstor r c one-image-1901 --auto-place 2 --replicas-on-same
opennebula-1=true --replicas-on-different moonshot -s thindata
  (creates resource on m13c15)

November 18th 2019, 11:46:07.159 |m13c15| drbd one-image-1901 m13c20:
Starting sender thread (from drbdsetup [7079])
November 18th 2019, 11:46:07.191 |m13c15| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 18th 2019, 11:46:07.219 |m13c15| drbd one-image-1901 m13c20:
Starting receiver thread (from drbd_w_one-imag [29649])
November 18th 2019, 11:46:07.219 |m13c15| drbd one-image-1901 m13c20: conn(
Unconnected -> Connecting )
November 18th 2019, 11:46:07.219 |m13c15| drbd one-image-1901 m13c20: conn(
StandAlone -> Unconnected )
November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20:
Starting ack_recv thread (from drbd_r_one-imag [7085])
November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 11:46:07.903 |m13c15| drbd one-image-1901 m13c20:
Handshake to peer 0 successful: Agreed network protocol version 115
November 18th 2019, 11:46:07.943 |m13c15| drbd one-image-1901 m13c20: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 11:46:07.943 |m13c15| drbd one-image-1901 m13c20:
Preparing remote state change 98375023
November 18th 2019, 11:46:07.943 |m13c15| drbd one-image-1901 m13c20:
Committing remote state change 98375023 (primary_nodes=0)
November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: drbd_sync_handshake:
November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: self
D7ECA5851EFE55DE:4B6493096F3F93A6:0000000000000000:0000000000000000
bits:64512 flags:20
November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: peer
4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: uuid_compare()=2 by rule 70
November 18th 2019, 11:46:07.967 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 99.9%
November 18th 2019, 11:46:07.975 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: pdsk( Consistent -> Outdated )
November 18th 2019, 11:46:07.975 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 99.9%
November 18th 2019, 11:46:07.975 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: helper command: /sbin/drbdadm before-resync-source
November 18th 2019, 11:46:07.976 |m13c15| Don't know which config file
belongs to resource one-image-1901, trying default ones...
November 18th 2019, 11:46:07.987 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
November 18th 2019, 11:46:07.987 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: Began resync as SyncSource (will sync 258048 KB [64512 bits set]).
November 18th 2019, 11:46:07.987 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource )
November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: updated UUIDs
D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: Resync done (total 1 sec; paused 0 sec; 258048 K/sec)
November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: 44 % had equal checksums, eliminated: 115436K; transferred 142612K
total 258048K
November 18th 2019, 11:46:08.347 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established )

November 18th 2019, 11:46:01.753 |m13c20| drbd one-image-1883 m13c11:
Starting sender thread (from drbdsetup [31258])
November 18th 2019, 11:46:01.753 |m13c20| drbd one-image-1883: Starting
worker thread (from drbdsetup [31253])
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0 drbd1353:
resync bitmap: bits=525190 words=57449 pages=113
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0 drbd1353:
rs_discard_granularity changed to 524288
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0 drbd1353:
drbd_bm_resize called with capacity == 4201520
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0 drbd1353:
size = 2052 MB (2100760 KB)
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0 drbd1353:
disk( Diskless -> Attaching )
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0 drbd1353:
meta-data IO uses: blk-bio
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883/0 drbd1353:
Maximum number of peer devices = 7
November 18th 2019, 11:46:01.794 |m13c20| drbd one-image-1883: Method to
ensure write ordering: flush
November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0 drbd1353:
size = 2052 MB (2100760 KB)
November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0 drbd1353:
recounting of set bits took additional 0ms
November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0 drbd1353:
disk( Attaching -> UpToDate )
November 18th 2019, 11:46:01.801 |m13c20| drbd one-image-1883/0 drbd1353:
attached to current UUID: 3454322884DFC87C
November 18th 2019, 11:46:01.802 |m13c20| drbd one-image-1883 m13c11: conn(
StandAlone -> Unconnected )
November 18th 2019, 11:46:01.803 |m13c20| drbd one-image-1883 m13c11:
Starting receiver thread (from drbd_w_one-imag [31254])
November 18th 2019, 11:46:01.803 |m13c20| drbd one-image-1883 m13c11: conn(
Unconnected -> Connecting )
November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11:
Starting ack_recv thread (from drbd_r_one-imag [31275])
November 18th 2019, 11:46:02.338 |m13c20| drbd one-image-1883 m13c11:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 11:46:02.438 |m13c20| drbd one-image-1883: Preparing
cluster-wide state change 3727902976 (0->1 499/146)
November 18th 2019, 11:46:02.438 |m13c20| drbd one-image-1883: State change
3727902976: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:46:02.439 |m13c20| drbd one-image-1883: Committing
cluster-wide state change 3727902976 (0ms)
November 18th 2019, 11:46:02.439 |m13c20| drbd one-image-1883 m13c11: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 11:46:02.458 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: self
3454322884DFC87C:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 11:46:02.458 |m13c20| drbd one-image-1883/0 drbd1353:
disk( UpToDate -> Outdated )
November 18th 2019, 11:46:02.458 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: drbd_sync_handshake:
November 18th 2019, 11:46:02.459 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: uuid_compare()=-2 by rule 50
November 18th 2019, 11:46:02.459 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 11:46:02.459 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: peer
FBE6531F94A3E69A:3454322884DFC87C:0000000000000000:0000000000000000
bits:262624 flags:20
November 18th 2019, 11:46:02.477 |m13c20| Don't know which config file
belongs to resource one-image-1883, trying default ones...
November 18th 2019, 11:46:02.478 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 11:46:02.478 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:02.478 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: Began resync as SyncTarget (will sync 1050496 KB [262624 bits set]).
November 18th 2019, 11:46:02.491 |m13c20| drbd one-image-1883/0 drbd1353:
disk( Outdated -> Inconsistent )
November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: 6 % had equal checksums, eliminated: 70996K; transferred 979500K
total 1050496K
November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: Resync done (total 1 sec; paused 0 sec; 1050496 K/sec)
November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0 drbd1353:
disk( Inconsistent -> UpToDate )
November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: repl( SyncTarget -> Established )
November 18th 2019, 11:46:04.142 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: updated UUIDs
FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
November 18th 2019, 11:46:04.145 |m13c20| Don't know which config file
belongs to resource one-image-1883, trying default ones...
November 18th 2019, 11:46:04.146 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 11:46:04.150 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:07.330 |m13c20| drbd one-image-1901 m13c15:
Starting sender thread (from drbdsetup [31507])
November 18th 2019, 11:46:07.330 |m13c20| drbd one-image-1901: Starting
worker thread (from drbdsetup [31501])
November 18th 2019, 11:46:07.366 |m13c20| drbd one-image-1901/0 drbd1355:
meta-data IO uses: blk-bio
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
size = 504 MB (515948 KB)
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
recounting of set bits took additional 0ms
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
disk( Attaching -> UpToDate )
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
resync bitmap: bits=128987 words=14112 pages=28
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
drbd_bm_resize called with capacity == 1031896
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
size = 504 MB (515948 KB)
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
attached to current UUID: 4B6493096F3F93A6
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
disk( Diskless -> Attaching )
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901: Method to
ensure write ordering: flush
November 18th 2019, 11:46:07.368 |m13c20| drbd one-image-1901/0 drbd1355:
Maximum number of peer devices = 7
November 18th 2019, 11:46:07.370 |m13c20| drbd one-image-1901 m13c15:
Starting receiver thread (from drbd_w_one-imag [31502])
November 18th 2019, 11:46:07.370 |m13c20| drbd one-image-1901 m13c15: conn(
Unconnected -> Connecting )
November 18th 2019, 11:46:07.370 |m13c20| drbd one-image-1901 m13c15: conn(
StandAlone -> Unconnected )
November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 11:46:07.905 |m13c20| drbd one-image-1901 m13c15:
Starting ack_recv thread (from drbd_r_one-imag [31523])
November 18th 2019, 11:46:07.946 |m13c20| drbd one-image-1901: Preparing
cluster-wide state change 98375023 (0->1 499/146)
November 18th 2019, 11:46:07.946 |m13c20| drbd one-image-1901: State change
98375023: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:46:07.946 |m13c20| drbd one-image-1901: Committing
cluster-wide state change 98375023 (0ms)
November 18th 2019, 11:46:07.947 |m13c20| drbd one-image-1901 m13c15: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 11:46:07.970 |m13c20| drbd one-image-1901/0 drbd1355:
disk( UpToDate -> Outdated )
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: peer
D7ECA5851EFE55DE:4B6493096F3F93A6:0000000000000000:0000000000000000
bits:64512 flags:20
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 99.9%
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: drbd_sync_handshake:
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: self
4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: uuid_compare()=-2 by rule 50
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 99.9%
November 18th 2019, 11:46:07.973 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 11:46:07.974 |m13c20| Don't know which config file
belongs to resource one-image-1901, trying default ones...
November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: Began resync as SyncTarget (will sync 258048 KB [64512 bits set]).
November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0 drbd1355:
disk( Outdated -> Inconsistent )
November 18th 2019, 11:46:07.986 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 11:46:08.340 |m13c20| Don't know which config file
belongs to resource one-image-1901, trying default ones...
November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: Resync done (total 1 sec; paused 0 sec; 258048 K/sec)
November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0 drbd1355:
disk( Inconsistent -> UpToDate )
November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: repl( SyncTarget -> Established )
November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: updated UUIDs
D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
November 18th 2019, 11:46:08.341 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: 44 % had equal checksums, eliminated: 115436K; transferred 142612K
total 258048K
November 18th 2019, 11:46:08.345 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:08.550 |m13c20| drbd one-image-1902: Starting
worker thread (from drbdsetup [31618])
November 18th 2019, 11:46:08.555 |m13c20| drbd one-image-1902 m13c17:
Starting sender thread (from drbdsetup [31623])
November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0 drbd1356:
meta-data IO uses: blk-bio
November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0 drbd1356:
Maximum number of peer devices = 7
November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0 drbd1356:
disk( Diskless -> Attaching )
November 18th 2019, 11:46:08.598 |m13c20| drbd one-image-1902/0 drbd1356:
rs_discard_granularity changed to 524288
November 18th 2019, 11:46:08.599 |m13c20| drbd one-image-1902: Method to
ensure write ordering: flush
November 18th 2019, 11:46:08.599 |m13c20| drbd one-image-1902/0 drbd1356:
drbd_bm_resize called with capacity == 4201520
November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0 drbd1356:
attached to current UUID: E55EC4875988BFDC
November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0 drbd1356:
size = 2052 MB (2100760 KB)
November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0 drbd1356:
recounting of set bits took additional 0ms
November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0 drbd1356:
disk( Attaching -> UpToDate )
November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0 drbd1356:
resync bitmap: bits=525190 words=57449 pages=113
November 18th 2019, 11:46:08.602 |m13c20| drbd one-image-1902/0 drbd1356:
size = 2052 MB (2100760 KB)
November 18th 2019, 11:46:08.605 |m13c20| drbd one-image-1902 m13c17: conn(
StandAlone -> Unconnected )
November 18th 2019, 11:46:08.612 |m13c20| drbd one-image-1902 m13c17:
Starting receiver thread (from drbd_w_one-imag [31619])
November 18th 2019, 11:46:08.612 |m13c20| drbd one-image-1902 m13c17: conn(
Unconnected -> Connecting )
November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17:
Starting ack_recv thread (from drbd_r_one-imag [31645])
November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 11:46:09.119 |m13c20| drbd one-image-1902 m13c17:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 11:46:09.149 |m13c20| drbd one-image-1902: Preparing
cluster-wide state change 2478793358 (0->1 499/146)
November 18th 2019, 11:46:09.150 |m13c20| drbd one-image-1902 m13c17: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 11:46:09.150 |m13c20| drbd one-image-1902: Committing
cluster-wide state change 2478793358 (4ms)
November 18th 2019, 11:46:09.150 |m13c20| drbd one-image-1902: State change
2478793358: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:46:09.186 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: drbd_sync_handshake:
November 18th 2019, 11:46:09.186 |m13c20| drbd one-image-1902/0 drbd1356:
disk( UpToDate -> Outdated )
November 18th 2019, 11:46:09.186 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: self
E55EC4875988BFDC:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: uuid_compare()=-2 by rule 50
November 18th 2019, 11:46:09.187 |m13c20| Don't know which config file
belongs to resource one-image-1902, trying default ones...
November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: peer
B298F86F65888842:E55EC4875988BFDC:0000000000000000:0000000000000000
bits:262624 flags:20
November 18th 2019, 11:46:09.187 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: Began resync as SyncTarget (will sync 1050496 KB [262624 bits set]).
November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 11:46:09.198 |m13c20| drbd one-image-1902/0 drbd1356:
disk( Outdated -> Inconsistent )
November 18th 2019, 11:46:09.762 |m13c20| drbd one-image-1903 m13c41:
Starting sender thread (from drbdsetup [31684])
November 18th 2019, 11:46:09.794 |m13c20| drbd one-image-1903/0 drbd1358:
rs_discard_granularity changed to 524288
November 18th 2019, 11:46:09.820 |m13c20| drbd one-image-1903 m13c41: conn(
StandAlone -> Unconnected )
November 18th 2019, 11:46:09.866 |m13c20| drbd one-image-1903 m13c41: conn(
Unconnected -> Connecting )
November 18th 2019, 11:46:09.866 |m13c20| drbd one-image-1903 m13c41:
Starting receiver thread (from drbd_w_one-imag [3407])
November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41:
Starting ack_recv thread (from drbd_r_one-imag [31696])
November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 11:46:10.386 |m13c20| drbd one-image-1903 m13c41:
Handshake to peer 0 successful: Agreed network protocol version 115
November 18th 2019, 11:46:10.402 |m13c20| drbd one-image-1903 m13c41: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 11:46:10.402 |m13c20| drbd one-image-1903 m13c41:
Committing remote state change 197112278 (primary_nodes=0)
November 18th 2019, 11:46:10.402 |m13c20| drbd one-image-1903 m13c41:
Preparing remote state change 197112278
November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: uuid_compare()=2 by rule 70
November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: drbd_sync_handshake:
November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: peer
399F1FC3E453DEAC:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
November 18th 2019, 11:46:10.431 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: self
B584A09F5803C860:399F1FC3E453DEAC:0000000000000000:0000000000000000
bits:262624 flags:20
November 18th 2019, 11:46:10.492 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: pdsk( Consistent -> Outdated )
November 18th 2019, 11:46:10.498 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: helper command: /sbin/drbdadm before-resync-source
November 18th 2019, 11:46:10.498 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:10.499 |m13c20| Don't know which config file
belongs to resource one-image-1903, trying default ones...
November 18th 2019, 11:46:10.506 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
November 18th 2019, 11:46:10.506 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: Began resync as SyncSource (will sync 1050496 KB [262624 bits set]).
November 18th 2019, 11:46:10.506 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource )
November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: 14 % had equal checksums, eliminated: 151004K; transferred 899492K
total 1050496K
November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: updated UUIDs
B298F86F65888842:0000000000000000:E55EC4875988BFDC:0000000000000000
November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0 drbd1356:
disk( Inconsistent -> UpToDate )
November 18th 2019, 11:46:11.350 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: Resync done (total 2 sec; paused 0 sec; 525248 K/sec)
November 18th 2019, 11:46:11.351 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: repl( SyncTarget -> Established )
November 18th 2019, 11:46:11.366 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 11:46:11.367 |m13c20| Don't know which config file
belongs to resource one-image-1902, trying default ones...
November 18th 2019, 11:46:11.378 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:13.878 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: updated UUIDs
B584A09F5803C860:0000000000000000:399F1FC3E453DEAC:0000000000000000
November 18th 2019, 11:46:13.882 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: 10 % had equal checksums, eliminated: 110592K; transferred 939904K
total 1050496K
November 18th 2019, 11:46:13.882 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established )
November 18th 2019, 11:46:13.882 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: Resync done (total 3 sec; paused 0 sec; 350164 K/sec)
November 18th 2019, 11:46:16.992 |m13c20| drbd one-image-1907: Starting
worker thread (from drbdsetup [31845])
November 18th 2019, 11:46:16.993 |m13c20| drbd one-image-1907 m13c22:
Starting sender thread (from drbdsetup [31850])
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
meta-data IO uses: blk-bio
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
disk( Diskless -> Attaching )
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
resync bitmap: bits=525190 words=57449 pages=113
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
recounting of set bits took additional 0ms
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
rs_discard_granularity changed to 524288
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
Maximum number of peer devices = 7
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
size = 2052 MB (2100760 KB)
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907: Method to
ensure write ordering: flush
November 18th 2019, 11:46:17.018 |m13c20| drbd one-image-1907/0 drbd1359:
drbd_bm_resize called with capacity == 4201520
November 18th 2019, 11:46:17.019 |m13c20| drbd one-image-1907/0 drbd1359:
size = 2052 MB (2100760 KB)
November 18th 2019, 11:46:17.019 |m13c20| drbd one-image-1907/0 drbd1359:
disk( Attaching -> UpToDate )
November 18th 2019, 11:46:17.019 |m13c20| drbd one-image-1907/0 drbd1359:
attached to current UUID: 93AC1857F962278E
November 18th 2019, 11:46:17.020 |m13c20| drbd one-image-1907 m13c22: conn(
StandAlone -> Unconnected )
November 18th 2019, 11:46:17.020 |m13c20| drbd one-image-1907 m13c22: conn(
Unconnected -> Connecting )
November 18th 2019, 11:46:17.020 |m13c20| drbd one-image-1907 m13c22:
Starting receiver thread (from drbd_w_one-imag [31846])
November 18th 2019, 11:46:17.538 |m13c20| drbd one-image-1907 m13c22:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 11:46:17.538 |m13c20| drbd one-image-1907 m13c22:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 11:46:17.539 |m13c20| drbd one-image-1907 m13c22: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 11:46:17.539 |m13c20| drbd one-image-1907 m13c22:
Starting ack_recv thread (from drbd_r_one-imag [31869])
November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907 m13c22: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907: State change
1214514917: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907: Committing
cluster-wide state change 1214514917 (0ms)
November 18th 2019, 11:46:17.578 |m13c20| drbd one-image-1907: Preparing
cluster-wide state change 1214514917 (0->1 499/146)
November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0 drbd1359:
disk( UpToDate -> Outdated )
November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: self
93AC1857F962278E:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: uuid_compare()=-2 by rule 50
November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: drbd_sync_handshake:
November 18th 2019, 11:46:17.600 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: peer
92983D105D747C54:93AC1857F962278E:0000000000000000:0000000000000000
bits:262624 flags:20
November 18th 2019, 11:46:17.604 |m13c20| Don't know which config file
belongs to resource one-image-1907, trying default ones...
November 18th 2019, 11:46:17.608 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 11:46:17.608 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:17.608 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 25(1), total
25; compression: 100.0%
November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0 drbd1359:
disk( Outdated -> Inconsistent )
November 18th 2019, 11:46:17.614 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: Began resync as SyncTarget (will sync 1050496 KB [262624 bits set]).
November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: 10 % had equal checksums, eliminated: 110784K; transferred 939712K
total 1050496K
November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: updated UUIDs
92983D105D747C54:0000000000000000:93AC1857F962278E:0000000000000000
November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: repl( SyncTarget -> Established )
November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: Resync done (total 1 sec; paused 0 sec; 1050496 K/sec)
November 18th 2019, 11:46:19.350 |m13c20| drbd one-image-1907/0 drbd1359:
disk( Inconsistent -> UpToDate )
November 18th 2019, 11:46:19.357 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 11:46:19.357 |m13c20| Don't know which config file
belongs to resource one-image-1907, trying default ones...
November 18th 2019, 11:46:19.362 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:23.686 |m13c20| drbd one-image-1915: Starting
worker thread (from drbdsetup [32126])
November 18th 2019, 11:46:23.688 |m13c20| drbd one-image-1915 m13c37:
Starting sender thread (from drbdsetup [32131])
November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0 drbd1370:
meta-data IO uses: blk-bio
November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0 drbd1370:
Maximum number of peer devices = 7
November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0 drbd1370:
disk( Diskless -> Attaching )
November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0 drbd1370:
drbd_bm_resize called with capacity == 41950384
November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915/0 drbd1370:
rs_discard_granularity changed to 524288
November 18th 2019, 11:46:23.722 |m13c20| drbd one-image-1915: Method to
ensure write ordering: flush
November 18th 2019, 11:46:23.725 |m13c20| drbd one-image-1915/0 drbd1370:
resync bitmap: bits=5243798 words=573545 pages=1121
November 18th 2019, 11:46:23.725 |m13c20| drbd one-image-1915/0 drbd1370:
size = 20 GB (20975192 KB)
November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0 drbd1370:
disk( Attaching -> UpToDate )
November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0 drbd1370:
attached to current UUID: 9217094A747642A4
November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0 drbd1370:
recounting of set bits took additional 4ms
November 18th 2019, 11:46:23.749 |m13c20| drbd one-image-1915/0 drbd1370:
size = 20 GB (20975192 KB)
November 18th 2019, 11:46:23.750 |m13c20| drbd one-image-1915 m13c37: conn(
StandAlone -> Unconnected )
November 18th 2019, 11:46:23.754 |m13c20| drbd one-image-1915 m13c37:
Starting receiver thread (from drbd_w_one-imag [32127])
November 18th 2019, 11:46:23.754 |m13c20| drbd one-image-1915 m13c37: conn(
Unconnected -> Connecting )
November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37:
Starting ack_recv thread (from drbd_r_one-imag [32148])
November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 11:46:24.281 |m13c20| drbd one-image-1915 m13c37: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915: Committing
cluster-wide state change 3354442230 (0ms)
November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915: Preparing
cluster-wide state change 3354442230 (0->1 499/146)
November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915 m13c37: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 11:46:24.326 |m13c20| drbd one-image-1915: State change
3354442230: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:46:24.352 |m13c20| drbd one-image-1915/0 drbd1370:
disk( UpToDate -> Outdated )
November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: self
9217094A747642A4:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: peer
F80052D0186111B8:9217094A747642A4:0000000000000000:0000000000000000
bits:2621920 flags:20
November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: drbd_sync_handshake:
November 18th 2019, 11:46:24.353 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: uuid_compare()=-2 by rule 50
November 18th 2019, 11:46:24.365 |m13c20| Don't know which config file
belongs to resource one-image-1915, trying default ones...
November 18th 2019, 11:46:24.366 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 28(1), total
28; compression: 100.0%
November 18th 2019, 11:46:24.366 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 11:46:24.366 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 28(1), total
28; compression: 100.0%
November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: Began resync as SyncTarget (will sync 10487680 KB [2621920 bits
set]).
November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0 drbd1370:
disk( Outdated -> Inconsistent )
November 18th 2019, 11:46:24.374 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 11:46:41.420 |m13c20| Don't know which config file
belongs to resource one-image-1915, trying default ones...
November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0 drbd1370:
disk( Inconsistent -> UpToDate )
November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: 75 % had equal checksums, eliminated: 7930080K; transferred
2557600K total 10487680K
November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: Resync done (total 17 sec; paused 0 sec; 616920 K/sec)
November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: repl( SyncTarget -> Established )
November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: updated UUIDs
F80052D0186111B8:0000000000000000:9217094A747642A4:0000000000000000
November 18th 2019, 11:46:41.422 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 11:46:41.426 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)


3) linstor r d m13c15 one-image-1901


November 18th 2019, 11:48:27.268 |m13c15| drbd one-image-1901 m13c20:
Preparing remote state change 364736091
November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20:
Committing remote state change 364736091 (primary_nodes=0)
November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20: conn(
Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901/0 drbd1355
m13c20: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20:
ack_receiver terminated
November 18th 2019, 11:48:27.269 |m13c15| drbd one-image-1901 m13c20:
Terminating ack_recv thread
November 18th 2019, 11:48:27.271 |m13c15| drbd one-image-1901 m13c20: conn(
TearDown -> Disconnecting )
November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20:
Connection closed
November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20:
Terminating receiver thread
November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:48:27.321 |m13c15| drbd one-image-1901 m13c20:
Restarting sender thread
November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901 m13c20:
Terminating sender thread
November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901/0 drbd1355:
drbd_bm_resize called with capacity == 0
November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901/0 drbd1355:
disk( Detaching -> Diskless )
November 18th 2019, 11:48:27.322 |m13c15| drbd one-image-1901/0 drbd1355:
disk( UpToDate -> Detaching )
November 18th 2019, 11:48:27.351 |m13c15| drbd one-image-1901: Terminating
worker thread

November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11:
Preparing remote state change 1038193810
November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11: conn(
Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883/0 drbd1353
m13c11: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11:
ack_receiver terminated
November 18th 2019, 11:48:26.762 |m13c20| drbd one-image-1883 m13c11:
Committing remote state change 1038193810 (primary_nodes=0)
November 18th 2019, 11:48:26.763 |m13c20| drbd one-image-1883 m13c11:
Terminating ack_recv thread
November 18th 2019, 11:48:26.782 |m13c20| drbd one-image-1883 m13c11: conn(
TearDown -> Disconnecting )
November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
Connection closed
November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
Restarting sender thread
November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
Terminating receiver thread
November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11:
Terminating sender thread
November 18th 2019, 11:48:26.826 |m13c20| drbd one-image-1883 m13c11: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:48:26.838 |m13c20| drbd one-image-1883/0 drbd1353:
rs_discard_granularity changed to 524288
November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901: Committing
cluster-wide state change 364736091 (0ms)
November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901 m13c15: conn(
Connected -> Disconnecting ) peer( Secondary -> Unknown )
November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901/0 drbd1355
m13c15: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901: Preparing
cluster-wide state change 364736091 (0->1 496/16)
November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901 m13c15:
Cluster is now split
November 18th 2019, 11:48:27.270 |m13c20| drbd one-image-1901: State change
364736091: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:48:27.271 |m13c20| drbd one-image-1901 m13c15:
ack_receiver terminated
November 18th 2019, 11:48:27.271 |m13c20| drbd one-image-1901 m13c15:
Terminating ack_recv thread
November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
Terminating receiver thread
November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
Terminating sender thread
November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
Connection closed
November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15:
Restarting sender thread
November 18th 2019, 11:48:27.334 |m13c20| drbd one-image-1901 m13c15: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:48:27.346 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
ack_receiver terminated
November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902/0 drbd1356
m13c17: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
Terminating ack_recv thread
November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
Committing remote state change 4188838677 (primary_nodes=0)
November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17:
Preparing remote state change 4188838677
November 18th 2019, 11:48:27.774 |m13c20| drbd one-image-1902 m13c17: conn(
Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 11:48:27.782 |m13c20| drbd one-image-1902 m13c17: conn(
TearDown -> Disconnecting )
November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
Connection closed
November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
Terminating sender thread
November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
Terminating receiver thread
November 18th 2019, 11:48:27.814 |m13c20| drbd one-image-1902 m13c17:
Restarting sender thread
November 18th 2019, 11:48:27.830 |m13c20| drbd one-image-1902/0 drbd1356:
rs_discard_granularity changed to 524288
November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41: conn(
Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
Terminating ack_recv thread
November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
Preparing remote state change 1248155320
November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
Committing remote state change 1248155320 (primary_nodes=0)
November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903 m13c41:
ack_receiver terminated
November 18th 2019, 11:48:28.294 |m13c20| drbd one-image-1903/0 drbd1358
m13c41: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:48:28.302 |m13c20| drbd one-image-1903 m13c41: conn(
TearDown -> Disconnecting )
November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
Terminating sender thread
November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
Restarting sender thread
November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
Connection closed
November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903/0 drbd1358:
disk( UpToDate -> Detaching )
November 18th 2019, 11:48:28.351 |m13c20| drbd one-image-1903 m13c41:
Terminating receiver thread
November 18th 2019, 11:48:28.352 |m13c20| drbd one-image-1903/0 drbd1358:
disk( Detaching -> Diskless )
November 18th 2019, 11:48:28.352 |m13c20| drbd one-image-1903/0 drbd1358:
drbd_bm_resize called with capacity == 0
November 18th 2019, 11:48:28.382 |m13c20| drbd one-image-1903: Terminating
worker thread
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22:
Terminating ack_recv thread
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907: Committing
cluster-wide state change 3182775954 (0ms)
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907: Preparing
cluster-wide state change 3182775954 (0->1 496/16)
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907: State change
3182775954: primary_nodes=0, weak_nodes=0
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22:
Cluster is now split
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22:
ack_receiver terminated
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907 m13c22: conn(
Connected -> Disconnecting ) peer( Secondary -> Unknown )
November 18th 2019, 11:48:30.349 |m13c20| drbd one-image-1907/0 drbd1359
m13c22: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
Restarting sender thread
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907/0 drbd1359:
disk( UpToDate -> Detaching )
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
Connection closed
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907/0 drbd1359:
drbd_bm_resize called with capacity == 0
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907/0 drbd1359:
disk( Detaching -> Diskless )
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
Terminating receiver thread
November 18th 2019, 11:48:30.407 |m13c20| drbd one-image-1907 m13c22:
Terminating sender thread
November 18th 2019, 11:48:30.438 |m13c20| drbd one-image-1907: Terminating
worker thread
November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
ack_receiver terminated
November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
Preparing remote state change 2963804021
November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
Terminating ack_recv thread
November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37: conn(
Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 11:48:33.186 |m13c20| drbd one-image-1915 m13c37:
Committing remote state change 2963804021 (primary_nodes=0)
November 18th 2019, 11:48:33.198 |m13c20| drbd one-image-1915 m13c37: conn(
TearDown -> Disconnecting )
November 18th 2019, 11:48:33.243 |m13c20| drbd one-image-1915 m13c37:
Connection closed
November 18th 2019, 11:48:33.243 |m13c20| drbd one-image-1915 m13c37:
Restarting sender thread
November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915 m13c37: conn(
Disconnecting -> StandAlone )
November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915 m13c37:
Terminating sender thread
November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915 m13c37:
Terminating receiver thread
November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915/0 drbd1370:
disk( Detaching -> Diskless )
November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915/0 drbd1370:
drbd_bm_resize called with capacity == 0
November 18th 2019, 11:48:33.244 |m13c20| drbd one-image-1915/0 drbd1370:
disk( UpToDate -> Detaching )
November 18th 2019, 11:48:33.282 |m13c20| drbd one-image-1915: Terminating
worker thread


4) linstor r c one-image-1901 --auto-place 2 --replicas-on-same
opennebula-1=true moonshot=14 -s thindata
   (creates resource on m14c14)


November 18th 2019, 12:06:57.002 |m14c14| drbd one-image-1883: Starting
worker thread (from drbdsetup [2120])
November 18th 2019, 12:06:57.006 |m14c14| drbd one-image-1883 m13c20:
Starting sender thread (from drbdsetup [2125])
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0 drbd1353:
meta-data IO uses: blk-bio
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0 drbd1353:
resync bitmap: bits=525190 words=57449 pages=113
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883: Method to
ensure write ordering: flush
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0 drbd1353:
size = 2052 MB (2100760 KB)
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0 drbd1353:
disk( Diskless -> Attaching )
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0 drbd1353:
drbd_bm_resize called with capacity == 4201520
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0 drbd1353:
rs_discard_granularity changed to 524288
November 18th 2019, 12:06:57.030 |m14c14| drbd one-image-1883/0 drbd1353:
Maximum number of peer devices = 7
November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0 drbd1353:
attached to current UUID: 3454322884DFC87C
November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0 drbd1353:
recounting of set bits took additional 0ms
November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0 drbd1353:
size = 2052 MB (2100760 KB)
November 18th 2019, 12:06:57.034 |m14c14| drbd one-image-1883/0 drbd1353:
disk( Attaching -> UpToDate )
November 18th 2019, 12:06:57.038 |m14c14| drbd one-image-1883 m13c20: conn(
StandAlone -> Unconnected )
November 18th 2019, 12:06:57.038 |m14c14| drbd one-image-1883 m13c20: conn(
Unconnected -> Connecting )
November 18th 2019, 12:06:57.038 |m14c14| drbd one-image-1883 m13c20:
Starting receiver thread (from drbd_w_one-imag [2121])
November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20:
Starting ack_recv thread (from drbd_r_one-imag [2140])
November 18th 2019, 12:06:57.570 |m14c14| drbd one-image-1883 m13c20:
Handshake to peer 0 successful: Agreed network protocol version 115
November 18th 2019, 12:06:57.618 |m14c14| drbd one-image-1883 m13c20: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:06:57.618 |m14c14| drbd one-image-1883 m13c20:
Committing remote state change 2284081876 (primary_nodes=0)
November 18th 2019, 12:06:57.618 |m14c14| drbd one-image-1883 m13c20:
Preparing remote state change 2284081876
November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: peer
FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
bits:244875 flags:20
November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: uuid_compare()=-3 by rule 60
November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: drbd_sync_handshake:
November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: self
3454322884DFC87C:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0 drbd1353:
disk( UpToDate -> Outdated )
November 18th 2019, 12:06:57.634 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: Writing the whole bitmap, full sync required after
drbd_sync_handshake.
November 18th 2019, 12:06:57.642 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 12:06:57.642 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:06:57.645 |m14c14| Don't know which config file
belongs to resource one-image-1883, trying default ones...
November 18th 2019, 12:06:57.646 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 12:06:57.646 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0 drbd1353:
disk( Outdated -> Inconsistent )
November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 12:06:57.662 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: Began resync as SyncTarget (will sync 2100760 KB [525190 bits set]).
November 18th 2019, 12:06:58.554 |m14c14| drbd one-image-1901: Starting
worker thread (from drbdsetup [2319])
November 18th 2019, 12:06:58.558 |m14c14| drbd one-image-1901 m13c20:
Starting sender thread (from drbdsetup [2324])
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
Maximum number of peer devices = 7
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
resync bitmap: bits=128987 words=14112 pages=28
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901: Method to
ensure write ordering: flush
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
disk( Diskless -> Attaching )
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
drbd_bm_resize called with capacity == 1031896
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
size = 504 MB (515948 KB)
November 18th 2019, 12:06:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
meta-data IO uses: blk-bio
November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0 drbd1355:
disk( Attaching -> UpToDate )
November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0 drbd1355:
recounting of set bits took additional 0ms
November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0 drbd1355:
attached to current UUID: 4B6493096F3F93A6
November 18th 2019, 12:06:58.614 |m14c14| drbd one-image-1901/0 drbd1355:
size = 504 MB (515948 KB)
November 18th 2019, 12:06:58.638 |m14c14| drbd one-image-1901 m13c20: conn(
StandAlone -> Unconnected )
November 18th 2019, 12:06:58.709 |m14c14| drbd one-image-1901 m13c20:
Starting receiver thread (from drbd_w_one-imag [2320])
November 18th 2019, 12:06:58.709 |m14c14| drbd one-image-1901 m13c20: conn(
Unconnected -> Connecting )
November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20:
Starting ack_recv thread (from drbd_r_one-imag [2350])
November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20:
Handshake to peer 0 successful: Agreed network protocol version 115
November 18th 2019, 12:06:59.235 |m14c14| drbd one-image-1901 m13c20:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:06:59.334 |m14c14| drbd one-image-1901 m13c20:
Preparing remote state change 2709062739
November 18th 2019, 12:06:59.334 |m14c14| drbd one-image-1901 m13c20:
Committing remote state change 2709062739 (primary_nodes=0)
November 18th 2019, 12:06:59.334 |m14c14| drbd one-image-1901 m13c20: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:06:59.358 |m14c14| drbd one-image-1901/0 drbd1355:
disk( UpToDate -> Outdated )
November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: peer
D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
bits:35653 flags:20
November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: uuid_compare()=-3 by rule 60
November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: Writing the whole bitmap, full sync required after
drbd_sync_handshake.
November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: drbd_sync_handshake:
November 18th 2019, 12:06:59.442 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: self
4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:06:59.446 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 12:06:59.448 |m14c14| Don't know which config file
belongs to resource one-image-1901, trying default ones...
November 18th 2019, 12:06:59.450 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 99.9%
November 18th 2019, 12:06:59.450 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 99.9%
November 18th 2019, 12:06:59.450 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0 drbd1355:
disk( Outdated -> Inconsistent )
November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 12:06:59.454 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: Began resync as SyncTarget (will sync 515948 KB [128987 bits set]).
November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: Resync done (total 1 sec; paused 0 sec; 515948 K/sec)
November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: updated UUIDs
D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0 drbd1355:
disk( Inconsistent -> UpToDate )
November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: repl( SyncTarget -> Established )
November 18th 2019, 12:07:00.330 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: 72 % had equal checksums, eliminated: 372660K; transferred 143288K
total 515948K
November 18th 2019, 12:07:00.331 |m14c14| Don't know which config file
belongs to resource one-image-1901, trying default ones...
November 18th 2019, 12:07:00.334 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 12:07:00.338 |m14c14| drbd one-image-1901/0 drbd1355
m13c20: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: updated UUIDs
FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: repl( SyncTarget -> Established )
November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: 53 % had equal checksums, eliminated: 1120792K; transferred 979968K
total 2100760K
November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0 drbd1353:
disk( Inconsistent -> UpToDate )
November 18th 2019, 12:07:01.110 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
November 18th 2019, 12:07:01.113 |m14c14| Don't know which config file
belongs to resource one-image-1883, trying default ones...
November 18th 2019, 12:07:01.114 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 12:07:01.116 |m14c14| drbd one-image-1883/0 drbd1353
m13c20: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 12:07:04.574 |m14c14| drbd one-image-1905: Starting
worker thread (from drbdsetup [2722])
November 18th 2019, 12:07:04.575 |m14c14| drbd one-image-1905 m13c44:
Starting sender thread (from drbdsetup [2728])
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0 drbd1360:
rs_discard_granularity changed to 524288
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0 drbd1360:
meta-data IO uses: blk-bio
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905: Method to
ensure write ordering: flush
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0 drbd1360:
size = 2052 MB (2100760 KB)
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0 drbd1360:
Maximum number of peer devices = 7
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0 drbd1360:
resync bitmap: bits=525190 words=57449 pages=113
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0 drbd1360:
disk( Diskless -> Attaching )
November 18th 2019, 12:07:04.607 |m14c14| drbd one-image-1905/0 drbd1360:
drbd_bm_resize called with capacity == 4201520
November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905 m13c44: conn(
Unconnected -> Connecting )
November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0 drbd1360:
disk( Attaching -> UpToDate )
November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0 drbd1360:
attached to current UUID: 0832983AA7F2AF02
November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0 drbd1360:
recounting of set bits took additional 0ms
November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905/0 drbd1360:
size = 2052 MB (2100760 KB)
November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905 m13c44: conn(
StandAlone -> Unconnected )
November 18th 2019, 12:07:04.614 |m14c14| drbd one-image-1905 m13c44:
Starting receiver thread (from drbd_w_one-imag [2723])
November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44:
Handshake to peer 0 successful: Agreed network protocol version 115
November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 12:07:05.146 |m14c14| drbd one-image-1905 m13c44:
Starting ack_recv thread (from drbd_r_one-imag [2743])
November 18th 2019, 12:07:05.170 |m14c14| drbd one-image-1905 m13c44:
Preparing remote state change 3787063659
November 18th 2019, 12:07:05.170 |m14c14| drbd one-image-1905 m13c44: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:07:05.170 |m14c14| drbd one-image-1905 m13c44:
Committing remote state change 3787063659 (primary_nodes=0)
November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0 drbd1360:
disk( UpToDate -> Outdated )
November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: drbd_sync_handshake:
November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: uuid_compare()=-3 by rule 60
November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: Writing the whole bitmap, full sync required after
drbd_sync_handshake.
November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: peer
C8AAB1C5FAA5A5B0:0000000000000000:0832983AA7F2AF02:0000000000000000
bits:236512 flags:20
November 18th 2019, 12:07:05.196 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: self
0832983AA7F2AF02:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:07:05.206 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:07:05.206 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 12:07:05.207 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 12:07:05.207 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:07:05.209 |m14c14| Don't know which config file
belongs to resource one-image-1905, trying default ones...
November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: Began resync as SyncTarget (will sync 2100760 KB [525190 bits set]).
November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0 drbd1360:
disk( Outdated -> Inconsistent )
November 18th 2019, 12:07:05.226 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 12:07:06.082 |m14c14| drbd one-image-1906: Starting
worker thread (from drbdsetup [3020])
November 18th 2019, 12:07:06.086 |m14c14| drbd one-image-1906 m13c44:
Starting sender thread (from drbdsetup [3026])
November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0 drbd1362:
rs_discard_granularity changed to 524288
November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0 drbd1362:
disk( Diskless -> Attaching )
November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0 drbd1362:
meta-data IO uses: blk-bio
November 18th 2019, 12:07:06.174 |m14c14| drbd one-image-1906/0 drbd1362:
Maximum number of peer devices = 7
November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906/0 drbd1362:
drbd_bm_resize called with capacity == 4201520
November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906/0 drbd1362:
resync bitmap: bits=525190 words=57449 pages=113
November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906/0 drbd1362:
size = 2052 MB (2100760 KB)
November 18th 2019, 12:07:06.178 |m14c14| drbd one-image-1906: Method to
ensure write ordering: flush
November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0 drbd1362:
disk( Attaching -> UpToDate )
November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0 drbd1362:
attached to current UUID: 4EBBBE865C77D32C
November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0 drbd1362:
size = 2052 MB (2100760 KB)
November 18th 2019, 12:07:06.182 |m14c14| drbd one-image-1906/0 drbd1362:
recounting of set bits took additional 0ms
November 18th 2019, 12:07:06.186 |m14c14| drbd one-image-1906 m13c44: conn(
StandAlone -> Unconnected )
November 18th 2019, 12:07:06.194 |m14c14| drbd one-image-1906 m13c44:
Starting receiver thread (from drbd_w_one-imag [3021])
November 18th 2019, 12:07:06.194 |m14c14| drbd one-image-1906 m13c44: conn(
Unconnected -> Connecting )
November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44:
Handshake to peer 0 successful: Agreed network protocol version 115
November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44:
Starting ack_recv thread (from drbd_r_one-imag [3043])
November 18th 2019, 12:07:06.716 |m14c14| drbd one-image-1906 m13c44:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:07:06.758 |m14c14| drbd one-image-1906 m13c44:
Committing remote state change 2466514477 (primary_nodes=0)
November 18th 2019, 12:07:06.758 |m14c14| drbd one-image-1906 m13c44:
Preparing remote state change 2466514477
November 18th 2019, 12:07:06.758 |m14c14| drbd one-image-1906 m13c44: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:07:06.782 |m14c14| drbd one-image-1906/0 drbd1362:
disk( UpToDate -> Outdated )
November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: Writing the whole bitmap, full sync required after
drbd_sync_handshake.
November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: self
4EBBBE865C77D32C:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: drbd_sync_handshake:
November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: peer
86A2611E5BFC6FB2:0000000000000000:4EBBBE865C77D32C:0000000000000000
bits:235872 flags:20
November 18th 2019, 12:07:06.826 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: uuid_compare()=-3 by rule 60
November 18th 2019, 12:07:06.850 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
November 18th 2019, 12:07:06.913 |m14c14| Don't know which config file
belongs to resource one-image-1906, trying default ones...
November 18th 2019, 12:07:06.914 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: helper command: /sbin/drbdadm before-resync-target
November 18th 2019, 12:07:06.914 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:07:06.914 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: repl( WFBitMapT -> SyncTarget )
November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0 drbd1362:
disk( Outdated -> Inconsistent )
November 18th 2019, 12:07:06.918 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: Began resync as SyncTarget (will sync 2100760 KB [525190 bits set]).
November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: 55 % had equal checksums, eliminated: 1155240K; transferred 945520K
total 2100760K
November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: repl( SyncTarget -> Established )
November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: updated UUIDs
C8AAB1C5FAA5A5B0:0000000000000000:0832983AA7F2AF02:0000000000000000
November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0 drbd1360:
disk( Inconsistent -> UpToDate )
November 18th 2019, 12:07:09.026 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
November 18th 2019, 12:07:09.194 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 12:07:09.195 |m14c14| Don't know which config file
belongs to resource one-image-1905, trying default ones...
November 18th 2019, 12:07:09.214 |m14c14| drbd one-image-1905/0 drbd1360
m13c44: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 12:07:10.640 |m14c14| Don't know which config file
belongs to resource one-image-1906, trying default ones...
November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: 55 % had equal checksums, eliminated: 1157144K; transferred 943616K
total 2100760K
November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: updated UUIDs
86A2611E5BFC6FB2:0000000000000000:4EBBBE865C77D32C:0000000000000000
November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: helper command: /sbin/drbdadm after-resync-target
November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: repl( SyncTarget -> Established )
November 18th 2019, 12:07:10.642 |m14c14| drbd one-image-1906/0 drbd1362:
disk( Inconsistent -> UpToDate )
November 18th 2019, 12:07:10.646 |m14c14| drbd one-image-1906/0 drbd1362
m13c44: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
November 18th 2019, 12:09:25.542 |m14c14| drbd one-vm-5440-disk-0: Starting
worker thread (from drbdsetup [5177])
November 18th 2019, 12:09:25.550 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Starting sender thread (from drbdsetup [5182])
November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: drbd_bm_resize called with capacity == 125834928
November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0: Method
to ensure write ordering: flush
November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: meta-data IO uses: blk-bio
November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: disk( Diskless -> Attaching )
November 18th 2019, 12:09:25.602 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: Maximum number of peer devices = 7
November 18th 2019, 12:09:25.622 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: size = 60 GB (62917464 KB)
November 18th 2019, 12:09:25.622 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: resync bitmap: bits=15729366 words=1720404 pages=3361
November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: recounting of set bits took additional 12ms
November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: size = 60 GB (62917464 KB)
November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: attached to current UUID: 0C79F47BDA4C7504
November 18th 2019, 12:09:25.663 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: disk( Attaching -> UpToDate )
November 18th 2019, 12:09:25.666 |m14c14| drbd one-vm-5440-disk-0 m13c37:
conn( StandAlone -> Unconnected )
November 18th 2019, 12:09:25.666 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Starting receiver thread (from drbd_w_one-vm-5 [5178])
November 18th 2019, 12:09:25.666 |m14c14| drbd one-vm-5440-disk-0 m13c37:
conn( Unconnected -> Connecting )
November 18th 2019, 12:09:25.842 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Starting ack_recv thread (from drbd_r_one-vm-5 [5210])
November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Peer authenticated using 20 bytes HMAC
November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Handshake to peer 0 successful: Agreed network protocol version 115
November 18th 2019, 12:09:26.202 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:09:26.270 |m14c14| drbd one-vm-5440-disk-0 m13c37:
conn( Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:09:26.270 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Committing remote state change 3586394475 (primary_nodes=0)
November 18th 2019, 12:09:26.270 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Preparing remote state change 3586394475
November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m13c37: self
0C79F47BDA4C7504:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m13c37: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m13c37: uuid_compare()=0 by rule 40
November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m13c37: drbd_sync_handshake:
November 18th 2019, 12:09:26.294 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m13c37: peer
0C79F47BDA4C7504:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:09:26.562 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:09:26.746 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:09:26.962 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:09:27.458 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Starting sender thread (from drbdsetup [5391])
November 18th 2019, 12:09:27.482 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:09:27.534 |m14c14| drbd one-vm-5440-disk-0 m5c14:
conn( StandAlone -> Unconnected )
November 18th 2019, 12:09:27.534 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Starting receiver thread (from drbd_w_one-vm-5 [5178])
November 18th 2019, 12:09:27.534 |m14c14| drbd one-vm-5440-disk-0 m5c14:
conn( Unconnected -> Connecting )
November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Starting ack_recv thread (from drbd_r_one-vm-5 [5397])
November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Peer authenticated using 20 bytes HMAC
November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Handshake to peer 2 successful: Agreed network protocol version 114
November 18th 2019, 12:09:28.058 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:09:28.138 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Preparing remote state change 576618137
November 18th 2019, 12:09:28.138 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Committing remote state change 576618137 (primary_nodes=0)
November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0:
Preparing cluster-wide state change 449952009 (1->2 499/146)
November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0: State
change 449952009: primary_nodes=0, weak_nodes=0
November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0 m5c14:
conn( Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:09:28.258 |m14c14| drbd one-vm-5440-disk-0:
Committing cluster-wide state change 449952009 (0ms)
November 18th 2019, 12:09:28.274 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m5c14: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0 m5c14:
peer( Secondary -> Primary )
November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Preparing remote state change 2365784547
November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m5c14: received new current UUID: E8E1C74482F2EB05
weak_nodes=FFFFFFFFFFFFFFF8
November 18th 2019, 12:09:28.510 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Committing remote state change 2365784547 (primary_nodes=4)
November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0 m5c14:
peer( Primary -> Secondary )
November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Preparing remote state change 2409146247
November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0 m5c14:
peer( Secondary -> Primary )
November 18th 2019, 12:09:28.522 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Committing remote state change 2409146247 (primary_nodes=4)
November 18th 2019, 12:09:33.658 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m5c14: received new current UUID: 6C91FAC1689A1645
weak_nodes=FFFFFFFFFFFFFFF8
November 18th 2019, 12:11:05.006 |m14c14| WARNING: Thin pool
data-thindata-tpool data is now 80.07% full.
November 18th 2019, 12:21:45.015 |m14c14| WARNING: Thin pool
data-thindata-tpool data is now 85.05% full.
November 18th 2019, 12:23:54.646 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Starting sender thread (from drbdsetup [19016])
November 18th 2019, 12:23:54.694 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:23:54.746 |m14c14| drbd one-vm-5440-disk-0 m10c3:
conn( StandAlone -> Unconnected )
November 18th 2019, 12:23:54.746 |m14c14| drbd one-vm-5440-disk-0 m10c3:
conn( Unconnected -> Connecting )
November 18th 2019, 12:23:54.746 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Starting receiver thread (from drbd_w_one-vm-5 [5178])
November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Starting ack_recv thread (from drbd_r_one-vm-5 [19023])
November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Handshake to peer 3 successful: Agreed network protocol version 115
November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:23:55.258 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Peer authenticated using 20 bytes HMAC
November 18th 2019, 12:23:55.330 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Committing remote state change 2626253254 (primary_nodes=4)
November 18th 2019, 12:23:55.330 |m14c14| drbd one-vm-5440-disk-0 m13c37:
Preparing remote state change 2626253254
November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0: State
change 3131838755: primary_nodes=4, weak_nodes=FFFFFFFFFFFFFFF8
November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0:
Preparing cluster-wide state change 3131838755 (1->3 499/146)
November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0:
Committing cluster-wide state change 3131838755 (0ms)
November 18th 2019, 12:23:55.370 |m14c14| drbd one-vm-5440-disk-0 m10c3:
conn( Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:23:55.398 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m10c3: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
November 18th 2019, 12:23:55.698 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:23:57.038 |m14c14| drbd one-vm-5440-disk-0 m10c3:
peer( Secondary -> Primary )
November 18th 2019, 12:23:57.038 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Preparing remote state change 1644389047
November 18th 2019, 12:23:57.038 |m14c14| drbd one-vm-5440-disk-0 m10c3:
Committing remote state change 1644389047 (primary_nodes=C)
November 18th 2019, 12:23:57.058 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m10c3: received new current UUID: BF79E942E02DA943
weak_nodes=FFFFFFFFFFFFFFF4
November 18th 2019, 12:24:04.138 |m14c14| drbd one-vm-5440-disk-0 m5c14:
peer( Primary -> Secondary )
November 18th 2019, 12:24:04.146 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: bitmap WRITE of 252 pages took 8 ms
November 18th 2019, 12:24:06.962 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:24:07.374 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Committing remote state change 396495173 (primary_nodes=8)
November 18th 2019, 12:24:07.374 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Preparing remote state change 396495173
November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Preparing remote state change 2772750783
November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Terminating ack_recv thread
November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0 m5c14:
conn( Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Committing remote state change 2772750783 (primary_nodes=8)
November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372 m5c14: pdsk( Diskless -> DUnknown ) repl( Established -> Off )
November 18th 2019, 12:24:07.422 |m14c14| drbd one-vm-5440-disk-0 m5c14:
ack_receiver terminated
November 18th 2019, 12:24:07.438 |m14c14| drbd one-vm-5440-disk-0 m5c14:
conn( TearDown -> Disconnecting )
November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Restarting sender thread
November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0/0
drbd2372: rs_discard_granularity changed to 524288
November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Terminating sender thread
November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Connection closed
November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0 m5c14:
conn( Disconnecting -> StandAlone )
November 18th 2019, 12:24:07.470 |m14c14| drbd one-vm-5440-disk-0 m5c14:
Terminating receiver thread

November 18th 2019, 12:03:13.710 |m13c20| drbd one-vm-5444-disk-0: Starting
worker thread (from drbdsetup [17328])
November 18th 2019, 12:03:13.714 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Starting sender thread (from drbdsetup [17333])
November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: meta-data IO uses: blk-bio
November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: Maximum number of peer devices = 7
November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: disk( Diskless -> Attaching )
November 18th 2019, 12:03:13.750 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: size = 15 GB (15729336 KB)
November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0: Method
to ensure write ordering: flush
November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: drbd_bm_resize called with capacity == 31458672
November 18th 2019, 12:03:13.753 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: resync bitmap: bits=3932334 words=430101 pages=841
November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: recounting of set bits took additional 0ms
November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: attached to current UUID: CDCBD9A0E058BAC4
November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: disk( Attaching -> UpToDate )
November 18th 2019, 12:03:13.786 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: size = 15 GB (15729336 KB)
November 18th 2019, 12:03:13.790 |m13c20| drbd one-vm-5444-disk-0 m14c16:
conn( StandAlone -> Unconnected )
November 18th 2019, 12:03:13.790 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Starting receiver thread (from drbd_w_one-vm-5 [17329])
November 18th 2019, 12:03:13.790 |m13c20| drbd one-vm-5444-disk-0 m14c16:
conn( Unconnected -> Connecting )
November 18th 2019, 12:03:13.982 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:03:14.317 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 12:03:14.317 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:03:14.318 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Peer authenticated using 20 bytes HMAC
November 18th 2019, 12:03:14.318 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Starting ack_recv thread (from drbd_r_one-vm-5 [17349])
November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0 m14c16:
conn( Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0:
Committing cluster-wide state change 1480358298 (0ms)
November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0:
Preparing cluster-wide state change 1480358298 (0->1 499/146)
November 18th 2019, 12:03:14.414 |m13c20| drbd one-vm-5444-disk-0: State
change 1480358298: primary_nodes=0, weak_nodes=0
November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m14c16: self
CDCBD9A0E058BAC4:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m14c16: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m14c16: uuid_compare()=0 by rule 40
November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m14c16: drbd_sync_handshake:
November 18th 2019, 12:03:14.454 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m14c16: peer
CDCBD9A0E058BAC4:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:03:14.694 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:03:14.914 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:03:15.126 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:03:15.622 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Starting sender thread (from drbdsetup [17513])
November 18th 2019, 12:03:15.646 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:03:15.687 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( StandAlone -> Unconnected )
November 18th 2019, 12:03:15.687 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Starting receiver thread (from drbd_w_one-vm-5 [17329])
November 18th 2019, 12:03:15.688 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( Unconnected -> Connecting )
November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Starting ack_recv thread (from drbd_r_one-vm-5 [17519])
November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Handshake to peer 2 successful: Agreed network protocol version 114
November 18th 2019, 12:03:16.218 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Peer authenticated using 20 bytes HMAC
November 18th 2019, 12:03:16.230 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Preparing remote state change 1510321921
November 18th 2019, 12:03:16.230 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Committing remote state change 1510321921 (primary_nodes=0)
November 18th 2019, 12:03:16.410 |m13c20| drbd one-vm-5444-disk-0:
Preparing cluster-wide state change 1326202137 (0->2 499/146)
November 18th 2019, 12:03:16.411 |m13c20| drbd one-vm-5444-disk-0:
Committing cluster-wide state change 1326202137 (0ms)
November 18th 2019, 12:03:16.411 |m13c20| drbd one-vm-5444-disk-0: State
change 1326202137: primary_nodes=0, weak_nodes=0
November 18th 2019, 12:03:16.411 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:03:16.446 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m5c5: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Preparing remote state change 3457834078
November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Committing remote state change 3457834078 (primary_nodes=4)
November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0 m5c5:
peer( Secondary -> Primary )
November 18th 2019, 12:03:16.682 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m5c5: received new current UUID: 48346FFFC6B44693
weak_nodes=FFFFFFFFFFFFFFF8
November 18th 2019, 12:03:16.690 |m13c20| drbd one-vm-5444-disk-0 m5c5:
peer( Primary -> Secondary )
November 18th 2019, 12:03:16.698 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Preparing remote state change 422036201
November 18th 2019, 12:03:16.698 |m13c20| drbd one-vm-5444-disk-0 m5c5:
peer( Secondary -> Primary )
November 18th 2019, 12:03:16.698 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Committing remote state change 422036201 (primary_nodes=4)
November 18th 2019, 12:03:21.850 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m5c5: received new current UUID: BE4FF7FC641F0E17
weak_nodes=FFFFFFFFFFFFFFF8
November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0 m15c41:
conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown )
November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0 m15c41:
PingAck did not arrive in time.
November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0 m15c41:
ack_receiver terminated
November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Terminating ack_recv thread
November 18th 2019, 12:05:14.303 |m13c20| drbd one-vm-5661-disk-0/0
drbd2340 m15c41: pdsk( Diskless -> DUnknown ) repl( Established -> Off )
November 18th 2019, 12:05:14.366 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Restarting sender thread
November 18th 2019, 12:05:14.405 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Restarting receiver thread
November 18th 2019, 12:05:14.405 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Connection closed
November 18th 2019, 12:05:14.405 |m13c20| drbd one-vm-5661-disk-0 m15c41:
conn( NetworkFailure -> Unconnected )
November 18th 2019, 12:05:14.406 |m13c20| drbd one-vm-5661-disk-0 m15c41:
conn( Unconnected -> Connecting )
November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37: conn(
Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915/0 drbd1370
m13c37: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
Preparing remote state change 3519185948
November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
ack_receiver terminated
November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
Committing remote state change 3519185948 (primary_nodes=0)
November 18th 2019, 12:05:41.674 |m13c20| drbd one-image-1915 m13c37:
Terminating ack_recv thread
November 18th 2019, 12:05:41.727 |m13c20| drbd one-image-1915 m13c37:
Restarting sender thread
November 18th 2019, 12:05:41.734 |m13c20| drbd one-image-1915 m13c37: conn(
TearDown -> Disconnecting )
November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37:
Connection closed
November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915/0 drbd1370:
disk( UpToDate -> Detaching )
November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915/0 drbd1370:
drbd_bm_resize called with capacity == 0
November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915/0 drbd1370:
disk( Detaching -> Diskless )
November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37:
Terminating sender thread
November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37: conn(
Disconnecting -> StandAlone )
November 18th 2019, 12:05:41.754 |m13c20| drbd one-image-1915 m13c37:
Terminating receiver thread
November 18th 2019, 12:05:41.785 |m13c20| drbd one-image-1915: Terminating
worker thread
November 18th 2019, 12:06:30.586 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Starting sender thread (from drbdsetup [21166])
November 18th 2019, 12:06:30.650 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:06:30.680 |m13c20| drbd one-vm-5444-disk-0 m15c13:
conn( StandAlone -> Unconnected )
November 18th 2019, 12:06:30.680 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Starting receiver thread (from drbd_w_one-vm-5 [17329])
November 18th 2019, 12:06:30.680 |m13c20| drbd one-vm-5444-disk-0 m15c13:
conn( Unconnected -> Connecting )
November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Starting ack_recv thread (from drbd_r_one-vm-5 [21173])
November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Handshake to peer 3 successful: Agreed network protocol version 115
November 18th 2019, 12:06:31.202 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Peer authenticated using 20 bytes HMAC
November 18th 2019, 12:06:31.398 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Preparing remote state change 2799790767
November 18th 2019, 12:06:31.398 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Committing remote state change 2799790767 (primary_nodes=4)
November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0:
Preparing cluster-wide state change 3271298677 (0->3 499/146)
November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0: State
change 3271298677: primary_nodes=4, weak_nodes=FFFFFFFFFFFFFFF8
November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0:
Committing cluster-wide state change 3271298677 (0ms)
November 18th 2019, 12:06:31.522 |m13c20| drbd one-vm-5444-disk-0 m15c13:
conn( Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:06:31.548 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m15c13: pdsk( DUnknown -> Diskless ) repl( Off -> Established )
November 18th 2019, 12:06:31.834 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:06:33.414 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Committing remote state change 3067680197 (primary_nodes=C)
November 18th 2019, 12:06:33.414 |m13c20| drbd one-vm-5444-disk-0 m15c13:
Preparing remote state change 3067680197
November 18th 2019, 12:06:33.415 |m13c20| drbd one-vm-5444-disk-0 m15c13:
peer( Secondary -> Primary )
November 18th 2019, 12:06:33.430 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m15c13: received new current UUID: D5AF2A094556BE75
weak_nodes=FFFFFFFFFFFFFFF4
November 18th 2019, 12:06:36.861 |m13c20| drbd one-vm-5444-disk-0 m5c5:
peer( Primary -> Secondary )
November 18th 2019, 12:06:39.454 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0 m5c5:
ack_receiver terminated
November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Preparing remote state change 3180102972
November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Committing remote state change 3180102972 (primary_nodes=8)
November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369 m5c5: pdsk( Diskless -> DUnknown ) repl( Established -> Off )
November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( Connected -> TearDown ) peer( Secondary -> Unknown )
November 18th 2019, 12:06:39.830 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Terminating ack_recv thread
November 18th 2019, 12:06:39.878 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Preparing remote state change 1753470888
November 18th 2019, 12:06:39.878 |m13c20| drbd one-vm-5444-disk-0 m14c16:
Committing remote state change 1753470888 (primary_nodes=8)
November 18th 2019, 12:06:39.886 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Restarting sender thread
November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Restarting receiver thread
November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( TearDown -> Unconnected )
November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( Unconnected -> Connecting )
November 18th 2019, 12:06:39.890 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Connection closed
November 18th 2019, 12:06:39.905 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Restarting sender thread
November 18th 2019, 12:06:39.905 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( Connecting -> Disconnecting )
November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Connection closed
November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0 m5c5:
conn( Disconnecting -> StandAlone )
November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Terminating sender thread
November 18th 2019, 12:06:39.906 |m13c20| drbd one-vm-5444-disk-0 m5c5:
Terminating receiver thread
November 18th 2019, 12:06:39.908 |m13c20| drbd one-vm-5444-disk-0/0
drbd2369: rs_discard_granularity changed to 524288
November 18th 2019, 12:06:56.878 |m13c20| drbd one-image-1883 m14c14:
Starting sender thread (from drbdsetup [21856])
November 18th 2019, 12:06:56.918 |m13c20| drbd one-image-1883/0 drbd1353:
rs_discard_granularity changed to 524288
November 18th 2019, 12:06:56.939 |m13c20| drbd one-image-1883 m14c14: conn(
StandAlone -> Unconnected )
November 18th 2019, 12:06:56.942 |m13c20| drbd one-image-1883 m14c14:
Starting receiver thread (from drbd_w_one-imag [31254])
November 18th 2019, 12:06:56.942 |m13c20| drbd one-image-1883 m14c14: conn(
Unconnected -> Connecting )
November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14:
Starting ack_recv thread (from drbd_r_one-imag [21863])
November 18th 2019, 12:06:57.570 |m13c20| drbd one-image-1883 m14c14:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883: Committing
cluster-wide state change 2284081876 (0ms)
November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883: Preparing
cluster-wide state change 2284081876 (0->1 499/146)
November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883: State change
2284081876: primary_nodes=0, weak_nodes=0
November 18th 2019, 12:06:57.616 |m13c20| drbd one-image-1883 m14c14: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: drbd_sync_handshake:
November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: Writing the whole bitmap, full sync required after
drbd_sync_handshake.
November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: peer
3454322884DFC87C:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: self
FBE6531F94A3E69A:0000000000000000:3454322884DFC87C:0000000000000000
bits:244875 flags:20
November 18th 2019, 12:06:57.634 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: uuid_compare()=3 by rule 80
November 18th 2019, 12:06:57.638 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
November 18th 2019, 12:06:57.638 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:06:57.642 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: pdsk( Consistent -> Outdated )
November 18th 2019, 12:06:57.642 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: helper command: /sbin/drbdadm before-resync-source
November 18th 2019, 12:06:57.642 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:06:57.643 |m13c20| Don't know which config file
belongs to resource one-image-1883, trying default ones...
November 18th 2019, 12:06:57.650 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
November 18th 2019, 12:06:57.650 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: Began resync as SyncSource (will sync 2100760 KB [525190 bits set]).
November 18th 2019, 12:06:57.650 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource )
November 18th 2019, 12:06:58.026 |m13c20| drbd one-image-1901 m14c14:
Starting sender thread (from drbdsetup [21903])
November 18th 2019, 12:06:58.061 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 18th 2019, 12:06:58.092 |m13c20| drbd one-image-1901 m14c14: conn(
StandAlone -> Unconnected )
November 18th 2019, 12:06:58.093 |m13c20| drbd one-image-1901 m14c14:
Starting receiver thread (from drbd_w_one-imag [31502])
November 18th 2019, 12:06:58.093 |m13c20| drbd one-image-1901 m14c14: conn(
Unconnected -> Connecting )
November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 12:06:59.238 |m13c20| drbd one-image-1901 m14c14:
Starting ack_recv thread (from drbd_r_one-imag [21910])
November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901: Preparing
cluster-wide state change 2709062739 (0->1 499/146)
November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901: State change
2709062739: primary_nodes=0, weak_nodes=0
November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901 m14c14: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:06:59.334 |m13c20| drbd one-image-1901: Committing
cluster-wide state change 2709062739 (0ms)
November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: drbd_sync_handshake:
November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: self
D7ECA5851EFE55DE:0000000000000000:4B6493096F3F93A6:0000000000000000
bits:35653 flags:20
November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: uuid_compare()=3 by rule 80
November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: Writing the whole bitmap, full sync required after
drbd_sync_handshake.
November 18th 2019, 12:06:59.375 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: peer
4B6493096F3F93A6:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:06:59.378 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
November 18th 2019, 12:06:59.378 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 99.9%
November 18th 2019, 12:06:59.445 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: pdsk( Consistent -> Outdated )
November 18th 2019, 12:06:59.447 |m13c20| Don't know which config file
belongs to resource one-image-1901, trying default ones...
November 18th 2019, 12:06:59.450 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: helper command: /sbin/drbdadm before-resync-source
November 18th 2019, 12:06:59.450 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 99.9%
November 18th 2019, 12:06:59.457 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource )
November 18th 2019, 12:06:59.457 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
November 18th 2019, 12:06:59.457 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: Began resync as SyncSource (will sync 515948 KB [128987 bits set]).
November 18th 2019, 12:06:59.950 |m13c20| drbd one-image-1902 m14c22:
Starting sender thread (from drbdsetup [21950])
November 18th 2019, 12:06:59.974 |m13c20| drbd one-image-1902/0 drbd1356:
rs_discard_granularity changed to 524288
November 18th 2019, 12:06:59.997 |m13c20| drbd one-image-1902 m14c22: conn(
StandAlone -> Unconnected )
November 18th 2019, 12:07:00.000 |m13c20| drbd one-image-1902 m14c22:
Starting receiver thread (from drbd_w_one-imag [31619])
November 18th 2019, 12:07:00.003 |m13c20| drbd one-image-1902 m14c22: conn(
Unconnected -> Connecting )
November 18th 2019, 12:07:00.336 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: Resync done (total 1 sec; paused 0 sec; 515948 K/sec)
November 18th 2019, 12:07:00.336 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: 72 % had equal checksums, eliminated: 372660K; transferred 143288K
total 515948K
November 18th 2019, 12:07:00.336 |m13c20| drbd one-image-1901/0 drbd1355
m14c14: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established )
November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22: Peer
authenticated using 20 bytes HMAC
November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22:
Handshake to peer 1 successful: Agreed network protocol version 115
November 18th 2019, 12:07:00.679 |m13c20| drbd one-image-1902 m14c22:
Starting ack_recv thread (from drbd_r_one-imag [21959])
November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902: State change
306230147: primary_nodes=0, weak_nodes=0
November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902: Committing
cluster-wide state change 306230147 (0ms)
November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902 m14c22: conn(
Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:07:00.740 |m13c20| drbd one-image-1902: Preparing
cluster-wide state change 306230147 (0->1 499/146)
November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: Writing the whole bitmap, full sync required after
drbd_sync_handshake.
November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: peer
E55EC4875988BFDC:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: self
B298F86F65888842:0000000000000000:E55EC4875988BFDC:0000000000000000
bits:224873 flags:20
November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: uuid_compare()=3 by rule 80
November 18th 2019, 12:07:00.767 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: drbd_sync_handshake:
November 18th 2019, 12:07:00.774 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:07:00.774 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
November 18th 2019, 12:07:00.784 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: pdsk( Consistent -> Outdated )
November 18th 2019, 12:07:00.786 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: helper command: /sbin/drbdadm before-resync-source
November 18th 2019, 12:07:00.786 |m13c20| Don't know which config file
belongs to resource one-image-1902, trying default ones...
November 18th 2019, 12:07:00.786 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 21(1), total
21; compression: 100.0%
November 18th 2019, 12:07:00.792 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: Began resync as SyncSource (will sync 2100760 KB [525190 bits set]).
November 18th 2019, 12:07:00.792 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
November 18th 2019, 12:07:00.792 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource )
November 18th 2019, 12:07:01.118 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
November 18th 2019, 12:07:01.118 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: 53 % had equal checksums, eliminated: 1120792K; transferred 979968K
total 2100760K
November 18th 2019, 12:07:01.118 |m13c20| drbd one-image-1883/0 drbd1353
m14c14: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established )
November 18th 2019, 12:07:03.894 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: Resync done (total 3 sec; paused 0 sec; 700252 K/sec)
November 18th 2019, 12:07:03.894 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: 57 % had equal checksums, eliminated: 1201364K; transferred 899396K
total 2100760K
November 18th 2019, 12:07:03.894 |m13c20| drbd one-image-1902/0 drbd1356
m14c22: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established )
November 18th 2019, 12:08:04.738 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Handshake to peer 3 successful: Agreed network protocol version 115
November 18th 2019, 12:08:04.739 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Peer authenticated using 20 bytes HMAC
November 18th 2019, 12:08:04.739 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Feature flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
WRITE_ZEROES.
November 18th 2019, 12:08:04.739 |m13c20| drbd one-vm-5661-disk-0 m15c41:
Starting ack_recv thread (from drbd_r_one-vm-5 [12782])
November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0:
Preparing cluster-wide state change 2061918045 (0->3 499/146)
November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0:
Committing cluster-wide state change 2061918045 (0ms)
November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0: State
change 2061918045: primary_nodes=0, weak_nodes=0
November 18th 2019, 12:08:04.776 |m13c20| drbd one-vm-5661-disk-0 m15c41:
conn( Connecting -> Connected ) peer( Unknown -> Secondary )
November 18th 2019, 12:08:04.798 |m13c20| drbd one-vm-5661-disk-0 m14c31:
Preparing remote state change 4147999259
November 18th 2019, 12:08:04.798 |m13c20| drbd one-vm-5661-disk-0 m14c31:
Committing remote state change 4147999259 (primary_nodes=0)
November 18th 2019, 12:08:04.818 |m13c20| drbd one-vm-5661-disk-0/0
drbd2340 m15c41: pdsk( DUnknown -> Diskless ) repl( Off -> Established )

5) Also we had these weird mesages after a while:

November 19th 2019, 01:15:35.522 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:15:42.158 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:18:57.694 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:19:04.018 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:26:58.610 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:27:04.486 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:27:14.310 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:27:20.074 |m14c14| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288

November 19th 2019, 01:15:35.222 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:15:42.110 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:18:57.354 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:19:04.406 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:26:58.322 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:27:04.458 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:27:13.678 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288
November 19th 2019, 01:27:20.034 |m13c20| drbd one-image-1901/0 drbd1355:
rs_discard_granularity changed to 524288

- kvaps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20191119/0422e924/attachment-0001.htm>