[DRBD-user] Down sync
Juan Sevilla
juan.sevilla.11 at gmail.com
Thu Jul 23 10:31:42 CEST 2020
The resource drbd02 is just now down between drbd02 and drbd03. Where can i
review the more logs?? Thanks in advance
Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: meta connection shut
> down by peer.
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: conn( Connected ->
> NetworkFailure ) peer( Secondary -> Unknown )
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02/1 drbd8 drbd03: pdsk( Diskless
> -> DUnknown ) repl( Established -> Off )
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: ack_receiver terminated
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: Terminating ack_recv
> thread
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: sock was shut down by
> peer
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: Restarting sender
> thread
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: Connection closed
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: conn( NetworkFailure
> -> Unconnected )
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: Restarting receiver
> thread
> Jul 23 10:05:57 drbd02 kernel: drbd MIGRA02 drbd03: conn( Unconnected ->
> Connecting )
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Handshake to peer 2
> successful: Agreed network protocol version 117
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Feature flags enabled
> on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES.
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Starting ack_recv
> thread (from drbd_r_MIGRA02 [2695])
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02: Preparing cluster-wide state
> change 1863242544 (1->2 499/145)
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02: Declined by peer drbd01 (id:
> 0), see the kernel log there
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02: Aborting cluster-wide state
> change 1863242544 (19ms) rv = -10
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Failure to connect;
> retrying
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: conn( Connecting ->
> NetworkFailure )
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: ack_receiver terminated
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Terminating ack_recv
> thread
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Restarting sender
> thread
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Connection closed
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: conn( NetworkFailure
> -> Unconnected )
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Restarting receiver
> thread
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: conn( Unconnected ->
> Connecting )
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Handshake to peer 2
> successful: Agreed network protocol version 117
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Feature flags enabled
> on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES.
> Jul 23 10:05:58 drbd02 kernel: drbd MIGRA02 drbd03: Starting ack_recv
> thread (from drbd_r_MIGRA02 [2695])
> Jul 23 10:05:59 drbd02 kernel: drbd MIGRA02: Preparing cluster-wide state
> change 1892110034 (1->2 499/145)
> Jul 23 10:05:59 drbd02 kernel: drbd MIGRA02: Declined by peer drbd01 (id:
> 0), see the kernel log there
> Jul 23 10:05:59 drbd02 kernel: drbd MIGRA02: Aborting cluster-wide state
> change 1892110034 (0ms) rv = -10
> Jul 23 10:05:59 drbd02 kernel: drbd MIGRA02 drbd03: Failure to connect;
> retrying
> Jul 23 10:05:59 drbd02 kernel: drbd MIGRA02 drbd03: conn( Connecting ->
> NetworkFailure )
..........
El jue., 23 jul. 2020 a las 9:19, Juan Sevilla (<juan.sevilla.11 at gmail.com>)
escribió:
> Hi,
>
> My configuration is this:
>
> A) Node drbd01: primary all
>
> B) Node drbd02: primary all
>
> C) Node drbd03: secondary all, diskless, for quorum proposal.
>
> Initially all run correctly, but after various hours the sync between drbd
> nodes is lost, in spite of the connections (ping) on the networks is ok.
>
> Some times, the witness (node drbd03) appears "connecting" to drbd01,
> another times is the node drbd02, etc. My OS is RHEL 7, and firewalld is
> stopped and disabled, also SELinux is disabled...
>
> What could be happening?
>
>
> [root at drbd01 drbd.d]# uname -a
>> Linux drbd01 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020
>> x86_64 x86_64 x86_64 GNU/Linux
>> [root at drbd01 drbd.d]#
>> [root at drbd01 drbd.d]# cat global_common.conf
>> global {
>> usage-count no;
>> udev-always-use-vnr;
>> }
>> common {
>> handlers {
>> }
>> startup {
>> }
>> options {
>> quorum majority;
>> # on-no-quorum io-error;
>> # quorum-minimum-redundancy 1;
>> }
>> disk {
>> }
>> net {
>> verify-alg crc32c;
>> }
>> }
>> [root at drbd01 drbd.d]# cat *.res |more
>> resource DATA01 {
>> volume 1 {
>> disk /dev/sdf;
>> device /dev/drbd4;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7791;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7791;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7791;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource DATA02 {
>> volume 1 {
>> disk /dev/sdg;
>> device /dev/drbd5;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7792;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7792;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7792;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource DATA03 {
>> volume 1 {
>> disk /dev/sdh;
>> device /dev/drbd6;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7793;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7793;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7793;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource GIMR01 {
>> volume 1 {
>> disk /dev/sde;
>> device /dev/drbd3;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7790;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7790;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7790;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>> resource MIGRA01 {
>> volume 1 {
>> disk /dev/sdi;
>> device /dev/drbd7;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7794;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7794;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7794;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource MIGRA02 {
>> volume 1 {
>> disk /dev/sdj;
>> device /dev/drbd8;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7795;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7795;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7795;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource MIGRA03 {
>> volume 1 {
>> disk /dev/sdk;
>> device /dev/drbd9;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7796;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7796;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7796;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource MIGRA04 {
>> volume 1 {
>> disk /dev/sdl;
>> device /dev/drbd10;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7797;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7797;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7797;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource OCR01 {
>> volume 1 {
>> disk /dev/sdb;
>> device /dev/drbd0;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7787;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7787;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7787;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>> resource OCR02 {
>> volume 1 {
>> disk /dev/sdc;
>> device /dev/drbd1;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7788;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7788;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7788;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>> resource OCR03 {
>> volume 1 {
>> disk /dev/sdd;
>> device /dev/drbd2;
>> meta-disk internal;
>> }
>> on drbd01 {
>> address 10.10.10.1:7789;
>> node-id 0;
>> }
>> on drbd02 {
>> address 10.10.10.2:7789;
>> node-id 1;
>> }
>> on drbd03 {
>> address 10.10.10.3:7789;
>> node-id 2;
>> volume 1 {
>> disk none;
>> }
>>
>>
>> }
>> connection-mesh {
>> hosts drbd01 drbd02 drbd03;
>> net {
>> protocol C;
>> allow-two-primaries yes;
>> }
>> }
>>
>> }
>>
>
>
> Best regards.
> Juan.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20200723/8c27bc33/attachment-0001.htm>
More information about the drbd-user
mailing list