[DRBD-user] DRBD on-line verify task will be abborted on-congestion pull-ahead.

Takeshi.Y yahagi.tgi at gmail.com
Mon Aug 4 07:18:10 CEST 2014

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Thank for your reply !!

I have reverified the volume.
But, on Ahead/Behind mode, the re-verify task seems to begin from first
sector.

I have tested below cases.
  Case1 => OK
  Case 2 => NG
  Case 3 => NG

@Case1@@@@@@@@@@@@@@@@@@@@@@@@@@@@
DRBD Connection Status = "Connected"
 -> Start Verify
 -> disconnect & connect from secondary site.
 -> re-Verify

@Case2@@@@@@@@@@@@@@@@@@@@@@@@@@@@
DRBD Connection Status = "Connected"
 -> Start Verify
 -> i/o write generate (Ahead SyncSource)
 -> Disconnect & Connect from DRBD_SD (After syncsource finished)
 -> re-Verify

@Case3@@@@@@@@@@@@@@@@@@@@@@@@@@@@
DRBD Connection Status = "Connected"
 -> Start Verify
 -> i/o write generate (Ahead SyncSource)
 -> Wait Connected
 -> re-Verify
    => Verify restart from first sector
      =>>>>> BAD !!

[Environment]
  + DRBD Servers
     DRBD_PR (DRBD Primary Site)
     DRBD_SD (DRBD Secondary Site)

  + uname of DRBD_PR & DRBD_SD
    Linux DRBD_PR 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013
x86_64 x86_64 x86_64 GNU/Linux

  + Use rpm
    drbd83-utils-8.3.15-1.el6.elrepo.x86_64
    kmod-drbd83-8.3.15-1.el6_3.elrepo.x86_64

  + DRBD Configuration
    global {
        usage-count yes;
    }
    common {
        protocol A;
        handlers {
        }
        startup {
                wfc-timeout 120;
                degr-wfc-timeout 120;
                outdated-wfc-timeout 80;
        }
        disk {
                on-io-error detach;
                no-disk-barrier;
        }
        net {
                sndbuf-size 0;
                on-congestion pull-ahead;

                max-buffers 1000;
                max-epoch-size 1000;
                unplug-watermark 1000;
                congestion-fill 1M;
                congestion-extents 10;
        }
        syncer {
                rate 10K;
                verify-alg sha1;
        }
    }

    resource TEST_VL {
        device  minor 5;
        meta-disk internal;
        on DRBD_SD {
                disk    /dev/sdvg/test_vl;
                address 192.168.1.109:7805;
        }
        on DRBD_PR {
                disk    /dev/prvg/test_vl;
                address 192.168.1.100:7805;
        }
    }

@Case1@@@@@@@@@@@@@@@@@@@@@@@@@@@@
DRBD Connection Status = "Connected"
 -> Start Verify
 -> disconnect & connect from secondary site.
 -> re-Verify
    => Verify restart from reset of recent-verify
      =>>>>> OK
-------------------------------------------------
Aug  4 10:21:14 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected
) pdsk( Inconsistent -> UpToDate )
Aug  4 10:21:14 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0
jiffies
Aug  4 10:21:14 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug  4 10:21:28 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )
Aug  4 10:21:28 DRBD_PR kernel: block drbd5: Starting Online Verify from
sector 0
<<disconnect from Secondary[remote]>>
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: peer( Secondary -> Unknown )
conn( VerifyS -> TearDown ) pdsk( UpToDate -> DUnknown )
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: Online Verify reached sector
16721800
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: new current UUID
BDC93C683F040729:894C0A6E1CEE0583:ABD34C46ABD736A9:A20E324B0A33CA9C
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: asender terminated
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: Terminating drbd5_asender
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: Connection closed
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0
jiffies
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: conn( TearDown -> Unconnected )
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: receiver terminated
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: Restarting drbd5_receiver
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: receiver (re)started
Aug  4 10:23:26 DRBD_PR kernel: block drbd5: conn( Unconnected ->
WFConnection )
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: Handshake successful: Agreed
network protocol version 97
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: conn( WFConnection ->
WFReportParams )
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: Starting asender thread (from
drbd5_receiver [8357])
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: data-integrity-alg: <not-used>
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: drbd_sync_handshake:
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: self
BDC93C683F040729:894C0A6E1CEE0583:ABD34C46ABD736A9:A20E324B0A33CA9C bits:0
flags:0
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: peer
894C0A6E1CEE0582:0000000000000000:ABD34C46ABD736A8:A20E324B0A33CA9C bits:0
flags:0
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: uuid_compare()=1 by rule 70
<<connect from Secondary[remote]>>
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: peer( Unknown -> Secondary )
conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> Consistent )
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5 exit code 0 (0x0)
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: conn( WFBitMapS -> SyncSource
) pdsk( Consistent -> Inconsistent )
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: Began resync as SyncSource
(will sync 0 KB [0 bits set]).
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: updated sync UUID
BDC93C683F040729:894D0A6E1CEE0583:894C0A6E1CEE0583:ABD34C46ABD736A9
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: Resync done (total 1 sec;
paused 0 sec; 0 K/sec)
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: updated UUIDs
BDC93C683F040729:0000000000000000:894D0A6E1CEE0583:894C0A6E1CEE0583
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected
) pdsk( Inconsistent -> UpToDate )
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: bitmap WRITE of 80 pages took
0 jiffies
Aug  4 10:23:46 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug  4 10:24:08 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )
Aug  4 10:24:08 DRBD_PR kernel: block drbd5: Starting Online Verify from
sector 16721800    <<<<<<<<<<<<<<<<<<<<<<<< Verify Restart for rest
sector!! >>>>>>>>
Aug  4 10:24:38 DRBD_PR kernel: block drbd5: Online verify done (total 29
sec; paused 0 sec; 73256 K/sec)
Aug  4 10:24:38 DRBD_PR kernel: block drbd5: conn( VerifyS -> Connected )
Aug  4 10:24:38 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0
jiffies
Aug  4 10:24:38 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.


@Case2@@@@@@@@@@@@@@@@@@@@@@@@@@@@
DRBD Connection Status = "Connected"
 -> Start Verify
 -> i/o write generate (Ahead SyncSource)
 -> Disconnect & Connect from DRBD_SD (After syncsource finished)
 -> re-Verify
    => Verify restart from first sector
      =>>>>> BAD !!
------------------------------------
Aug  4 13:24:53 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )
Aug  4 13:24:53 DRBD_PR kernel: block drbd5: Starting Online Verify from
sector 0
Aug  4 13:25:15 DRBD_PR kernel: block drbd5: Congestion-fill threshold
reached
Aug  4 13:25:15 DRBD_PR kernel: block drbd5: conn( VerifyS -> Ahead ) pdsk(
UpToDate -> Consistent )
Aug  4 13:25:16 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5
Aug  4 13:25:16 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5 exit code 0 (0x0)
Aug  4 13:25:16 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource )
pdsk( Consistent -> Inconsistent )
Aug  4 13:25:16 DRBD_PR kernel: block drbd5: Began resync as SyncSource
(will sync 716888 KB [179222 bits set]).
Aug  4 13:25:16 DRBD_PR kernel: block drbd5: updated sync UUID
E24B239F571815C9:F60BD2AC0F7D8378:D36150D3C5DA6466:D36050D3C5DA6466
Aug  4 13:25:16 DRBD_PR kernel: block drbd5: Congestion-fill threshold
reached
Aug  4 13:25:16 DRBD_PR kernel: block drbd5: conn( SyncSource -> Ahead )
Aug  4 13:25:17 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5
Aug  4 13:25:17 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5 exit code 0 (0x0)
Aug  4 13:25:17 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource )
Aug  4 13:25:17 DRBD_PR kernel: block drbd5: Began resync as SyncSource
(will sync 1040468 KB [260117 bits set]).
Aug  4 13:25:17 DRBD_PR kernel: block drbd5: updated sync UUID
E24B239F571815C9:F60CD2AC0F7D8378:F60BD2AC0F7D8378:D36150D3C5DA6466
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: peer( Secondary -> Unknown )
conn( SyncSource -> TearDown )
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: bitmap WRITE of 7 pages took 0
jiffies
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: 882 MB (225791 bits) marked
out-of-sync by on disk bit-map.
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: asender terminated
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: Terminating drbd5_asender
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: Connection closed
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: conn( TearDown -> Unconnected )
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: receiver terminated
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: Restarting drbd5_receiver
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: receiver (re)started
Aug  4 13:25:31 DRBD_PR kernel: block drbd5: conn( Unconnected ->
WFConnection )
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: Handshake successful: Agreed
network protocol version 97
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: conn( WFConnection ->
WFReportParams )
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: Starting asender thread (from
drbd5_receiver [19133])
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: data-integrity-alg: <not-used>
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: drbd_sync_handshake:
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: self
E24B239F571815C9:F60CD2AC0F7D8378:F60BD2AC0F7D8378:D36150D3C5DA6466
bits:225791 flags:0
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: peer
F60CD2AC0F7D8378:0000000000000000:D36150D3C5DA6466:D36050D3C5DA6466
bits:225791 flags:0
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: uuid_compare()=1 by rule 70
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: Becoming sync source due to
disk states.
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: peer( Unknown -> Secondary )
conn( WFReportParams -> WFBitMapS )
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5 exit code 0 (0x0)
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: conn( WFBitMapS -> SyncSource )
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: Began resync as SyncSource
(will sync 903164 KB [225791 bits set]).
Aug  4 13:25:35 DRBD_PR kernel: block drbd5: updated sync UUID
E24B239F571815C9:F60DD2AC0F7D8378:F60CD2AC0F7D8378:F60BD2AC0F7D8378
Aug  4 13:25:53 DRBD_PR kernel: block drbd5: State change failed: Can not
start OV/resync since it is already active
Aug  4 13:25:53 DRBD_PR kernel: block drbd5:   state = { cs:SyncSource
ro:Primary/Secondary ds:UpToDate/Inconsistent r----- }
Aug  4 13:25:53 DRBD_PR kernel: block drbd5:  wanted = { cs:VerifyS
ro:Primary/Secondary ds:UpToDate/UpToDate r----- }
Aug  4 13:27:03 DRBD_PR kernel: block drbd5: Resync done (total 88 sec;
paused 0 sec; 10260 K/sec)
Aug  4 13:27:03 DRBD_PR kernel: block drbd5: updated UUIDs
E24B239F571815C9:0000000000000000:F60DD2AC0F7D8378:F60CD2AC0F7D8378
Aug  4 13:27:03 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected
) pdsk( Inconsistent -> UpToDate )
Aug  4 13:27:03 DRBD_PR kernel: block drbd5: bitmap WRITE of 73 pages took
0 jiffies
Aug  4 13:27:03 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug  4 13:29:13 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )
Aug  4 13:29:13 DRBD_PR kernel: block drbd5: Starting Online Verify from
sector 0  <<<<<<BAD!!>>>>>

@Case3@@@@@@@@@@@@@@@@@@@@@@@@@@@@
DRBD Connection Status = "Connected"
 -> Start verify
 -> i/o write generate (Ahead SyncSource)
 -> Wait Connected
 -> re-Verify
    => Verify restart from first sector
      =>>>>> BAD !!
-----------------------------------------------------
Aug  4 13:40:00 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug  4 13:41:33 DRBD_PR kernel: block drbd5: Congestion-fill threshold
reached
Aug  4 13:41:33 DRBD_PR kernel: block drbd5: conn( Connected -> Ahead )
pdsk( UpToDate -> Consistent )
Aug  4 13:41:34 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5
Aug  4 13:41:34 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5 exit code 0 (0x0)
Aug  4 13:41:34 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource )
pdsk( Consistent -> Inconsistent )
Aug  4 13:41:34 DRBD_PR kernel: block drbd5: Began resync as SyncSource
(will sync 715444 KB [178861 bits set]).
Aug  4 13:41:34 DRBD_PR kernel: block drbd5: updated sync UUID
E24B239F571815C9:6B4023CE85C9E0F5:F60DD2AC0F7D8378:F60CD2AC0F7D8378
Aug  4 13:41:34 DRBD_PR kernel: block drbd5: Congestion-fill threshold
reached
Aug  4 13:41:34 DRBD_PR kernel: block drbd5: conn( SyncSource -> Ahead )
Aug  4 13:41:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5
Aug  4 13:41:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm
before-resync-source minor-5 exit code 0 (0x0)
Aug  4 13:41:35 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource )
Aug  4 13:41:35 DRBD_PR kernel: block drbd5: Began resync as SyncSource
(will sync 1042292 KB [260573 bits set]).
Aug  4 13:41:35 DRBD_PR kernel: block drbd5: updated sync UUID
E24B239F571815C9:6B4123CE85C9E0F5:6B4023CE85C9E0F5:F60DD2AC0F7D8378
Aug  4 13:43:54 DRBD_PR kernel: block drbd5: Resync done (total 139 sec;
paused 0 sec; 7496 K/sec)
Aug  4 13:43:54 DRBD_PR kernel: block drbd5: updated UUIDs
E24B239F571815C9:0000000000000000:6B4123CE85C9E0F5:6B4023CE85C9E0F5
Aug  4 13:43:54 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected
) pdsk( Inconsistent -> UpToDate )
Aug  4 13:43:54 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0
jiffies
Aug  4 13:43:54 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
Aug  4 13:44:37 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )
Aug  4 13:44:37 DRBD_PR kernel: block drbd5: Starting Online Verify from
sector 0    <<< BAD!! >>>
Aug  4 13:47:55 DRBD_PR kernel: block drbd5: Online verify done (total 197
sec; paused 0 sec; 53224 K/sec)
Aug  4 13:47:55 DRBD_PR kernel: block drbd5: conn( VerifyS -> Connected )
Aug  4 13:47:55 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0
jiffies
Aug  4 13:47:55 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked
out-of-sync by on disk bit-map.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20140804/3d4b92a4/attachment.htm>


More information about the drbd-user mailing list