<div dir="ltr"><div style><br></div><div style>Thank for your reply !!</div><div style><br></div><div style>I have reverified the volume.<br></div><div style>But, on Ahead/Behind mode, the re-verify task seems to begin from first sector.</div>
<div style> </div><div style><div>I have tested below cases.</div><div> Case1 => OK</div><div> Case 2 => NG</div><div> Case 3 => NG</div></div><div style><br></div><div style>@Case1@@@@@@@@@@@@@@@@@@@@@@@@@@@@</div>
<div style>DRBD Connection Status = "Connected"</div><div style> -> Start Verify</div><div style> -> disconnect & connect from secondary site.</div><div style> -> re-Verify</div><div style><br></div>
<div style>@Case2@@@@@@@@@@@@@@@@@@@@@@@@@@@@</div><div style>DRBD Connection Status = "Connected"</div><div style> -> Start Verify</div><div style> -> i/o write generate (Ahead SyncSource)</div><div style>
-> Disconnect & Connect from DRBD_SD (After syncsource finished)</div><div style> -> re-Verify</div><div style><br></div><div style>@Case3@@@@@@@@@@@@@@@@@@@@@@@@@@@@</div><div style>DRBD Connection Status = "Connected"</div>
<div style> -> Start Verify</div><div style> -> i/o write generate (Ahead SyncSource)</div><div style> -> Wait Connected</div><div style> -> re-Verify</div><div style> => Verify restart from first sector</div>
<div style> =>>>>> BAD !!</div><div style><br></div><div style>[Environment]</div><div style> + DRBD Servers</div><div style> DRBD_PR (DRBD Primary Site)</div><div style> DRBD_SD (DRBD Secondary Site)</div>
<div style><br></div><div style> + uname of DRBD_PR & DRBD_SD</div><div style> Linux DRBD_PR 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux</div><div style><br></div><div style>
+ Use rpm</div><div style> drbd83-utils-8.3.15-1.el6.elrepo.x86_64</div><div style> kmod-drbd83-8.3.15-1.el6_3.elrepo.x86_64</div><div style><br></div><div style> + DRBD Configuration</div><div style> global {</div>
<div style> usage-count yes;</div><div style> }</div><div style> common {</div><div style> protocol A;</div><div style> handlers {</div><div style> }</div><div style> startup {</div>
<div style> wfc-timeout 120;</div><div style> degr-wfc-timeout 120;</div><div style> outdated-wfc-timeout 80;</div><div style> }</div><div style> disk {</div><div style>
on-io-error detach;</div><div style> no-disk-barrier;</div><div style> }</div><div style> net {</div><div style> sndbuf-size 0;</div><div style> on-congestion pull-ahead;</div>
<div style><br></div><div style> max-buffers 1000;</div><div style> max-epoch-size 1000;</div><div style> unplug-watermark 1000;</div><div style> congestion-fill 1M;</div>
<div style> congestion-extents 10;</div><div style> }</div><div style> syncer {</div><div style> rate 10K;</div><div style> verify-alg sha1;</div><div style> }</div>
<div style> }</div><div style><br></div><div style> resource TEST_VL {</div><div style> device minor 5;</div><div style> meta-disk internal;</div><div style> on DRBD_SD {</div><div style> disk /dev/sdvg/test_vl;</div>
<div style> address <a href="http://192.168.1.109:7805">192.168.1.109:7805</a>;</div><div style> }</div><div style> on DRBD_PR {</div><div style> disk /dev/prvg/test_vl;</div>
<div style> address <a href="http://192.168.1.100:7805">192.168.1.100:7805</a>;</div><div style> }</div><div style> }</div><div style><br></div><div style>@Case1@@@@@@@@@@@@@@@@@@@@@@@@@@@@</div><div style>
DRBD Connection Status = "Connected"</div><div style> -> Start Verify</div><div style> -> disconnect & connect from secondary site.</div><div style> -> re-Verify</div><div style> => Verify restart from reset of recent-verify</div>
<div style> =>>>>> OK</div><div style>-------------------------------------------------</div><div style>Aug 4 10:21:14 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )</div>
<div style>Aug 4 10:21:14 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0 jiffies</div><div style>Aug 4 10:21:14 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style>
Aug 4 10:21:28 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )</div><div style>Aug 4 10:21:28 DRBD_PR kernel: block drbd5: Starting Online Verify from sector 0</div><div style><<disconnect from Secondary[remote]>></div>
<div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: peer( Secondary -> Unknown ) conn( VerifyS -> TearDown ) pdsk( UpToDate -> DUnknown )</div><div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: Online Verify reached sector 16721800</div>
<div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: new current UUID BDC93C683F040729:894C0A6E1CEE0583:ABD34C46ABD736A9:A20E324B0A33CA9C</div><div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: asender terminated</div>
<div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: Terminating drbd5_asender</div><div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: Connection closed</div><div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0 jiffies</div>
<div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: conn( TearDown -> Unconnected )</div><div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style>
Aug 4 10:23:26 DRBD_PR kernel: block drbd5: receiver terminated</div><div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: Restarting drbd5_receiver</div><div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: receiver (re)started</div>
<div style>Aug 4 10:23:26 DRBD_PR kernel: block drbd5: conn( Unconnected -> WFConnection )</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: Handshake successful: Agreed network protocol version 97</div><div style>
Aug 4 10:23:46 DRBD_PR kernel: block drbd5: conn( WFConnection -> WFReportParams )</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: Starting asender thread (from drbd5_receiver [8357])</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: data-integrity-alg: <not-used></div>
<div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: drbd_sync_handshake:</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: self BDC93C683F040729:894C0A6E1CEE0583:ABD34C46ABD736A9:A20E324B0A33CA9C bits:0 flags:0</div>
<div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: peer 894C0A6E1CEE0582:0000000000000000:ABD34C46ABD736A8:A20E324B0A33CA9C bits:0 flags:0</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: uuid_compare()=1 by rule 70</div>
<div style><<connect from Secondary[remote]>></div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> Consistent )</div>
<div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5 exit code 0 (0x0)</div>
<div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: conn( WFBitMapS -> SyncSource ) pdsk( Consistent -> Inconsistent )</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: Began resync as SyncSource (will sync 0 KB [0 bits set]).</div>
<div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: updated sync UUID BDC93C683F040729:894D0A6E1CEE0583:894C0A6E1CEE0583:ABD34C46ABD736A9</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: Resync done (total 1 sec; paused 0 sec; 0 K/sec)</div>
<div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: updated UUIDs BDC93C683F040729:0000000000000000:894D0A6E1CEE0583:894C0A6E1CEE0583</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )</div>
<div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: bitmap WRITE of 80 pages took 0 jiffies</div><div style>Aug 4 10:23:46 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style>
Aug 4 10:24:08 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )</div><div style>Aug 4 10:24:08 DRBD_PR kernel: block drbd5: Starting Online Verify from sector 16721800 <<<<<<<<<<<<<<<<<<<<<<<< Verify Restart for rest sector!! >>>>>>>></div>
<div style>Aug 4 10:24:38 DRBD_PR kernel: block drbd5: Online verify done (total 29 sec; paused 0 sec; 73256 K/sec)</div><div style>Aug 4 10:24:38 DRBD_PR kernel: block drbd5: conn( VerifyS -> Connected )</div><div style>
Aug 4 10:24:38 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0 jiffies</div><div style>Aug 4 10:24:38 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style><br></div>
<div style><br></div><div style>@Case2@@@@@@@@@@@@@@@@@@@@@@@@@@@@</div><div style>DRBD Connection Status = "Connected"</div><div style> -> Start Verify</div><div style> -> i/o write generate (Ahead SyncSource)</div>
<div style> -> Disconnect & Connect from DRBD_SD (After syncsource finished)</div><div style> -> re-Verify</div><div style> => Verify restart from first sector</div><div style> =>>>>> BAD !!</div>
<div style>------------------------------------</div><div style>Aug 4 13:24:53 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )</div><div style>Aug 4 13:24:53 DRBD_PR kernel: block drbd5: Starting Online Verify from sector 0</div>
<div style>Aug 4 13:25:15 DRBD_PR kernel: block drbd5: Congestion-fill threshold reached</div><div style>Aug 4 13:25:15 DRBD_PR kernel: block drbd5: conn( VerifyS -> Ahead ) pdsk( UpToDate -> Consistent )</div><div style>
Aug 4 13:25:16 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5</div><div style>Aug 4 13:25:16 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5 exit code 0 (0x0)</div>
<div style>Aug 4 13:25:16 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource ) pdsk( Consistent -> Inconsistent )</div><div style>Aug 4 13:25:16 DRBD_PR kernel: block drbd5: Began resync as SyncSource (will sync 716888 KB [179222 bits set]).</div>
<div style>Aug 4 13:25:16 DRBD_PR kernel: block drbd5: updated sync UUID E24B239F571815C9:F60BD2AC0F7D8378:D36150D3C5DA6466:D36050D3C5DA6466</div><div style>Aug 4 13:25:16 DRBD_PR kernel: block drbd5: Congestion-fill threshold reached</div>
<div style>Aug 4 13:25:16 DRBD_PR kernel: block drbd5: conn( SyncSource -> Ahead )</div><div style>Aug 4 13:25:17 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5</div><div style>
Aug 4 13:25:17 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5 exit code 0 (0x0)</div><div style>Aug 4 13:25:17 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource )</div><div style>
Aug 4 13:25:17 DRBD_PR kernel: block drbd5: Began resync as SyncSource (will sync 1040468 KB [260117 bits set]).</div><div style>Aug 4 13:25:17 DRBD_PR kernel: block drbd5: updated sync UUID E24B239F571815C9:F60CD2AC0F7D8378:F60BD2AC0F7D8378:D36150D3C5DA6466</div>
<div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: peer( Secondary -> Unknown ) conn( SyncSource -> TearDown )</div><div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: bitmap WRITE of 7 pages took 0 jiffies</div>
<div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: 882 MB (225791 bits) marked out-of-sync by on disk bit-map.</div><div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: asender terminated</div><div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: Terminating drbd5_asender</div>
<div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: Connection closed</div><div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: conn( TearDown -> Unconnected )</div><div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: receiver terminated</div>
<div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: Restarting drbd5_receiver</div><div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: receiver (re)started</div><div style>Aug 4 13:25:31 DRBD_PR kernel: block drbd5: conn( Unconnected -> WFConnection )</div>
<div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: Handshake successful: Agreed network protocol version 97</div><div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: conn( WFConnection -> WFReportParams )</div>
<div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: Starting asender thread (from drbd5_receiver [19133])</div><div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: data-integrity-alg: <not-used></div><div style>
Aug 4 13:25:35 DRBD_PR kernel: block drbd5: drbd_sync_handshake:</div><div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: self E24B239F571815C9:F60CD2AC0F7D8378:F60BD2AC0F7D8378:D36150D3C5DA6466 bits:225791 flags:0</div>
<div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: peer F60CD2AC0F7D8378:0000000000000000:D36150D3C5DA6466:D36050D3C5DA6466 bits:225791 flags:0</div><div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: uuid_compare()=1 by rule 70</div>
<div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: Becoming sync source due to disk states.</div><div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS )</div>
<div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5</div><div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5 exit code 0 (0x0)</div>
<div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: conn( WFBitMapS -> SyncSource )</div><div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: Began resync as SyncSource (will sync 903164 KB [225791 bits set]).</div>
<div style>Aug 4 13:25:35 DRBD_PR kernel: block drbd5: updated sync UUID E24B239F571815C9:F60DD2AC0F7D8378:F60CD2AC0F7D8378:F60BD2AC0F7D8378</div><div style>Aug 4 13:25:53 DRBD_PR kernel: block drbd5: State change failed: Can not start OV/resync since it is already active</div>
<div style>Aug 4 13:25:53 DRBD_PR kernel: block drbd5: state = { cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent r----- }</div><div style>Aug 4 13:25:53 DRBD_PR kernel: block drbd5: wanted = { cs:VerifyS ro:Primary/Secondary ds:UpToDate/UpToDate r----- }</div>
<div style>Aug 4 13:27:03 DRBD_PR kernel: block drbd5: Resync done (total 88 sec; paused 0 sec; 10260 K/sec)</div><div style>Aug 4 13:27:03 DRBD_PR kernel: block drbd5: updated UUIDs E24B239F571815C9:0000000000000000:F60DD2AC0F7D8378:F60CD2AC0F7D8378</div>
<div style>Aug 4 13:27:03 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )</div><div style>Aug 4 13:27:03 DRBD_PR kernel: block drbd5: bitmap WRITE of 73 pages took 0 jiffies</div>
<div style>Aug 4 13:27:03 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style>Aug 4 13:29:13 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )</div><div style>Aug 4 13:29:13 DRBD_PR kernel: block drbd5: Starting Online Verify from sector 0 <<<<<<BAD!!>>>>></div>
<div style><br></div><div style>@Case3@@@@@@@@@@@@@@@@@@@@@@@@@@@@</div><div style>DRBD Connection Status = "Connected"</div><div style> -> Start verify</div><div style> -> i/o write generate (Ahead SyncSource)</div>
<div style> -> Wait Connected</div><div style> -> re-Verify</div><div style> => Verify restart from first sector</div><div style> =>>>>> BAD !!</div><div style>-----------------------------------------------------</div>
<div style>Aug 4 13:40:00 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style>Aug 4 13:41:33 DRBD_PR kernel: block drbd5: Congestion-fill threshold reached</div><div style>Aug 4 13:41:33 DRBD_PR kernel: block drbd5: conn( Connected -> Ahead ) pdsk( UpToDate -> Consistent )</div>
<div style>Aug 4 13:41:34 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5</div><div style>Aug 4 13:41:34 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5 exit code 0 (0x0)</div>
<div style>Aug 4 13:41:34 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource ) pdsk( Consistent -> Inconsistent )</div><div style>Aug 4 13:41:34 DRBD_PR kernel: block drbd5: Began resync as SyncSource (will sync 715444 KB [178861 bits set]).</div>
<div style>Aug 4 13:41:34 DRBD_PR kernel: block drbd5: updated sync UUID E24B239F571815C9:6B4023CE85C9E0F5:F60DD2AC0F7D8378:F60CD2AC0F7D8378</div><div style>Aug 4 13:41:34 DRBD_PR kernel: block drbd5: Congestion-fill threshold reached</div>
<div style>Aug 4 13:41:34 DRBD_PR kernel: block drbd5: conn( SyncSource -> Ahead )</div><div style>Aug 4 13:41:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5</div><div style>
Aug 4 13:41:35 DRBD_PR kernel: block drbd5: helper command: /sbin/drbdadm before-resync-source minor-5 exit code 0 (0x0)</div><div style>Aug 4 13:41:35 DRBD_PR kernel: block drbd5: conn( Ahead -> SyncSource )</div><div style>
Aug 4 13:41:35 DRBD_PR kernel: block drbd5: Began resync as SyncSource (will sync 1042292 KB [260573 bits set]).</div><div style>Aug 4 13:41:35 DRBD_PR kernel: block drbd5: updated sync UUID E24B239F571815C9:6B4123CE85C9E0F5:6B4023CE85C9E0F5:F60DD2AC0F7D8378</div>
<div style>Aug 4 13:43:54 DRBD_PR kernel: block drbd5: Resync done (total 139 sec; paused 0 sec; 7496 K/sec)</div><div style>Aug 4 13:43:54 DRBD_PR kernel: block drbd5: updated UUIDs E24B239F571815C9:0000000000000000:6B4123CE85C9E0F5:6B4023CE85C9E0F5</div>
<div style>Aug 4 13:43:54 DRBD_PR kernel: block drbd5: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )</div><div style>Aug 4 13:43:54 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0 jiffies</div>
<div style>Aug 4 13:43:54 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style>Aug 4 13:44:37 DRBD_PR kernel: block drbd5: conn( Connected -> VerifyS )</div><div style>Aug 4 13:44:37 DRBD_PR kernel: block drbd5: Starting Online Verify from sector 0 <<< BAD!! >>></div>
<div style>Aug 4 13:47:55 DRBD_PR kernel: block drbd5: Online verify done (total 197 sec; paused 0 sec; 53224 K/sec)</div><div style>Aug 4 13:47:55 DRBD_PR kernel: block drbd5: conn( VerifyS -> Connected )</div><div style>
Aug 4 13:47:55 DRBD_PR kernel: block drbd5: bitmap WRITE of 0 pages took 0 jiffies</div><div style>Aug 4 13:47:55 DRBD_PR kernel: block drbd5: 0 KB (0 bits) marked out-of-sync by on disk bit-map.</div><div style><br></div>
<div style="font-style:italic"><br></div></div>