<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd">
<html><head><meta name="qrichtext" content="1" /><style type="text/css">
p, li { white-space: pre-wrap; }
</style></head><body style=" font-family:'Oxygen-Sans'; font-size:11pt; font-weight:400; font-style:normal;">
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hello guys,</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">please help us to understand strange verify behaviour described below.</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I have a feeling of verify emulation instead of really verify the data.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Thanks!</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-weight:600; font-style:italic; color:#000000;">---</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-weight:600; font-style:italic; color:#000000;">Best regards,</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-style:italic; color:#000000;">Eugene Istomin</span></p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br />On Friday, 19 February 2016 12:52:53 EET Eugene Istomin wrote:<br /></p>
<p style=" margin-top:12px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hello,</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">any comments? =)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Verify test results are seems very strange for us.</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-weight:600; font-style:italic; color:#000000;">---</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-weight:600; font-style:italic; color:#000000;">Best regards,</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-style:italic; color:#000000;">Eugene Istomin</span></p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br />On Wednesday, 17 February 2016 18:34:27 EET Eugene Istomin wrote:<br /></p>
<p style=" margin-top:12px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hello,</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">we done some tests with DRBD9 9.0.1<br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Some preliminary results:</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">#### 1. Verify depends on "connection-mesh" position. ####</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> connection-mesh {</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> hosts 6787-dblapro-edss 6788-dblapro-edss 6789-dblapro-edss;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> }</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">If the first node is switched off - verify on the second if failed:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">6788-dblapro-edss# drbdadm verify all</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">storage: State change failed: (-15) Need a connection to start verify or resync</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Command 'drbdsetup verify storage 0 0' terminated with exit code 11</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">storage role:Secondary</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> disk:UpToDate</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> 6787-dblapro-edss connection:Connecting</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> 6789-dblapro-edss role:Secondary</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> peer-disk:UpToDate</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">After "drbdadm up all" on the first (6787-dblapro-edss) and "drbdadm down all" on the second (6788-dblapro-edss) seems working correctly (with node 2 unavailability problems, but thats ok)</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">storage role:Secondary</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> disk:UpToDate</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> 6787-dblapro-edss role:Secondary</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> replication:VerifyS peer-disk:UpToDate done:100.00</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> 6788-dblapro-edss connection:Connecting</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">So, "connection-mesh" host position matters (not just for verify, may be).</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">#### 2. Verify strange verify behaviour ####</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">the same three nodes, start with fully OK status for all nodes.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">#storage role:Primary</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> disk:UpToDate</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> 6788-dblapro-edss role:Secondary</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> peer-disk:UpToDate</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> 6789-dblapro-edss role:Secondary</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> peer-disk:UpToDate</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Than, the magic:</p>
<ol style="margin-top: 0px; margin-bottom: 0px; margin-right: 0px; -qt-list-indent: 1;"><li style=" margin-top:12px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">On first (Primary role) node<br /># mount /dev/drbd/by-res/storage/0 /media/storage<br /># dd if=/dev/urandom of=/media/storage/1 bs=1M count=100<br />-rw-r--r-- 1 root root 104857600 Feb 17 18:17 /media/storage/1<br /></li>
<li style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Down Second node and mount as ordinary disk (we use external metadata)<span style=" font-family:'monospace';"><br /># drbdadm down all<br /># mount /dev/disk/by-label/storage /media/storage/<br /># dd if=/dev/urandom of=/media/storage/2 bs=1M count=100<br /># umount /media/storage/<br /># drbdadm up all<br /># drbdadm status - all UpToDate. Thats OK, no changes from DRBD metadata point of view.<br /></span></li>
<li style=" font-family:'monospace';" style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Second node - verify<br />wait till sync is done<br /></li></ol>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> # ... storage role:Secondary</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> disk:UpToDate</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> 6787-dblapro-edss role:Primary</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> replication:VerifyS peer-disk:UpToDate done:100.00</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> 6789-dblapro-edss role:Secondary</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> replication:VerifyS peer-disk:UpToDate done:100.00<br /></span></p>
<ol style="margin-top: 0px; margin-bottom: 0px; margin-right: 0px; -qt-list-indent: 1;"><li style=" font-family:'monospace';" style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Mount Secondary node disk (assuming to see only "./1" file)<br /># drbdadm down all<br /># mount /dev/disk/by-label/storage /media/storage/<br /># ls -l /media/storage/</li></ol>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> -rw-r--r-- 1 root root 104857600 Feb 17 18:21 1</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';"> -rw-r--r-- 1 root root 104857600 Feb 17 18:21 2<br /><br /></span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'monospace';">What really verify do? </span></p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">#### 3. No information about speeds, timing & bandwith ####</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">How long will replication lasts?</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">What is current bandwidth?</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">The questions are for initial sync. resync & verify.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">BTW, current implementation (for ex. done:80.94) is faulty sometimes.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Thanks for DRBD9 mesh topology! =)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-weight:600; font-style:italic; color:#000000;">---</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-weight:600; font-style:italic; color:#000000;">Best regards,</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0; background-color:#ffffff;"><span style=" font-style:italic; color:#000000;">Eugene Istomin</span></p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /><br /></p></body></html>