<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=ProgId content=Word.Document>
<meta name=Generator content="Microsoft Word 11">
<meta name=Originator content="Microsoft Word 11">
<link rel=File-List href="cid:filelist.xml@01C730DE.DAD62A10">
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="State"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="place"/>
<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:DoNotRelyOnCSS/>
</o:OfficeDocumentSettings>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:WordDocument>
<w:SpellingState>Clean</w:SpellingState>
<w:GrammarState>Clean</w:GrammarState>
<w:DocumentKind>DocumentEmail</w:DocumentKind>
<w:EnvelopeVis/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:UseWord2002TableStyleRules/>
</w:Compatibility>
<w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
</w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" LatentStyleCount="156">
</w:LatentStyles>
</xml><![endif]--><!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;
mso-font-alt:"?l?r ?????????????????????????";
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:1627421319 -2147483648 8 0 66047 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-parent:"";
margin:0in;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman";
mso-fareast-font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;
text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;
text-underline:single;}
span.EmailStyle17
{mso-style-type:personal-compose;
mso-style-noshow:yes;
mso-ansi-font-size:10.0pt;
mso-bidi-font-size:10.0pt;
font-family:Arial;
mso-ascii-font-family:Arial;
mso-hansi-font-family:Arial;
mso-bidi-font-family:Arial;
color:windowtext;}
span.SpellE
{mso-style-name:"";
mso-spl-e:yes;}
span.GramE
{mso-style-name:"";
mso-gram-e:yes;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;
mso-header-margin:.5in;
mso-footer-margin:.5in;
mso-paper-source:0;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman";
mso-ansi-language:#0400;
mso-fareast-language:#0400;
mso-bidi-language:#0400;}
</style>
<![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple style='tab-interval:.5in'>
<div class=Section1>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Under a <span class=SpellE>xen</span> migration test with
moderate I/O we easily get into this state where both nodes <span class=GramE>enters</span>:
<span class=SpellE>WFBitMapT</span>.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>We can recover from this by rebooting one node. <span
style='mso-spacerun:yes'> </span>This is broken by <span class=SpellE>som</span>
recent changes. <span style='mso-spacerun:yes'> </span>I can say that it worked
<o:p></o:p></span></font></p>
<p class=MsoNormal><span class=GramE><font size=2 face=Arial><span
style='font-size:10.0pt;font-family:Arial'>Before <span
style='mso-spacerun:yes'> </span>these</span></font></span><font size=2
face=Arial><span style='font-size:10.0pt;font-family:Arial'> changes:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=1 color=black face=Tahoma><span style='font-size:
9.0pt;font-family:Tahoma;color:black'>Broken with these changes:<br>
------------------------------------------------------------------------<br>
r8476 | <span class=SpellE>emontros</span> | 2007-01-03 11:01:31 -0500 (Wed, 03
Jan 2007) | 62 lines<br>
<br>
Merged revisions 8360-8456 via <span class=SpellE>svnmerge</span> from<br>
<a href="http://svn/vendor/drbd-8/trunk"><font color="#0000b4"><span
style='color:#0000B4'>http://svn/vendor/drbd-8/trunk</span></font></a><br>
<br>
........<br>
r8408 | <span class=SpellE>sgraham</span> | 2006-12-28 13:53:59 -0500 (Thu, 28
Dec 2006) | 38 lines<br>
<br>
[DRBD-8 @ 2648]<br>
With this patch I try to make DRBD to send more big packets,<br>
<span class=GramE>and</span> less small packets. <span class=SpellE>Somethimes</span>
I experienced that enabling<br>
Jumbo frames decreased <span class=SpellE>DRBD's</span> <span class=SpellE>preformance</span>.
<span class=SpellE>Altough</span>, I could<br>
<span class=GramE>not</span> reproduce this at time of writing the patch, I
managed to<br>
<span class=GramE>measure</span> the distribution of the packet's sizes. This
indicates<br>
<span class=GramE>that</span> the patch has the intended effect:<br>
<br>
Distribution of packet <span class=SpellE>sizen</span> when writing 512MB of
zeros:<br>
<br>
<span class=SpellE><span class=GramE>pkt</span></span>
old-code new-code<br>
<span class=GramE>sizes</span> send
<span class=SpellE>recv</span> send <span
class=SpellE>recv</span><br>
---------- ------------------------------<br>
1 to<span class=GramE> 450</span>: 42047
42259 38629 38784<br>
451 to<span class=GramE> 900</span>: 4057
4230 2023 2176<br>
901 to 1350: 112 2919
54 2289<br>
1351 to 1800: 6697 7439 6698
7368<br>
1801 to 2250: 11 43
4 8<br>
2251 to 2700: 1643 1684 1838
2307<br>
2701 to 3150: 5 43
3 18<br>
3151 to 3600: 2 70
0 51<br>
3601 to 4050: 95 218
2 251<br>
4051 to 4500: 3159 3296 1
58<br>
4501 to 4950: 45 270
6 176<br>
4951 to 5400: 75 717
60 453<br>
5401 to 5850: 867 901 1376
1383<br>
5851 to 6300: 683 696 245
251<br>
6301 to 6750: 129 214
44 47<br>
6751 to 7200: 24 39
266 274<br>
7201 to 7650: 101 118 256
263<br>
7651 to 8100: 30 1829
70 2109<br>
8101 to 8550: 36 48
49 56<br>
8551 to 9000+: 14971<span class=GramE> 53657</span>
15755 54992<br>
<br>
-Phil<br>
<br>
Original author: <span class=SpellE><span class=GramE>phil</span></span><br>
Date: 2006-12-28 12:07:12.682853+00:00<br>
........<br>
r8409 | <span class=SpellE>sgraham</span> | 2006-12-28 13:54:04 -0500 (Thu, 28
Dec 2006) | 5 lines<br>
<br>
[DRBD-8 @ 2649]<br>
Make DRBD compile even on SLES9 (Kernel 2.6.5)<br>
<br>
Original author: <span class=SpellE><span class=GramE>phil</span></span><br>
Date: 2006-12-28 15:54:28.853546+00:00<br>
........<br>
r8455 | <span class=SpellE>sgraham</span> | 2007-01-02 09:04:11 -0500 (Tue, 02
Jan 2007) | 5 lines<br>
<br>
[DRBD-8 @ 2650]<br>
Fixed a few error strings...<br>
<br>
Original author: <span class=SpellE><span class=GramE>phil</span></span><br>
Date: 2007-01-02 13:47:28.435731+00:00<br>
........<br>
<br>
------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=1 color=black face=Tahoma><span style='font-size:
9.0pt;font-family:Tahoma;color:black'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=1 color=black face=Tahoma><span style='font-size:
9.0pt;font-family:Tahoma;color:black'>Here the state traces:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=1 color=black face=Tahoma><span style='font-size:
9.0pt;font-family:Tahoma;color:black'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=1 color=black face=Tahoma><span style='font-size:
9.0pt;font-family:Tahoma;color:black'>Here is the log for <span class=SpellE>JErry</span>:<br>
drbd2: <span class=GramE>disk(</span> Diskless -> Attaching )<br>
drbd2: Found 6 transactions (267 active extents) in activity log.<br>
drbd2: <span class=SpellE>max_segment_size</span> <span class=GramE>( =</span>
BIO size ) = 32768<br>
drbd2: <span class=SpellE>drbd_bm_resize</span> called with capacity ==
30735368<br>
drbd2: <span class=SpellE>resync</span> bitmap: bits=3841921 words=120062<br>
drbd2: size = 14 GB (15367684 KB)<br>
drbd2: reading of bitmap took 18 jiffies<br>
drbd2: recounting of set bits took additional 1 <span class=GramE>jiffies</span><br>
drbd2: 0 KB marked out-of-sync by on disk bit-map.<br>
drbd2: <span class=GramE>disk(</span> Attaching -> <span class=SpellE>UpToDate</span>
)<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>StandAlone</span> -> Unconnected )<br>
drbd2: receiver (re)started<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> Unconnected -> <span class=SpellE>WFConnection</span> )<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>WFConnection</span> -> <span
class=SpellE>WFReportParams</span> )<br>
drbd2: Handshake successful: DRBD Network Protocol version 85<br>
drbd2: <span class=SpellE>drbd_sync_handshake</span>:<br>
drbd2: self 1999C2A90658DA42:0000000000000000:30A5702D832E1702:0000000000000004<br>
drbd2: peer 1999C2A90658DA43:0000000000000000:30A5702D832E1702:0000000000000004<br>
drbd2: <span class=SpellE>uuid_<span class=GramE>compare</span></span><span
class=GramE>(</span>)=-1<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span><span class=SpellE>History_start</span>] now
1999C2A90658DA42<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Current] now 0000000000000000<br>
drbd2: <span class=GramE>peer(</span> Unknown -> Secondary ) <span
class=SpellE>conn</span>( <span class=SpellE>WFReportParams</span> -> <span
class=SpellE>WFBitMapT</span> ) <span class=SpellE>pdsk</span>( <span
class=SpellE>DUnknown</span> -> <span class=SpellE>UpToDate</span> )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE>aftr_<span class=GramE>isp</span></span><span
class=GramE>(</span> 0 -> 1 )<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>WFBitMapT</span> -> <span
class=SpellE>WFSyncUUID</span> )<br>
drbd2: <span class=SpellE>peer_<span class=GramE>isp</span></span><span
class=GramE>(</span> 0 -> 1 )<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Current] now FC968953A6746EEC<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Bitmap] now 0000000000000000<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>WFSyncUUID</span> -> <span
class=SpellE>PausedSyncT</span> ) disk( <span class=SpellE>UpToDate</span>
-> Inconsistent )<br>
drbd2: Began <span class=SpellE>resync</span> as <span class=SpellE>PausedSyncT</span>
(will sync 1052672 KB [263168 bits set]).<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=GramE>peer(</span> Secondary -> Primary )<br>
drbd2: <span class=SpellE>peer_<span class=GramE>isp</span></span><span
class=GramE>(</span> 1 -> 0 )<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>PausedSyncT</span> -> <span
class=SpellE>SyncTarget</span> ) <span class=SpellE>aftr_isp</span>( 1 -> 0
)<br>
drbd2: <span class=SpellE>Syncer</span> continues.<br>
drbd2: <span class=SpellE>Resync</span> done (total 225 sec; paused 61 sec;
6416 K/sec)<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Bitmap] now 0000000000000000<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span><span class=SpellE>History_start</span>] now
30A5702D832E1702<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span><span class=SpellE>History_end</span>] now 0000000000000004<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Bitmap] now FC968953A6746EEC<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Current] now 1999C2A90658DA42<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span><span class=SpellE>History_start</span>] now
FC968953A6746EEC<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Bitmap] now 0000000000000000<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>SyncTarget</span> -> Connected )
disk( Inconsistent -> <span class=SpellE>UpToDate</span> )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE>drbd_open</span>: Allowing open in Secondary state<br>
drbd2: <span class=SpellE>drbd_sync_handshake</span>:<br>
drbd2: self 1999C2A90658DA42:0000000000000000:FC968953A6746EEC:30A5702D832E1702<br>
drbd2: peer 1999C2A90658DA42:0000000000000000:FC968953A6746EEC:30A5702D832E1702<br>
drbd2: <span class=SpellE>uuid_<span class=GramE>compare</span></span><span
class=GramE>(</span>)=-1<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span><span class=SpellE>History_start</span>] now
1999C2A90658DA42<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Current] now 0000000000000000<br>
drbd2: <span class=GramE>peer(</span> Primary -> Secondary ) <span
class=SpellE>conn</span>( Connected -> <span class=SpellE>WFBitMapT</span> )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=GramE>role(</span> Secondary -> Primary )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
<br>
<br>
On <span class=SpellE>ben</span> the other node we have:<br>
<br>
drbd2: <span class=GramE>disk(</span> Diskless -> Attaching )<br>
drbd2: Found 6 transactions (324 active extents) in activity log.<br>
drbd2: <span class=SpellE>max_segment_size</span> <span class=GramE>( =</span>
BIO size ) = 32768<br>
drbd2: <span class=SpellE>drbd_bm_resize</span> called with capacity == 30735368<br>
drbd2: <span class=SpellE>resync</span> bitmap: bits=3841921 words=120062<br>
drbd2: size = 14 GB (15367684 KB)<br>
drbd2: reading of bitmap took 26 jiffies<br>
drbd2: recounting of set bits took additional 1 <span class=GramE>jiffies</span><br>
drbd2: 0 KB marked out-of-sync by on disk bit-map.<br>
drbd2: Marked additional 1028 MB as out-of-sync based on <st1:State w:st="on"><st1:place
w:st="on">AL</st1:place></st1:State>.<br>
drbd2: <span class=GramE>disk(</span> Attaching -> <span class=SpellE>UpToDate</span>
)<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>StandAlone</span> -> Unconnected )<br>
drbd2: receiver (re)started<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> Unconnected -> <span class=SpellE>WFConnection</span> )<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>WFConnection</span> -> <span
class=SpellE>WFReportParams</span> )<br>
drbd2: Handshake successful: DRBD Network Protocol version 85<br>
drbd2: <span class=SpellE>drbd_sync_handshake</span>:<br>
drbd2: self 1999C2A90658DA43:0000000000000000:30A5702D832E1702:0000000000000004<br>
drbd2: peer 1999C2A90658DA42:0000000000000000:30A5702D832E1702:0000000000000004<br>
drbd2: <span class=SpellE>uuid_<span class=GramE>compare</span></span><span
class=GramE>(</span>)=1<br>
drbd2: <span class=GramE>peer(</span> Unknown -> Secondary ) <span
class=SpellE>conn</span>( <span class=SpellE>WFReportParams</span> -> <span
class=SpellE>WFBitMapS</span> ) <span class=SpellE>pdsk</span>( <span
class=SpellE>DUnknown</span> -> <span class=SpellE>UpToDate</span> )<br>
drbd2: <span class=SpellE>aftr_<span class=GramE>isp</span></span><span
class=GramE>(</span> 0 -> 1 )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE>peer_<span class=GramE>isp</span></span><span
class=GramE>(</span> 0 -> 1 )<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Bitmap] now FC968953A6746EEC<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>WFBitMapS</span> -> <span
class=SpellE>PausedSyncS</span> ) <span class=SpellE>pdsk</span>( <span
class=SpellE>UpToDate</span> -> Inconsistent )<br>
drbd2: Began <span class=SpellE>resync</span> as <span class=SpellE>PausedSyncS</span>
(will sync 1052672 KB [263168 bits set]).<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=GramE>role(</span> Secondary -> Primary )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE>aftr_<span class=GramE>isp</span></span><span
class=GramE>(</span> 1 -> 0 )<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>PausedSyncS</span> -> <span
class=SpellE>SyncSource</span> ) <span class=SpellE>peer_isp</span>( 1 -> 0
)<br>
drbd2: <span class=SpellE>Syncer</span> continues.<br>
drbd2: <span class=SpellE>Resync</span> done (total 225 sec; paused 61 sec;
6416 K/sec)<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span><span class=SpellE>History_start</span>] now
FC968953A6746EEC<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Bitmap] now 0000000000000000<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> <span class=SpellE>SyncSource</span> -> Connected ) <span
class=SpellE>pdsk</span>( Inconsistent -> <span class=SpellE>UpToDate</span>
)<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=GramE>role(</span> Primary -> Secondary )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE>drbd_sync_handshake</span>:<br>
drbd2: self 1999C2A90658DA43:0000000000000000:FC968953A6746EEC:30A5702D832E1702<br>
drbd2: peer 0000000000000000:0000000000000000:1999C2A90658DA42:FC968953A6746EEC<br>
drbd2: <span class=SpellE>uuid_<span class=GramE>compare</span></span><span
class=GramE>(</span>)=-2<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: writing of bitmap took 59 jiffies<br>
drbd2: 14 GB marked out-of-sync by on disk bit-map.<br>
drbd2: 15367684 KB now marked out-of-sync by on disk bit-map.<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span><span class=SpellE>History_start</span>] now
1999C2A90658DA43<br>
drbd2: <span class=SpellE><span class=GramE>uuid</span></span><span
class=GramE>[</span>Current] now 0000000000000000<br>
drbd2: <span class=SpellE><span class=GramE>conn</span></span><span
class=GramE>(</span> Connected -> <span class=SpellE>WFBitMapT</span> )<br>
drbd2: Writing <span class=GramE>meta</span> data super block now.<br>
drbd2: <span class=GramE>peer(</span> Secondary -> Primary )<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
</div>
</body>
</html>