<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=SL link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Hello,<br><br>Thank you for taking the time to review my (rather lengthy) description.<br><br>> From your config below:<br>>> local-io-error "/usr/lib/drbd/notify-io-error.sh; drbdadm detach $DRBD_RESOURCE";<br><br>> VERY bad idea.<br>> Synchronously calling drbdadm from inside a synchronous handler will block<br>> (until that drbdadm will eventually timeout, 121 seconds later).<br>> And it is absolutely useless: DRBD will detach after local io error all by itself.<br><br>This is not really obvious from the documentation:<br><br>on-io-error handler<br><br>handler is taken, if the lower level device reports io-errors to the upper layers.<br>handler may be pass_on, call-local-io-error *<b>or</b>* detach.<br><br>If on-io-error is set to call-local-io-error, DRBD will also detach?<br><br>On a similar note, I’ve also been abusing the out-of-sync handler pretty much the same way: to issue a disconnect and reconnect on out of sync.<br>How would DRBD need to be configured to automatically do a disconnect/reconnect after verify has found out-of-sync blocks?<br><br>I don’t really remember where I got the idea to use the handlers, but a quick Google shows this:<br><br>“The reason of having out-of-sync handler is exactly tp provide possibility of automation.”<br><br>Perhaps it should be noted stronger in the documentation not to change DRBD states in the handlers?<br><br>> I would expect that you have a few<br>> "INFO: task drbd_w_... blocked for more than 120 seconds."<br>> and then call traces in the kernel log as well?<br><br>Not really, but I just found out my hung_task_warnings is set to 0. No idea why as I can’t remember setting this neither does a search in /etc find anything related to this…I’ll need to investigate more.<br><br>> There.<br>> This is actually something we may need to fix.<br>> But that's in fact a multiple error scenario including misconfiguration we did not think of yet.<br><br>There always is a multiple error scenario ;-)<br><br>> Because we did not anticipate that you would block the worker,<br>> and did call the handler before we notify the peer.<br>…<br>> *again* your blocking drbdadm detach from the local-io-error handler is the trigger.<br>> Don't do that. You also should even background the "notify", if you really insist on using it.<br><br>A local-io-error handler in the documentation example even has ‘halt –f’ in the sequence. How does that run before notifying the peer?<br>However no documentation examples show handlers being backgrounded… ?<br><br>> really? for 7 megabyte you want to pull ahead already?<br>> you basically won't have a consistent secondary, ever.<br><br>Another problem for me with DRBD. Without proxy, DRBD even in protocol A blocks when network buffer is full. If buffer is set to 10MB, pull ahead needs to be before buffer is full or DRBD blocks. Our write load is low enough not to trigger resync too often, but when it does I want it to pull ahead, not slow down to WAN link speed. Somehow I don’t feel comfortable with the idea of having hundreds of MBs of network buffer in the kernel… In this case, the secondary is more a standby to try to have the last possible data, so I don’t mind it being a bit out of sync from time to time.</span><span lang=EN-US style='color:#1F497D'><br><br></span><span lang=EN-US>Regards,<br>Saso Slavicic<span style='color:#1F497D'><br><br></span><o:p></o:p></span></p></div></body></html>