<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin-top:0in;
margin-right:0in;
margin-bottom:10.0pt;
margin-left:0in;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:10.0pt;
margin-left:.5in;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:10.0pt;
margin-left:.5in;
mso-add-space:auto;
line-height:115%;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:62990196;
mso-list-type:hybrid;
mso-list-template-ids:1752710924 67698705 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Hello,<o:p></o:p></p><p class=MsoNormal> I use DRBD 8.4.4 on SLES 11 SP3 and I’m having difficulties setting up a 3 node DR configuration. I have a 2-node pacemaker cluster with another stacked resource to the DR site. My DR node is not part of the cluster. My DRBD config is as follows:<o:p></o:p></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>resource pg {<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> on usrv-ffm12 {<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> device /dev/drbd0;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> disk /dev/sda3;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> address 172.29.xx.xx:7788;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> meta-disk internal;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> }<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> on usrv-ffm11 {<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> device /dev/drbd0;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> disk /dev/sda3;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> address 172.29.xx.xx:7788;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> meta-disk internal;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>}<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>}<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>resource pg-dr {<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> net {<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> protocol A; # async replication<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> }<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> stacked-on-top-of pg {<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> device /dev/drbd10;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> address 192.61.xx.xx:7788; # floating IP of the primary cluster<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> }<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'># this is the dr site<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> on usrv-ffm2 {<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> device /dev/drbd10;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> disk /dev/sda3;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> address 192.61.xx.xx:7788; # ip of dr site<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> meta-disk internal;<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'> }<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>}<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>I bring up all three nodes and both the stacked and regular resource all get to UpToDate. When I demote the 2-node cluster master and make the DR site primary the fsck of the file system shows data is bad. The fsck on the 2 node cluster shows the data is clean.<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>usrv-ffm2:~ # fsck -n /dev/drbd10<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>fsck from util-linux 2.19.1<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>e2fsck 1.41.9 (22-Aug-2009)<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>fsck.ext3: Group descriptors look bad... trying backup blocks...<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>fsck.ext3: Bad magic number in super-block when using the backup blocks<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>fsck.ext3: going back to original superblock<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>fsck.ext3: Device or resource busy while trying to open /dev/drbd10<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>Filesystem mounted or opened exclusively by another program?<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>If I let fsck run, the journal inode 8 is bad.<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>My setup<o:p></o:p></span></p><p class=MsoListParagraphCxSpFirst style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>1)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Create the meta base resource pg on the first 2 nodes<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>2)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Connect these two nodes with drbdadm up pg<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-size:9.0pt;line-height:115%;font-family:"Courier New"'><span style='mso-list:Ignore'>3)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Clear the bitmap so data is UpToDate for first 2 nodes: </span><span style='font-size:9.0pt;line-height:115%;font-family:"Courier New"'><o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto'><b><span style='font-size:9.0pt;line-height:115%;font-family:"Courier New"'>drbdadm -- --clear-bitmap new-current-uuid pg</span></b><span style='font-size:9.0pt;line-height:115%;font-family:"Courier New"'><o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>4)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Make one primary – (called master here)<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>5)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Calculate the size of the meta needed for the base resource and stacked resource. Create an ext3 filesystem with the size of disk partition – size of the 2 meta areas.<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto'><span style='font-family:"Courier New"'>mkfs.ext3 -b 4096 /dev/drbd0 $size<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>6)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Make sure the .res files are on the DR system<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>7)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Create the meta on the master system for pg-dr (stacked res)<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>8)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>drbdadm –S up pg-dr on the master system<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>9)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>On DR site create the meta for pg-dr<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>10)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>On DR site up the pg-dr resource so it can connect. It’s now connected, both are secondary <o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>11)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>On the master site, clear the bitmap for pg-dr to get data in UpToDate<o:p></o:p></span></p><p class=MsoListParagraphCxSpMiddle style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>12)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>On master site make it primary for pg-dr.<o:p></o:p></span></p><p class=MsoListParagraphCxSpLast style='margin-bottom:0in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='font-family:"Courier New"'><span style='mso-list:Ignore'>13)<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='font-family:"Courier New"'>Mount the file system to /dev/drbd10 (stacked device) on the Master node<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>I use the clear-bitmap new-current-uuid command since I start with no data anywhere. This allows me to get to UpToDate without transporting data.<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>I’m a bit confused over the creation of the drbd meta area. Is it done on all 3 nodes for both the base and the stacked resource? Does it matter if it’s done before or after I create the file system (I don’t think it matters since I believe the meta is written at the end of the device).<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>Thanks for any help<o:p></o:p></span></p><p class=MsoNormal style='margin-bottom:0in;margin-bottom:.0001pt'><span style='font-family:"Courier New"'>Diane Schaefer<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p></div></body></html>