<div dir="ltr"><br><div>hi, </div><div>i'm hoping someone here can help me, since no-one seems to be using the ocfs2 mailing-list.</div><div><br></div><div>i'm new to drbd, heartbeat and ocfs2, but i have managed to get partitions created and exported for iscsi and nfs, and they are working fine. (iscsi performance is great, nfs is slow), but have been having a nightmare with ocfs2</div>
<div>the iscsi and nfs partitions are primary/secondary. the ocfs2 partitions are dual-primary. i was having problems getting both machines to mount /dev/drbd0 simultaneously in a reliable manner on virtualized servers, i have now tried it using physical servers, and it's now reliable although concurrent writing to a file is slow. </div>
<div>i've been using ubuntu 12.04, and sticking with the packages installed by default using apt-get, i've now tried it with 13.10 beta2 since performance is supposed to be greatly improved in drbd 8.4.3. again sticking with what's installed using apt-get.</div>
<div>i want to export the ocfs2 cluster, so that it's used for all the website data for up to 20 servers (ubuntu 12.04), but i can't seem to get it working. all the website stats/logging goes into the same files, so it has to be able to write concurrently. </div>
<div>can i export it using heartbeat and nfs? does heartbeat work with dual-primaries? does exporting an ocfs2 partition over nfs still allow concurrent writes to a single file?</div><div>everything seems ok as far as drbd and ocfs2 goes when accessing it locally on these 2 servers, but i have no idea how to get it working for the webservers,i don't want to create a 22 node cluster due to the bandwidth overheads associated with that. </div>
<div>what do i need to put in the configuration files to get it working?</div><div><br></div><div>here's what i currently have on a system i'm just trying ocfs2 on for simplicity:</div><div><br></div><div>dual-homed servers, nas1 : 192.168.0.91 and 10.10.10.1 </div>
<div> nas2 : 192.168.0.92 and 10.10.10.2</div><div>so as i understand it, all the synchronisation/replication for the config as below, should occur on the 10.10.10.* subnet, (direct connection with crossover cable, 1Gb network, mtu 9000) </div>
<div>webservers should connect using the 192.168.0.* subnet</div><div><br></div><div><br></div><div>/etc/drbd.conf:</div><div><br></div><div><div># You can find an example in /usr/share/doc/drbd.../drbd.conf.example</div>
<div><br></div><div>include "drbd.d/global_common.conf";</div><div>include "drbd.d/*.res";</div><div><br></div><div>resource ocfs2.config {</div><div><span class="" style="white-space:pre"> </span>protocol C;</div>
<div><span class="" style="white-space:pre"> </span>handlers {</div><div><span class="" style="white-space:pre"> </span>pri-on-incon-degr "echo o > /proc/sysreq-trigger ; halt -f";</div><div><span class="" style="white-space:pre"> </span>pri-lost-after-sb "echo o > /proc/sysreq-trigger ; halt -f";</div>
<div><span class="" style="white-space:pre"> </span>local-io-error "echo o > /proc/sysreq-trigger ; halt -f";</div><div><span class="" style="white-space:pre"> </span>outdate-peer "/usr/lib/heartbeat/drbd-peer-outdater -t 5";</div>
<div><span class="" style="white-space:pre"> </span>}</div><div><br></div><div><span class="" style="white-space:pre"> </span>startup {</div><div><span class="" style="white-space:pre"> </span>wfc-timeout 120;</div><div>
<span class="" style="white-space:pre"> </span>degr-wfc-timeout 120;</div><div><span class="" style="white-space:pre"> </span>become-primary-on both;</div><div><span class="" style="white-space:pre"> </span>}</div><div>
<br></div><div><span class="" style="white-space:pre"> </span>disk {</div><div><span class="" style="white-space:pre"> </span>on-io-error detach;</div><div><span class="" style="white-space:pre"> </span>resync-rate 100M;</div>
<div><span class="" style="white-space:pre"> </span>}</div><div><br></div><div><span class="" style="white-space:pre"> </span>net {</div><div><span class="" style="white-space:pre"> </span>allow-two-primaries;</div><div>
<span class="" style="white-space:pre"> </span>cram-hmac-alg sha1;</div><div><span class="" style="white-space:pre"> </span>shared-secret "password";</div><div><span class="" style="white-space:pre"> </span>after-sb-0pri discard-zero-changes;</div>
<div><span class="" style="white-space:pre"> </span>after-sb-1pri discard-secondary;</div><div><span class="" style="white-space:pre"> </span>after-sb-2pri disconnect;</div><div><span class="" style="white-space:pre"> </span>rr-conflict disconnect;</div>
<div><span class="" style="white-space:pre"> </span>}</div><div><br></div><div><br></div><div><span class="" style="white-space:pre"> </span>on nas1 {</div><div><span class="" style="white-space:pre"> </span>device /dev/drbd0;</div>
<div><span class="" style="white-space:pre"> </span>disk /dev/sda4;</div><div><span class="" style="white-space:pre"> </span>address <a href="http://10.10.10.1:7788">10.10.10.1:7788</a>;</div><div><span class="" style="white-space:pre"> </span>meta-disk /dev/sda3[0];</div>
<div><span class="" style="white-space:pre"> </span>}</div><div><br></div><div><span class="" style="white-space:pre"> </span>on nas2 {</div><div><span class="" style="white-space:pre"> </span>device /dev/drbd0;</div><div>
<span class="" style="white-space:pre"> </span>disk /dev/sda4;</div><div><span class="" style="white-space:pre"> </span>address <a href="http://10.10.10.2:7788">10.10.10.2:7788</a>;</div><div><span class="" style="white-space:pre"> </span>meta-disk /dev/sda3[0];</div>
<div><span class="" style="white-space:pre"> </span>}</div><div>}</div><div><br></div></div><div><br></div><div><br></div><div>/etc/ocfs2/cluster.conf:</div><div><div>cluster:</div><div><span class="" style="white-space:pre"> </span>node_count = 2</div>
<div><span class="" style="white-space:pre"> </span>name = datacluster</div><div>node:</div><div><span class="" style="white-space:pre"> </span>ip_port = 7777</div><div><span class="" style="white-space:pre"> </span>ip_address = 10.10.10.1</div>
<div><span class="" style="white-space:pre"> </span>number = 1</div><div><span class="" style="white-space:pre"> </span>name = nas1</div><div><span class="" style="white-space:pre"> </span>cluster = datacluster</div><div>
node:</div><div><span class="" style="white-space:pre"> </span>ip_port = 7777</div><div><span class="" style="white-space:pre"> </span>ip_address = 10.10.10.2</div><div><span class="" style="white-space:pre"> </span>number = 2</div>
<div><span class="" style="white-space:pre"> </span>name = nas2</div><div><span class="" style="white-space:pre"> </span>cluster = datacluster</div><div><br></div></div><div><br></div><div>formatted using: mkfs.ocfs2 -T mail -L "datacluster" /dev/drbd0</div>
<div>mounted using mount -t ocfs2 /dev/drbd0 /test</div><div><br></div><div><br></div><div>/etc/exports:</div><div><div>/test/exportedfs <a href="http://192.168.0.95/24(rw,no_subtree_check,sync,no_root_squash,no_all_squash)">192.168.0.95/24(rw,no_subtree_check,sync,no_root_squash,no_all_squash)</a></div>
</div><div><br></div><div>/etc/heartbeat/haresources:</div><div>nas1 IPaddr::<a href="http://192.168.0.95/24/em1">192.168.0.95/24/em1</a> drbddisk::ocfs2.config Filesystem::/dev/drbd0::/test::ocfs2 nfs-kernel-server<br></div>
<div><br></div><div><br></div><div>i can find articles that say ocfs2 can be exported over nfs, but i can't find anything that actually shows how,</div><div>any help or advice would be greatly appreciated, and would alleviate weeks of head-bashing :(</div>
<div><br></div><div><br></div><div>thanks</div><div>lee.</div><div><br></div></div>