<div dir="ltr">In addition, as long as you're using proxmox, it would be way easier to setup the native drbd9 plugin for proxmox instead of using the iscsi method. In this case both drbd and proxmox should be hosted on the same servers (hyper-converged setup). Each vm will reside in a separate drbd9 resource/volume and you can control the redundancy level as well.</div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 3, 2017 at 1:04 PM, Adam Goryachev <span dir="ltr"><<a href="mailto:mailinglists@websitemanagers.com.au" target="_blank">mailinglists@websitemanagers.com.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Note, all the below relates to my uses of DRBD 8.4 in production. I'm assuming most of it will be equally applicable to DRBD9.<span class=""><br>
<br>
<br>
On 3/10/17 19:52, Gandalf Corvotempesta wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Just trying to figure out if drbd9 can do the job.<br>
<br>
Requirement: a scale-out storage for VMs image hosting (and other<br>
services, but they would be made by creating, in example, an NFS VM on<br>
top of DRBD)<br>
<br>
Let's assume a 3-nodes DRBDv9 cluster.<br>
I would like to share this cluster by using iSCSI (or better protocol, if any).<br>
Multiple proxmox nodes sharing this drbd cluster.<br>
<br>
Probably, one drbd resource is created for each VM.<br>
<br>
Now, some question:<br>
<br>
how can I ensure that my iSCSI target is redundant across all nodes in<br>
the cluster ?<br>
</blockquote></span>
What do you mean by redundant? You only have a single iscsi server, this is the current DRBD primary server. You would use heartbeat or similar to automatically stop the iscsi server, change primary to a different server, and then start iscsi server on that machine. Your iscsi clients will get no response during this time, which looks like a disk stall. Note, it's important to ensure you do this in the correct order:<br>
1) Remove IP address (or firewall so that no response is sent back, no ICMP port closed message, no TCP packets, nothing at all).<br>
2) Stop iscsi service<br>
3) Change to secondary<br>
4) Change other server to primary<br>
5) Start iscsi service on new primary server<br>
6) Add IP address, or fix firewall to allow traffic in/out.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
When I have to add a fourth or fifth node to drbd cluster, should I<br>
replicate the iscsi target configuration on both ?<br>
</blockquote></span>
Yes, you must ensure the iscsi config is identical on every server which could potentially become primary.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Will the drbd resources automatically rebalanced across the new nodes ?<br>
</blockquote></span>
I'm not sure, I suspect you are considering to make one of your DRBD nodes primary for some of the resources, and another primary for a different group of resources, and then somehow your peers will work out which primary to talk to for their iscsi service. This could be possible (thinking, definitely you will want to test this first).<br>
<br>
Consider if each DRBD resource will have a dedicated IP address. You will need to somehow dynamically configure the iscsi service (it is possible with iet and messing around in /proc) to listen on this extra IP, and serve this extra resource. Doing this individually for each resource (ie, the above 6 steps would be repeated once for each resource). However, I wonder if this would get you any significant benefit? All data will still need to be written to all servers, though I suppose reads will be better balanced than an all on one primary system.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Should I change something in the iscsi/proxmox configuration after the<br>
rebalance or is it transparent?<br>
</blockquote></span>
I'm thinking yes... I suspect your heartbeat layer will need to manage these changes for you.<span class=""><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Any pitfalls or drawbacks ?<br>
</blockquote></span>
Lots.... make sure you test.... a lot... including any and all failure modes you can think of, as well as a complete failure (all nodes die and recover).<br>
<br>
Hopefully someone with more hands on experience with DRBD9 can comment further....<br>
<br>
Regards,<br>
Adam<div class="HOEnZb"><div class="h5"><br>
______________________________<wbr>_________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" rel="noreferrer" target="_blank">http://lists.linbit.com/mailma<wbr>n/listinfo/drbd-user</a><br>
</div></div></blockquote></div><br></div>