<div dir="ltr"><div dir="ltr"><div dir="ltr"><br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I do have the controller running on a dedicated Dabian VM ---- Linux linstor-ctlr 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64<br>
Its storage is mounted from a NFS share on a FreeNAS server available to all the nodes in my cluster. <br></blockquote><div><br></div><div>In my case, the controller VM is stored on DRBD itself, hence the need to make sure that its resource file is available at all times.<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I somewhat get why it deletes the res files, but the weird thing is that it will restore some of files and leave others missing after rebooting. I haven't seen a pattern of specific resources not being restored, and it's a different number of missing res files each time also.<br></blockquote><div><br></div><div>Never seen this behavior with LINSTOR (used to see this when I was using drbdmanage for managing DRBD though, sometimes). All resource files become available on all nodes as soon as linstor-satellite service is able to communicate properly with the controller from that node. <br></div><div>The only case where a resource file does not show up on a node, is when that resource is Diskless on that specific node. In that case, LINSTOR creates a resource file only when that resource becomes Primary on that specific node and then removes the resource file as soon as you relocate the resource to the other node, but I guess this is not your case ?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
With the last restart only 13 of the 27 resources were restored, even after nearly 2 hours. I decided to do another restart and after that it eventually restored all of the resources. ¯\_(ツ)_/¯<br></blockquote><div><br></div><div>It might worth checking the system logs and/or linstor specific logs to check what's causing this behavior. Assuming that the controller VM is up and running at all times, satellites should be able to export the res files on the nodes.<br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I have started maintaining a copy of the "/var/lib/linstor.d/" from each node so that I can simply copy them back in manually if the controller fails to do it.<br></blockquote><div><br></div><div>That is a workaround, but it defeats the logic of having LINSTOR for the management layer in the first place. Especially as your cluster scales up, this can easily become a tedious task..<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Using the "--keep-res" method and matching all resources might do the trick. </blockquote><div><br></div><div>No, I would not recommend this for preserving all resource files, but would recommend it only in the case that you need to preserve just one single resource file, that is the controller VM resource file, assuming that controller VM is stored within DRBD (that's not your case, you are using NFS for that). LINSTOR controller should be able to export the resource files properly on each node, if that's not happening, then something is not functioning properly somewhere or you have misconfigured something.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Does this only affect deleting during startup though? The controller can still tell a satellite to delete a resource even if the "--keep-res" is used, correct?<br></blockquote><div><br></div><div>Yes, this is used only during startup. The controller can still delete a resource when instructed to do so.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I didn't find direct documentation on overriding the service configuration, but I will try it in the next few days. <br></blockquote><div><br></div><div>Just to mention at this point that what worked for me was the following ..</div><div><br></div><div># On each node<br></div><div>$ systemctl edit linstor-satellite <br></div><div><br></div><div># Add the following content</div><div>[Unit]<br>After=drbd.service<br><br>[Service]<br>Type=oneshot # <--- very important<br>ExecStart=/usr/share/linstor-server/bin/Satellite --logs=/var/log/linstor-satellite --config-directory=/etc/linstor --keep-res vm-100-disk-1</div><div><br></div><div>Assuming that
vm-100-disk-1 is your controller VM resource file. But again, this is not your case ..</div><div><br></div><div>G.<br></div></div></div></div>