[DRBD-user] Missing Linstor/DRBD resource files after reboot

[Gábor Hernádi]

On Thu, Mar 28, 2019 at 7:07 AM Nicholas Morton <kmorton at cancinc.com> wrote:

> I am using Linstor to manage DRBD volumes on a 4 node Proxmox cluster and I
> am having an issue during node reboots.
> After a reboot the node is usually missing some of the .res files in
> /var/lib/linstor.d
> For whatever reason they aren't being repopulated correctly.
> I have a few times manually recreated the .res files but with so many
> resources this is arduous.
>
> Does Linstor have an export function to write out the .res files?
> I remember drbdmanage having a feature that did that, it had a similar
> problem.
>
> It could be something simple I missed, but I haven't found it in the hours
> I've put in.
> I have a 7 storage pool setup, 3 NVMe, 3 SSD, 1 HDD, with 27 drbd resources
> spread across 22 VMs.
>
> All nodes have the same package versions.
> ----------------
> Kernel Version: Linux 4.15.18-11-pve #1 SMP PVE 4.15.18-34 (Mon, 25 Feb
> 2019
> 14:51:06 +0100)
> PVE Manager Version: pve-manager/5.3-11/d4907f84
>
> cat /proc/drbd
> version: 9.0.16-1 (api:2/proto:86-114)
> GIT-hash: ab9777dfeaf9d619acc9a5201bfcae8103e9529c build by root at vmhost1,
> 2019-03-13 15:28:08
> Transports (api:16): tcp (9.0.16-1)
>
> drbdadm
> Version: 9.8.0 (api:2)
> GIT-hash: c30216b49330216bf8a567b7727da6e24f099f08 build by
> @buildsystem.linbit, 2019-01-16 12:50:14
>
> linstor 0.7.3; GIT-hash: b99bcbba16e98b0c55d695795deebfafeef93636
>
> LINSTOR ==> c version
> linstor controller 0.7.5; GIT-hash:
> d74305b420fdc878182afa162378a317e6a4a3b9
>
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>


Hello,

Linstor basically only deletes its own .res files (within
/var/lib/linstor.d/*.res) when the satellite starts (that is to make sure
to clean up possibly already removed / lost resources). Usually that is not
an issue, because as soon as the controller connects to the satellite, the
satellite receives all information needed to regenerate the corresponding
.res files.

If your .res files are (still) missing, then I guess the satellite was able
to start up (as apparently it was able to delete the .res files) but the
controller failed to connect and therefor the satellite did not receive the
information needed for the regeneration. That could be because of a version
missmatch, or something else. Check the `linstor node list` or something
"new" in `linstor err list` where error reports can be viewed with `linstor
err show <error-report-id>`

If you need those .res files for whatever reason before the controller is
able to connect the the satellite, you will have to modify the service file
starting the satellite and add a `--keep-res "your regex here"`. Every full
path like "/var/lib/linstor.d/resource_name.res" that matches your regular
expression will not be deleted on startup.

Hope that helps.
Regards,
Gabor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20190328/44f6ca3b/attachment.htm>