<html><head><title>Re: [DRBD-user] linstor-satellite does not bring up drbd resource under proxmox 6</title>
</head>
<body>
<span style=" font-family:'Courier New'; font-size: 10pt;">Thank you, man! Yeah, I've restarted Proxmox after file modification. Your new code looks promisingly, I'll give it a try.<br>
<br>
PS I'm replying to the list, so the others could you the workaround.<br>
<br>
</span><table>
<tr>
<td width=3 bgcolor= #0000ff><br>
</td>
<td><span style=" font-family:'calibri'; font-size: 11pt;">Sorry, mistake on line :<br>
system ('drbdadm up vm-101-disk-1');<br>
<br>
must be :<br>
system ('drbdadm up vm-'.$controller_vm.'-disk-1');<br>
<br>
<br>
<b>De :</b> drbd-user-bounces@lists.linbit.com <drbd-user-bounces@lists.linbit.com> <b>De la part de</b> Alex Kolesnik<br>
<b>Envoyé :</b> vendredi 16 août 2019 16:14<br>
<b>À :</b> drbd-user@lists.linbit.com<br>
<b>Objet :</b> Re: [DRBD-user] linstor-satellite does not bring up drbd resource under proxmox 6<br>
<br>
<span style=" font-family:'courier new'; font-size: 10pt;">Hello Brice,<br>
<br>
Thanks for your suggestion! Unfortunately nothing has changed after adding your code to that .pm file on both nodes. I also added some debugging and didn't see that activate_volume function was called.<br>
<br>
Will anyone from Linbit comment this, please?<br>
</span></span><table>
<tr>
<td width=4 bgcolor= #0000ff><br>
</td>
<td><span style=" font-family:'arial'; font-size: 11pt; color: #4f4c49; background-color: #f6f6f6;">Hi Alex,<br>
<br>
I used it too, not with keep-res option (didn't know it), I 'll try it :)<br>
<br>
And yes it is expected. I read the proxmox plugin code and change somethings to work with one vm which contains linstor-controller. The problem is your device is not writable.<br>
<br>
To do it writable, I had to change the proxmox plugin code.<br>
In the file </span><span style=" font-family:'courier new'; font-size: 10pt;">/usr/share/perl5/PVE/Storage/Custom/LINSTORPlugin.pm, you have to change the code with :<br>
<br>
sub activate_volume {<br>
my ( $class, $storeid, $scfg, $volname, $snapname, $cache ) = @_;<br>
<br>
die "Snapshot not implemented on DRBD\n" if $snapname;<br>
<br>
###THE IMPORTANT PART###<br>
if( ignore_volume( $scfg, $volname ) ) {<br>
print "It is the controller vm volume, so set blockdev r/w\n";<br>
system ('blockdev --setrw ' . get_dev_path $volname);<br>
return undef;<br>
}<br>
###########################<br>
return undef if ignore_volume( $scfg, $volname );<br>
<span style=" font-family:'arial'; font-size: 11pt; color: #4f4c49; background-color: #f6f6f6;"> ....<br>
<br>
With it, when your proxmox start the vm it will call the activate_volume function and most importantly the blockdev system call. It will put your device in rw state, and proxmox will start your vm.<br>
<br>
Give it a try. I wrote some code to get vm controller starting after a complete cluster outage, and it is working like a charm :)<br>
When I'll finish it, I'll publish it.<br>
<br>
<br>
Best regards<br>
Brice<br>
<br>
<br>
<br>
</span><span style=" font-family:'arial'; font-size: 11pt; color: #4f4c49;">------------------------<br>
<span style=" background-color: #f6f6f6;">Hello, <br>
<br>
To be honest, I'm not sure if that's a linstor-satellite issue. Please, correct <br>
me. <br>
<br>
There're two proxmox nodes pve1 and pve2. DRBD resource has been configured: <br>
root@linstor-controller:~# linstor resource list <br>
??????????????????????????????????????????????????? <br>
? ResourceName ? Node ? Port ? Usage ? State ? <br>
??????????????????????????????????????????????????? <br>
? vm-100-disk-0 ? pve1 ? 7000 ? InUse ? UpToDate ? <br>
? vm-100-disk-0 ? pve2 ? 7000 ? Unused ? UpToDate ? <br>
??????????????????????????????????????????????????? <br>
<br>
The linstor-controller itself is stored onto that DRBD resource: <br>
root@pve1:~# pct config 100 <br>
arch: amd64 <br>
cores: 2 <br>
hostname: linstor-controller <br>
memory: 512 <br>
net0: name=eth0,bridge=vmbr10,gw=10.1.1.1,hwaddr=26:81:53:50:FB:48,ip=</span></span></span><a style=" color: #4f4c49; font-family:'arial'; font-size: 11pt; background-color: #f6f6f6;" href="http://10.1.1.254/24,type=veth">10.1.1.254/24,type=veth</a> <br>
<span style=" font-family:'arial'; font-size: 11pt; color: #4f4c49; background-color: #f6f6f6;">onboot: 1 <br>
ostype: debian <br>
rootfs: drbdpool:vm-100-disk-0,mountoptions=noatime,size=2G <br>
swap: 512 <br>
unprivileged: 1 <br>
<br>
Linstor-satellite is started on both pve nodes within this systemd service: <br>
<br>
root@pve1:~# systemctl cat linstor-satellite.service <br>
<span style=" color: #660066;"># /lib/systemd/system/linstor-satellite.service <br>
<span style=" color: #4f4c49;">[Unit] <br>
Description=LINSTOR Satellite Service <br>
Wants=network-online.target <br>
After=network-online.target <br>
<br>
[Service] <br>
Type=simple <br>
ExecStart=/usr/share/linstor-server/bin/Satellite --logs=/var/log/linstor-satellite --config-directory=/etc/linstor <br>
<span style=" color: #660066;"># if killed by signal 143 -> SIGTERM, 129 -> SIGHUP <br>
<span style=" color: #4f4c49;">SuccessExitStatus=0 143 129 <br>
User=root <br>
PrivateTmp=yes <br>
<br>
[Install] <br>
WantedBy=multi-user.target <br>
<br>
<span style=" color: #660066;"># /etc/systemd/system/linstor-satellite.service.d/override.conf <br>
<span style=" color: #4f4c49;">[Service] <br>
ExecStart= <br>
ExecStart=/usr/share/linstor-server/bin/Satellite --logs=/var/log/linstor-satellite --config-directory=/etc/linstor --keep-res=vm-100 <br>
<br>
The Satellite does have the drbd resource in /var/lib/linstor.d and it's got <br>
read when the service was started: <br>
<br>
root@pve1:~# cat /var/lib/linstor.d/vm-100-disk-0.res <br>
<span style=" color: #660066;"># This file was generated by linstor(1.0.1), do not edit manually. <br>
<br>
<span style=" color: #4f4c49;">resource "vm-100-disk-0" <br>
{ <br>
template-file "linstor_common.conf"; <br>
<br>
net <br>
{ <br>
cram-hmac-alg sha1; <br>
shared-secret "6cKQWOfF+w97AlSyyLZT"; <br>
} <br>
<br>
on pve1 <br>
{ <br>
volume 0 <br>
{ <br>
disk /dev/drbdpool/vm-100-disk-0_00000; <br>
disk <br>
{ <br>
discard-zeroes-if-aligned yes; <br>
rs-discard-granularity 65536; <br>
} <br>
meta-disk internal; <br>
device minor 1000; <br>
} <br>
node-id 0; <br>
} <br>
<br>
on pve2 <br>
{ <br>
volume 0 <br>
{ <br>
disk /dev/drbd/this/is/not/used; <br>
disk <br>
{ <br>
discard-zeroes-if-aligned yes; <br>
rs-discard-granularity 65536; <br>
} <br>
meta-disk internal; <br>
device minor 1000; <br>
} <br>
node-id 1; <br>
} <br>
<br>
connection <br>
{ <br>
host pve1 address ipv4 </span></span></span></span></span></span></span></span></span><a style=" color: #4f4c49; font-family:'arial'; font-size: 11pt; background-color: #f6f6f6;" href="http://10.1.1.1:7000">10.1.1.1:7000</a><span style=" font-family:'arial'; font-size: 11pt; color: #4f4c49; background-color: #f6f6f6;">; <br>
host pve2 address ipv4 </span><a style=" color: #4f4c49; font-family:'arial'; font-size: 11pt; background-color: #f6f6f6;" href="http://10.1.1.2:7000">10.1.1.2:7000</a><span style=" font-family:'arial'; font-size: 11pt; color: #4f4c49; background-color: #f6f6f6;">; <br>
} <br>
} <br>
<br>
Aug 15 13:03:24 pve1 systemd[1]: Started LINSTOR Satellite Service. <br>
Aug 15 13:03:25 pve1 Satellite[3223]: LINSTOR, Module Satellite <br>
Aug 15 13:03:25 pve1 Satellite[3223]: Version: 1.0.1 (98a9905de43631b745c7c0741c2ef8f577513b23) <br>
Aug 15 13:03:25 pve1 Satellite[3223]: Build time: 2019-08-09T06:50:31+00:00 <br>
Aug 15 13:03:25 pve1 Satellite[3223]: Java Version: 11 <br>
Aug 15 13:03:25 pve1 Satellite[3223]: Java VM: Debian, Version 11.0.4+11-post-Debian-1deb10u1 <br>
Aug 15 13:03:25 pve1 Satellite[3223]: Operating system: Linux, Version 5.0.18-1-pve <br>
Aug 15 13:03:25 pve1 Satellite[3223]: Environment: amd64, 2 processors, 351 MiB memory reserved for allocations <br>
Aug 15 13:03:25 pve1 Satellite[3223]: System components initialization in progress <br>
Aug 15 13:03:25 pve1 Satellite[3223]: 13:03:25.657 [main] INFO LINSTOR/Satellite - Log directory set to: '/var/log/linstor-satellite' <br>
Aug 15 13:03:25 pve1 Satellite[3223]: 13:03:25.662 [Main] INFO LINSTOR/Satellite - Loading API classes started. <br>
Aug 15 13:03:26 pve1 Satellite[3223]: 13:03:26.126 [Main] INFO LINSTOR/Satellite - API classes loading finished: 462ms <br>
Aug 15 13:03:26 pve1 Satellite[3223]: 13:03:26.126 [Main] INFO LINSTOR/Satellite - Dependency injection started. <br>
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: An illegal reflective access operation has occurred <br>
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/linstor-server/lib/guice-4.2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) <br>
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1 <br>
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations <br>
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: All illegal access operations will be denied in a future release <br>
Aug 15 13:03:27 pve1 Satellite[3223]: 13:03:27.544 [Main] INFO LINSTOR/Satellite - Dependency injection finished: 1417ms <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.397 [Main] INFO LINSTOR/Satellite - Removing res files from /var/lib/linstor.d, keeping files matching regex: vm-100 <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.401 [Main] INFO LINSTOR/Satellite - Starting service instance 'DeviceManager' of type DeviceManager <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.403 [Main] INFO LINSTOR/Satellite - Starting service instance 'DrbdEventPublisher-1' of type DrbdEventPublisher <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.404 [Main] INFO LINSTOR/Satellite - Starting service instance 'DrbdEventService-1' of type DrbdEventService <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.410 [Main] INFO LINSTOR/Satellite - Starting service instance 'FileEventService' of type FileEventService <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.415 [Main] INFO LINSTOR/Satellite - Starting service instance 'TimerEventService' of type TimerEventService <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.416 [Main] INFO LINSTOR/Satellite - Initializing main network communications service <br>
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.441 [Main] INFO LINSTOR/Satellite - NetComService started on port /0:0:0:0:0:0:0:0:3366 <br>
<br>
So, I expect the controller will be started up automatically at system startup. <br>
<br>
However, after both pve nodes got started, the controller is not starting up. <br>
It's b/c the drbd resource (vm-100-disk-0) is not up: <br>
<br>
root@pve1:~# drbdadm status <br>
<span style=" color: #660066;"># No currently configured DRBD found. <br>
<br>
<span style=" color: #4f4c49;">I had to manually bring it up (drbdadm up vm-100-disk-0) and after that the <br>
controller was able to get started. <br>
<br>
Is that a bug or an expected behavior? </td>
</tr>
</table>
<br>
<br>
<span style=" font-family:'arial'; font-size: 11pt; color: #c0c0c0;"><i>-- <br>
Best regards,<br>
Alex Kolesnik</td>
</tr>
</table>
<br>
<br>
<br>
<span style=" font-family:'arial'; color: #c0c0c0;"><i>-- <br>
Best regards,<br>
Alex Kolesnik</body></html>