[DRBD-user] linstor-satellite does not bring up drbd resource under proxmox 6

Brice Chappe bricechappe at gmail.com
Thu Aug 15 17:29:40 CEST 2019 

Hi Alex,

I used it too, not with keep-res option (didn't know it), I 'll try it :)

And yes it is expected. I read the proxmox plugin code and change
somethings to work with one vm which contains linstor-controller. The
problem is your device is not writable.

To do it writable, I had to change the proxmox plugin code.
In the file /usr/share/perl5/PVE/Storage/Custom/LINSTORPlugin.pm, you have
to change the code with :

sub activate_volume {
    my ( $class, $storeid, $scfg, $volname, $snapname, $cache ) = @_;

    die "Snapshot not implemented on DRBD\n" if $snapname;

   ###THE IMPORTANT PART###
   if( ignore_volume( $scfg, $volname ) ) {
          print "It is the controller vm volume, so set blockdev r/w\n";
          system ('blockdev --setrw ' . get_dev_path $volname);
          return undef;
    }
    ###########################
    return undef if ignore_volume( $scfg, $volname );
   ....

With it, when your proxmox start the vm it will call the activate_volume
function and most importantly the blockdev system call. It will put your
device in rw state, and proxmox will start your vm.

Give it a try. I wrote some code to get vm controller starting after a
complete cluster outage, and it is working like a charm :)
When I'll finish it, I'll publish it.


Best regards
Brice



------------------------
Hello,

To be honest, I'm not sure if that's a linstor-satellite issue. Please,
correct
me.

There're two proxmox nodes pve1 and pve2. DRBD resource has been
configured:
root at linstor-controller:~# linstor resource list
???????????????????????????????????????????????????
? ResourceName ? Node ? Port ? Usage ? State ?
???????????????????????????????????????????????????
? vm-100-disk-0 ? pve1 ? 7000 ? InUse ? UpToDate ?
? vm-100-disk-0 ? pve2 ? 7000 ? Unused ? UpToDate ?
???????????????????????????????????????????????????

The linstor-controller itself is stored onto that DRBD resource:
root at pve1:~# pct config 100
arch: amd64
cores: 2
hostname: linstor-controller
memory: 512
net0: name=eth0,bridge=vmbr10,gw=10.1.1.1,hwaddr=26:81:53:50:FB:48,ip=
10.1.1.254/24,type=veth
onboot: 1
ostype: debian
rootfs: drbdpool:vm-100-disk-0,mountoptions=noatime,size=2G
swap: 512
unprivileged: 1

Linstor-satellite is started on both pve nodes within this systemd service:

root at pve1:~# systemctl cat linstor-satellite.service
# /lib/systemd/system/linstor-satellite.service
[Unit]
Description=LINSTOR Satellite Service
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
ExecStart=/usr/share/linstor-server/bin/Satellite
--logs=/var/log/linstor-satellite --config-directory=/etc/linstor
# if killed by signal 143 -> SIGTERM, 129 -> SIGHUP
SuccessExitStatus=0 143 129
User=root
PrivateTmp=yes

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/linstor-satellite.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/share/linstor-server/bin/Satellite
--logs=/var/log/linstor-satellite --config-directory=/etc/linstor
--keep-res=vm-100

The Satellite does have the drbd resource in /var/lib/linstor.d and it's
got
read when the service was started:

root at pve1:~# cat /var/lib/linstor.d/vm-100-disk-0.res
# This file was generated by linstor(1.0.1), do not edit manually.

resource "vm-100-disk-0"
{
template-file "linstor_common.conf";

net
{
cram-hmac-alg sha1;
shared-secret "6cKQWOfF+w97AlSyyLZT";
}

on pve1
{
volume 0
{
disk /dev/drbdpool/vm-100-disk-0_00000;
disk
{
discard-zeroes-if-aligned yes;
rs-discard-granularity 65536;
}
meta-disk internal;
device minor 1000;
}
node-id 0;
}

on pve2
{
volume 0
{
disk /dev/drbd/this/is/not/used;
disk
{
discard-zeroes-if-aligned yes;
rs-discard-granularity 65536;
}
meta-disk internal;
device minor 1000;
}
node-id 1;
}

connection
{
host pve1 address ipv4 10.1.1.1:7000;
host pve2 address ipv4 10.1.1.2:7000;
}
}

Aug 15 13:03:24 pve1 systemd[1]: Started LINSTOR Satellite Service.
Aug 15 13:03:25 pve1 Satellite[3223]: LINSTOR, Module Satellite
Aug 15 13:03:25 pve1 Satellite[3223]: Version: 1.0.1
(98a9905de43631b745c7c0741c2ef8f577513b23)
Aug 15 13:03:25 pve1 Satellite[3223]: Build time: 2019-08-09T06:50:31+00:00
Aug 15 13:03:25 pve1 Satellite[3223]: Java Version: 11
Aug 15 13:03:25 pve1 Satellite[3223]: Java VM: Debian, Version
11.0.4+11-post-Debian-1deb10u1
Aug 15 13:03:25 pve1 Satellite[3223]: Operating system: Linux, Version
5.0.18-1-pve
Aug 15 13:03:25 pve1 Satellite[3223]: Environment: amd64, 2 processors, 351
MiB memory reserved for allocations
Aug 15 13:03:25 pve1 Satellite[3223]: System components initialization in
progress
Aug 15 13:03:25 pve1 Satellite[3223]: 13:03:25.657 [main] INFO
LINSTOR/Satellite - Log directory set to: '/var/log/linstor-satellite'
Aug 15 13:03:25 pve1 Satellite[3223]: 13:03:25.662 [Main] INFO
LINSTOR/Satellite - Loading API classes started.
Aug 15 13:03:26 pve1 Satellite[3223]: 13:03:26.126 [Main] INFO
LINSTOR/Satellite - API classes loading finished: 462ms
Aug 15 13:03:26 pve1 Satellite[3223]: 13:03:26.126 [Main] INFO
LINSTOR/Satellite - Dependency injection started.
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: An illegal reflective access
operation has occurred
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: Illegal reflective access by
com.google.inject.internal.cglib.core.$ReflectUtils$1
(file:/usr/share/linstor-server/lib/guice-4.2.2.jar) to method
java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: Please consider reporting
this to the maintainers of
com.google.inject.internal.cglib.core.$ReflectUtils$1
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: Use --illegal-access=warn to
enable warnings of further illegal reflective access operations
Aug 15 13:03:26 pve1 Satellite[3223]: WARNING: All illegal access
operations will be denied in a future release
Aug 15 13:03:27 pve1 Satellite[3223]: 13:03:27.544 [Main] INFO
LINSTOR/Satellite - Dependency injection finished: 1417ms
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.397 [Main] INFO
LINSTOR/Satellite - Removing res files from /var/lib/linstor.d, keeping
files matching regex: vm-100
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.401 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DeviceManager' of type
DeviceManager
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.403 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DrbdEventPublisher-1' of
type DrbdEventPublisher
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.404 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DrbdEventService-1' of type
DrbdEventService
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.410 [Main] INFO
LINSTOR/Satellite - Starting service instance 'FileEventService' of type
FileEventService
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.415 [Main] INFO
LINSTOR/Satellite - Starting service instance 'TimerEventService' of type
TimerEventService
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.416 [Main] INFO
LINSTOR/Satellite - Initializing main network communications service
Aug 15 13:03:28 pve1 Satellite[3223]: 13:03:28.441 [Main] INFO
LINSTOR/Satellite - NetComService started on port /0:0:0:0:0:0:0:0:3366

So, I expect the controller will be started up automatically at system
startup.

However, after both pve nodes got started, the controller is not starting
up.
It's b/c the drbd resource (vm-100-disk-0) is not up:

root at pve1:~# drbdadm status
# No currently configured DRBD found.

I had to manually bring it up (drbdadm up vm-100-disk-0) and after that the
controller was able to get started.

Is that a bug or an expected behavior?

-- 
Best regards,
Alex Kolesnik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20190815/bd17c090/attachment-0001.htm>

More information about the drbd-user mailing list