[DRBD-user] drbd with a third node diskless - how ?

[Martin]

Because drbd needs a third node for quorum i try to build
a proxmox cluster with drbd/linstor with 2 nodes with storage
an one diskless.


  3 Nodes
     2 with one 2.5GB for drbd replication and one 1 GB nic as failover
     1 with only 1 GB nic only for quorum




This steps i've made

All nodes

wget -O- https://packages.linbit.com/package-signing-pubkey.asc | 
apt-key add -
echo "deb http://packages.linbit.com/proxmox/ proxmox-6 drbd-9.0" > 
/etc/apt/sources.list.d/linbit.list
apt update
apt install -y drbd-dkms drbd-utils lvm2
apt install linstor-controller linstor-satellite linstor-client

systemctl start linstor-satellite
systemctl enable linstor-satellite

systemctl start linstor-controller
systemctl enable linstor-controller


Node 1

instor node create proxmoxn1 192.168.3.1 --node-type Combined
linstor node create proxmoxn2 192.168.3.2 --node-type Combined
linstor node create proxmoxn3 192.168.2.243

linstor node interface create proxmoxn1 G_nic 192.168.2.245
linstor node interface create proxmoxn2 G_nic 192.168.2.246

Nodes 1/2
mcedit /etc/linstor/linstor-client.conf
[global]
     controllers=proxmoxn1,proxmoxn2

fdisk /dev/nvme0n1  ...
vgcreate vg_ssd /dev/nvme0n1p4

Node 1

linstor storage-pool create lvm proxmoxn1 pool_ssd vg_ssd
linstor storage-pool create lvm proxmoxn2 pool_ssd vg_ssd
linstor resource-group create adcgrp --storage-pool pool_ssd 
--place-count 2
linstor vg create adcgrp

Nodes 1/2
apt install linstor-proxmox

Node 1
mcedit /etc/pve/storage.cfg

     drbd: drbdstorage
     content images, rootdir
      controller 192.168.3.1,192.168.3.2
      resourcegroup adcgrp


After that proxmox shows drbdstorage. But no chance to create vm's
After some moments proxmox stop creating vm.

but drbdsetup status shows
vm-100-disk_1 and very slow sync

after sync to 100% linstor show vm-100-disk_1,
but no vm in proxmox




BTW in my last experiments i could start linstor only with the command
linstor  but now i must start with linstor -interactive. Why ?


╭─────────────────────────────────────────────────────────────╮
┊ Node      ┊ NodeType  ┊ Addresses                  ┊ State  ┊
╞═════════════════════════════════════════════════════════════╡
┊ proxmoxn1 ┊ COMBINED  ┊ 192.168.3.1:3366 (PLAIN)   ┊ Online ┊
┊ proxmoxn2 ┊ COMBINED  ┊ 192.168.3.2:3366 (PLAIN)   ┊ Online ┊
┊ proxmoxn3 ┊ SATELLITE ┊ 192.168.2.243:3366 (PLAIN) ┊ Online ┊
╰─────────────────────────────────────────────────────────────╯

LINSTOR ==> r l
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ ResourceName  ┊ Node      ┊ Port ┊ Usage  ┊ 
Conns                           ┊    State ┊ CreatedOn           ┊
╞══════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ vm-100-disk-1 ┊ proxmoxn1 ┊ 7000 ┊ Unused ┊ 
Connecting(proxmoxn3,proxmoxn2) ┊ UpToDate ┊ 2021-06-20 21:57:55 ┊
┊ vm-100-disk-1 ┊ proxmoxn2 ┊ 7000 ┊ ┊                                 
┊  Unknown ┊ 2021-06-20 21:57:55 ┊
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

LINSTOR ==> rg l
╭─────────────────────────────────────────────────────────────────╮
┊ ResourceGroup ┊ SelectFilter             ┊ VlmNrs ┊ Description ┊
╞═════════════════════════════════════════════════════════════════╡
┊ DfltRscGrp    ┊ PlaceCount: 2            ┊        ┊ ┊
╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡
┊ adcgrp        ┊ PlaceCount: 2            ┊ 0      ┊ ┊
┊               ┊ StoragePool(s): pool_ssd ┊        ┊ ┊
╰─────────────────────────────────────────────────────────────────╯

LINSTOR ==> v l
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ Node      ┊ Resource      ┊ StoragePool ┊ VolNr ┊ MinorNr ┊ 
DeviceName    ┊ Allocated ┊ InUse  ┊    State ┊
╞═══════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ proxmoxn1 ┊ vm-100-disk-1 ┊ pool_ssd    ┊     0 ┊    1000 ┊ 
/dev/drbd1000 ┊ 32.01 GiB ┊ Unused ┊ UpToDate ┊
┊ proxmoxn2 ┊ vm-100-disk-1 ┊ pool_ssd    ┊     0 ┊    1000 ┊ 
None          ┊           ┊        ┊  Unknown ┊
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯


root at proxmoxn1:~# drbdadm status
vm-100-disk-1 role:Secondary
   disk:UpToDate quorum:no
   proxmoxn2 connection:Connecting
   proxmoxn3 connection:Connecting


root at proxmoxn2:~# drbdadm status
# No currently configured DRBD found.


root at proxmoxn3:~# drbdadm status
vm-100-disk-1 role:Secondary
   disk:Diskless quorum:no
   proxmoxn1 connection:Connecting
   proxmoxn2 connection:Connecting


I think i must have done something wrong, But i don´t see what.


Best regards

Martin