[DRBD-user] Create a third diskless node on an active 2-node cluster
Ferran Alchimia
ferran.alchimia at gmail.com
Wed Mar 3 15:24:10 CET 2021
Hello again,
Ok, I have resolved the issue, it was the interface used to communicate
between hosts.
host H has 1 NIC with 2 ports.
host T has 1 NIC with 2 ports.
host C has 1 NIC with 1 port.
H and T are connected with a crossover cable using port 1.
H, T and C are connected through a virtual switch using port 2.
So I defined a "Data" interface that goes through the crossover table and I
assigned it as PrefNic to the storagepool.
And I defined a "NormalTraffic" interface that goes through the
virtualswitch in order to communicate the 3 nodes. I have rebuild the nodes
with "linstor create node" using this interface.
I couldn't make it work, it only works if the PrefNic of the storagepool is
also the the NormalTraffic interface.
Is it possible to have a Data interface for the drbdadm to exchange data
and another interface for the node communication?
cheers,
Ferran
Missatge de Ferran Alchimia <ferran.alchimia at gmail.com> del dia dc., 3 de
març 2021 a les 13:41:
> Hi,
>
> Thank you for your help I reloaded de drbd so it now has the correct
> version:
>
> DRBDADM_BUILDTAG=GIT-hash:\ a7820b3c14497a34f955ba5ce56cf1bc9d2d353e\
> build\ by\ @buildsystem\,\ 2021-02-09\ 09:27:27
> DRBDADM_API_VERSION=2
> DRBD_KERNEL_VERSION_CODE=0x09001c
> DRBD_KERNEL_VERSION=9.0.28
> DRBDADM_VERSION_CODE=0x091000
> DRBDADM_VERSION=9.16.0
>
> Now, when I try to create the test resource it fails with another message:
>
> linstor resource create C test
> INFO:
> Resource-definition property 'DrbdOptions/Resource/quorum' updated
> from 'off' to 'majority' by auto-quorum
> INFO:
> Resource-definition property 'DrbdOptions/Resource/on-no-quorum'
> updated from 'off' to 'io-error' by auto-quorum
> SUCCESS:
> Description:
> New resource 'test' on node 'C' registered.
> Details:
> Resource 'test' on node 'C' UUID is:
> 25c11e12-d136-41fb-ad25-68049a7a6bac
> SUCCESS:
> Description:
> Volume with number '0' on resource 'test' on node 'C' successfully
> registered
> Details:
> Volume UUID is: 70309d61-bf39-43aa-a41f-532587031ccd
> ERROR:
> (Node: 'C') An unknown exception occurred while processing the
> resource test
> Show reports:
> linstor error-reports show 603F7FA5-7AC40-000000
> SUCCESS:
> Added peer(s) 'C' to resource 'test' on 'T'
> SUCCESS:
> Added peer(s) 'C' to resource 'test' on 'H'
>
>
> # drbdadm status
> test role:Secondary
> disk:UpToDate
> C connection:*Connecting*
> H role:Secondary
> peer-disk:UpToDate
>
> And the error in linstor-satellite:
>
> ERROR REPORT 603F7FA5-7AC40-000004
>
> ============================================================
>
> Application: LINBIT® LINSTOR
> Module: Satellite
> Version: 1.11.1
> Build ID:
> fe95a94d86c66c6c9846a3cf579a1a776f95d3f4
> Build time: 2021-01-13T08:34:55+00:00
> Error time: 2021-03-03 13:31:51
> Node: C
>
> ============================================================
>
> Reported error:
> ===============
>
> Category: RuntimeException
> Class name: NullPointerException
> Class canonical name: java.lang.NullPointerException
> Generated at: Method 'normalizeAndCheck', Source
> file 'UnixPath.java', Line #75
>
>
> Error context:
> An error occurred while processing resource 'Node: 'C', Rsc: 'test''
>
> Call backtrace:
>
> Method Native Class:Line number
> normalizeAndCheck N sun.nio.fs.UnixPath:75
> <init> N sun.nio.fs.UnixPath:69
> getPath N
> sun.nio.fs.UnixFileSystem:279
> readObject N
> com.linbit.linstor.layer.drbd.utils.MdSuperblockBuffer:73
> hasMetaData N
> com.linbit.linstor.layer.drbd.DrbdLayer:906
> adjustDrbd N
> com.linbit.linstor.layer.drbd.DrbdLayer:580
> process N
> com.linbit.linstor.layer.drbd.DrbdLayer:387
> process N
> com.linbit.linstor.core.devmgr.DeviceHandlerImpl:789
> processResourcesAndSnapshots N
> com.linbit.linstor.core.devmgr.DeviceHandlerImpl:335
> dispatchResources N
> com.linbit.linstor.core.devmgr.DeviceHandlerImpl:145
> dispatchResources N
> com.linbit.linstor.core.devmgr.DeviceManagerImpl:268
> phaseDispatchDeviceHandlers N
> com.linbit.linstor.core.devmgr.DeviceManagerImpl:924
> devMgrLoop N
> com.linbit.linstor.core.devmgr.DeviceManagerImpl:627
> run N
> com.linbit.linstor.core.devmgr.DeviceManagerImpl:545
> run N java.lang.Thread:834
>
>
> END OF ERROR REPORT.
>
>
> Note that the lvm storage is not defined on C, do I have to define some
> dummy space for the diskless "allocations"?
>
> regards,
> Ferran
>
> Missatge de Rene Peinthor <rene.peinthor at linbit.com> del dia dc., 3 de
> març 2021 a les 13:08:
>
>> DRBD_KERNEL_VERSION_CODE=0x08040b
>>
>> Says you have the old DRBD8 kernel module loaded, Linstor only supports
>> DRBD9 >=
>>
>> Btw. you don't need to create a diskless storage pool (until you want
>> special properties on that pool), Linstor already has
>> DfltDisklessStorPool.
>> Also diskless-on-remaing is not really needed, as you will get a
>> tiebreaker diskless resource anyway on a 3 node cluster, with auto-quorum
>> enabled.
>>
>> Cheers,
>> Rene
>>
>> On Wed, Mar 3, 2021 at 12:48 PM Gábor Hernádi <gabor.hernadi at linbit.com>
>> wrote:
>>
>>> Hello,
>>>
>>> when you executed `linstor node create C ...` Linstor should have given
>>> you a list of supported layers + providers or reasons why layers /
>>> providers are not supported. You can recheck that list with
>>> linstor node info
>>> That command should tell you why Linstor thinks that C does not support
>>> DRBD.
>>>
>>> Best regards,
>>> Gabor
>>>
>>> On Wed, Mar 3, 2021 at 12:29 PM Ferran Alchimia <
>>> ferran.alchimia at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I currently have a 2 node (combined) cluster and I want to add an HA
>>>> linstor-controller. I was following this guide
>>>> https://www.linbit.com/drbd-user-guide/linstor-guide-1_0-en/#s-linstor_ha
>>>>
>>>> The first step asks for an AutoQuorum capable resource. So I think I
>>>> have to add a third node to my cluster. This is my starting position:
>>>>
>>>> ╭─────────────────────────────────────────────────────╮
>>>> ┊ Node ┊ NodeType ┊ Addresses ┊ State ┊
>>>> ╞═════════════════════════════════════════════════════╡
>>>> ┊ H ┊ COMBINED ┊ 10.0.7.106:3366 (PLAIN) ┊ Online ┊
>>>> ┊ T ┊ COMBINED ┊ 10.0.7.105:3366 (PLAIN) ┊ Online ┊
>>>> ╰─────────────────────────────────────────────────────╯
>>>>
>>>> ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
>>>> ┊ StoragePool ┊ Node ┊ Driver ┊ PoolName ┊
>>>> FreeCapacity ┊ TotalCapacity ┊ CanSnapshots ┊ State ┊
>>>>
>>>> ╞══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
>>>> ┊ DfltDisklessStorPool ┊ H ┊ DISKLESS ┊ ┊
>>>> ┊ ┊ False ┊ Ok ┊
>>>> ┊ DfltDisklessStorPool ┊ T ┊ DISKLESS ┊ ┊
>>>> ┊ ┊ False ┊ Ok ┊
>>>> ┊ drbdpool ┊ H ┊ LVM_THIN ┊ vgthc1/drbdpool ┊
>>>> 3.13 TiB ┊ 3.49 TiB ┊ True ┊ Ok ┊
>>>> ┊ drbdpool ┊ T ┊ LVM_THIN ┊ vgthc1/drbdpool ┊
>>>> 3.13 TiB ┊ 3.49 TiB ┊ True ┊ Ok ┊
>>>>
>>>> ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>>>
>>>> Since my third node won't have any storage devices dedicated to drbd I
>>>> presumed that I could add a DISKLESS node (I don't know if this is true),
>>>> just for quorum and not for replica.
>>>>
>>>> linstor n create --node-type satellite C 10.100.1.3
>>>>
>>>> linstor n l
>>>> ╭───────────────────────────────────────────────────────────╮
>>>> ┊ Node ┊ NodeType ┊ Addresses ┊ State ┊
>>>> ╞═══════════════════════════════════════════════════════════╡
>>>> ┊ C ┊ SATELLITE ┊ 10.100.1.3:3366 (PLAIN) ┊ Online ┊
>>>> ┊ H ┊ COMBINED ┊ 10.0.7.106:3366 (PLAIN) ┊ Online ┊
>>>> ┊ T ┊ COMBINED ┊ 10.0.7.105:3366 (PLAIN) ┊ Online ┊
>>>> ╰───────────────────────────────────────────────────────────╯
>>>> linstor sp create diskless C drbdpool
>>>>
>>>>
>>>> ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
>>>> ┊ StoragePool ┊ Node ┊ Driver ┊ PoolName ┊
>>>> FreeCapacity ┊ TotalCapacity ┊ CanSnapshots ┊ State ┊
>>>>
>>>> ╞══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
>>>> ┊ DfltDisklessStorPool ┊ C ┊ DISKLESS ┊ ┊
>>>> ┊ ┊ False ┊ Ok ┊
>>>> ┊ DfltDisklessStorPool ┊ H ┊ DISKLESS ┊ ┊
>>>> ┊ ┊ False ┊ Ok ┊
>>>> ┊ DfltDisklessStorPool ┊ T ┊ DISKLESS ┊ ┊
>>>> ┊ ┊ False ┊ Ok ┊
>>>> ┊ drbdpool ┊ C ┊ DISKLESS ┊ ┊
>>>> ┊ ┊ False ┊ Ok ┊
>>>> ┊ drbdpool ┊ H ┊ LVM_THIN ┊ vgthc1/drbdpool ┊
>>>> 3.13 TiB ┊ 3.49 TiB ┊ True ┊ Ok ┊
>>>> ┊ drbdpool ┊ T ┊ LVM_THIN ┊ vgthc1/drbdpool ┊
>>>> 3.13 TiB ┊ 3.49 TiB ┊ True ┊ Ok ┊
>>>>
>>>> ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>>>
>>>> I add the --diskless-on-remaining flag for the rg definition (I don't
>>>> know if this is needed)
>>>> linstor rg m drbdpoolvm --place-count 2 --diskless-on-remaining
>>>>
>>>> linstor rg l
>>>> ╭──────────────────────────────────────────────────────────────────╮
>>>> ┊ ResourceGroup ┊ SelectFilter ┊ VlmNrs ┊ Description ┊
>>>> ╞══════════════════════════════════════════════════════════════════╡
>>>> ┊ DfltRscGrp ┊ PlaceCount: 2 ┊ ┊ ┊
>>>> ╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡
>>>> ┊ drbdpoolvm ┊ PlaceCount: 2 ┊ 0 ┊ ┊
>>>> ┊ ┊ StoragePool(s): drbdpool ┊ ┊ ┊
>>>> ┊ ┊ DisklessOnRemaining: True ┊ ┊ ┊
>>>> ╰──────────────────────────────────────────────────────────────────╯
>>>>
>>>> And then I try to create a test resource
>>>> linstor resource create T test
>>>> SUCCESS:(...)
>>>>
>>>> linstor resource create H test
>>>> WARNING: Could not find suitable node to automatically create a tie
>>>> breaking resource for 'test'.
>>>> SUCCESS:(...)
>>>>
>>>> linstor resource create C test
>>>> ERROR:Description: Satellite 'C' does not support the following
>>>> layers: [DRBD]Details: Node(s): 'C', Resource: 'test'Show reports:
>>>> linstor error-reports show 603F507D-00000-000005
>>>>
>>>> As you can see the resource creation fails on C node. I have installed
>>>> drbd on that node just in case, here goes the output:
>>>> drbdadm --version
>>>> DRBDADM_BUILDTAG=GIT-hash:\ a7820b3c14497a34f955ba5ce56cf1bc9d2d353e\
>>>> build\ by\ @buildsystem\,\ 2021-02-09\ 09:27:27
>>>> DRBDADM_API_VERSION=1
>>>> DRBD_KERNEL_VERSION_CODE=0x08040b
>>>> DRBDADM_VERSION_CODE=0x091000
>>>> DRBDADM_VERSION=9.16.0
>>>>
>>>> Overall my C node does not have any "storage" for the drbd cluster, I
>>>> just want it to be a diskless node in order to achieve HA for the
>>>> resources. It seems I have misunderstood something but I just don't know
>>>> where to look for more info. Any hints/help would be appreciated.
>>>>
>>>> Regards,
>>>> Ferran
>>>> _______________________________________________
>>>> Star us on GITHUB: https://github.com/LINBIT
>>>> drbd-user mailing list
>>>> drbd-user at lists.linbit.com
>>>> https://lists.linbit.com/mailman/listinfo/drbd-user
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Gabor Hernadi
>>> _______________________________________________
>>> Star us on GITHUB: https://github.com/LINBIT
>>> drbd-user mailing list
>>> drbd-user at lists.linbit.com
>>> https://lists.linbit.com/mailman/listinfo/drbd-user
>>>
>> _______________________________________________
>> Star us on GITHUB: https://github.com/LINBIT
>> drbd-user mailing list
>> drbd-user at lists.linbit.com
>> https://lists.linbit.com/mailman/listinfo/drbd-user
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20210303/4f828128/attachment-0001.htm>
More information about the drbd-user
mailing list