[DRBD-user] linstor controller errors

Nicholas Papadakos rick.dangerous at gmail.com
Sun Oct 1 12:30:07 CEST 2023


hello everyone,

on every linstor controller i have setup on proxmox (3 different clusters
on different locations, not related between them)

got the following error at the same time :


root at pve02:~# systemctl status linstor-controller
● linstor-controller.service - drbd-reactor controlled linstor-controller
     Loaded: loaded (/lib/systemd/system/linstor-controller.service;
disabled; preset: enabled)
    Drop-In: /run/systemd/system/linstor-controller.service.d
             └─reactor.conf
     Active: active (running) since Sun 2023-09-24 21:41:01 EEST; 6 days ago
   Main PID: 7156 (java)
      Tasks: 58 (limit: 38227)
     Memory: 481.8M
        CPU: 21min 54.401s
     CGroup: /system.slice/linstor-controller.service
             └─7156 java -Xms32M -classpath
"/usr/share/linstor-server/lib/conf:/usr/share/linstor-server/lib/*"
com.linbit.linstor.core.Controller --logs=/var/log/linstor-controller
--config-directory=/etc/linstor

Oct 01 01:12:02 pve02 Controller[7156]: 01:12:02.885
[grizzly-http-server-4] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.122; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:12:11 pve02 Controller[7156]: 01:12:11.810
[grizzly-http-server-7] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.121; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:12:33 pve02 Controller[7156]: 01:12:33.507
[grizzly-http-server-9] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.123; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:13:02 pve02 Controller[7156]: 01:13:02.844
[grizzly-http-server-11] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.122; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:13:11 pve02 Controller[7156]: 01:13:11.607
[grizzly-http-server-1] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.121; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:13:33 pve02 Controller[7156]: 01:13:33.955
[grizzly-http-server-2] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.123; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:14:02 pve02 Controller[7156]: 01:14:02.687
[grizzly-http-server-4] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.122; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:14:11 pve02 Controller[7156]: 01:14:11.690
[grizzly-http-server-7] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.121; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:14:43 pve02 Controller[7156]: 01:14:43.390
[grizzly-http-server-9] INFO  LINSTOR/Controller - SYSTEM - REST/API
RestClient(10.0.3.123; 'linstor-proxmox/7.0.1')/QryAllSizeInfo
Oct 01 01:15:00 pve02 Controller[7156]: 01:15:00.043 [SpaceTrackingService]
ERROR LINSTOR/Controller - SYSTEM - Uncaught exception in j [Report number
651082BA-00000-000000]


the reports shows :


root at pve02:~# linstor error-reports show 651082BA-00000-000000
ERROR REPORT 651082BA-00000-000000

============================================================

Application:                        LINBIT® LINSTOR
Module:                             Controller
Version:                            1.24.2
Build ID:                           adb19ca96a07039401023410c1ea116f09929295
Build time:                         2023-08-30T05:15:08+00:00
Error time:                         2023-10-01 01:15:00
Node:                               pve02

============================================================

Reported error:
===============

Category:                           Error
Class name:                         ImplementationError
Class canonical name:               com.linbit.ImplementationError
Generated at:                       Method 'run', Source file
'SpaceTrackingTask.java', Line #315

Error message:                      Uncaught exception in j

Call backtrace:

    Method                                   Native Class:Line number
    run                                      N
 com.linbit.linstor.spacetracking.j:315
    run                                      N      java.lang.Thread:833

Caused by:
==========

Category:                           RuntimeException
Class name:                         ProvisionException
Class canonical name:               com.google.inject.ProvisionException
Generated at:                       Method 'toProvisionException', Source
file 'InternalProvisionException.java', Line #251

Error message:                      Unable to provision, see the following
errors:

1) [Guice/ErrorInCustomProvider]: OutOfScopeException: Cannot access
Key[type=TransactionMgrSQL, annotation=[none]] outside of a scoping block
  at ApiModule.configure(ApiModule.java:79)
  while locating TransactionMgrSQL

Learn more:
  https://github.com/google/guice/wiki/ERROR_IN_CUSTOM_PROVIDER

1 error

======================
Full classname legend:
======================
ApiModule:           "com.linbit.linstor.api.ApiModule"
OutOfScopeException: "com.google.inject.OutOfScopeException"
TransactionMgrSQL:
"com.linbit.linstor.transaction.manager.TransactionMgrSQL"
========================
End of classname legend:
========================


Call backtrace:

    Method                                   Native Class:Line number
    toProvisionException                     N
 com.google.inject.internal.InternalProvisionException:251
    get                                      N
 com.google.inject.internal.InjectorImpl$1:1104
    getConnection                            N
 com.linbit.linstor.dbdrivers.sql.SQLEngine:689
    loadAll                                  N
 com.linbit.linstor.dbdrivers.sql.SQLEngine:491
    loadAll                                  N
 com.linbit.linstor.dbdrivers.AbsDatabaseDriver:170
    loadAllAsList                            N
 com.linbit.linstor.dbdrivers.ControllerDatabaseDriver:33
    b                                        N
 com.linbit.linstor.spacetracking.h:260
    e                                        N
 com.linbit.linstor.spacetracking.j:737
    run                                      N
 com.linbit.linstor.spacetracking.j:283
    run                                      N      java.lang.Thread:833

Caused by:
==========

Category:                           RuntimeException
Class name:                         OutOfScopeException
Class canonical name:               com.google.inject.OutOfScopeException
Generated at:                       Method 'getScopedObjectMap', Source
file 'LinStorScope.java', Line #125

Error message:                      Cannot access
Key[type=com.linbit.linstor.transaction.manager.TransactionMgrSQL,
annotation=[none]] outside of a scoping block

Call backtrace:

    Method                                   Native Class:Line number
    getScopedObjectMap                       N
 com.linbit.linstor.api.LinStorScope:125
    get                                      N
 com.linbit.linstor.api.LinStorScope$2:91
    get                                      N
 com.google.inject.internal.InternalFactoryToProviderAdapter:45
    get                                      N
 com.google.inject.internal.InjectorImpl$1:1101
    getConnection                            N
 com.linbit.linstor.dbdrivers.sql.SQLEngine:689
    loadAll                                  N
 com.linbit.linstor.dbdrivers.sql.SQLEngine:491
    loadAll                                  N
 com.linbit.linstor.dbdrivers.AbsDatabaseDriver:170
    loadAllAsList                            N
 com.linbit.linstor.dbdrivers.ControllerDatabaseDriver:33
    b                                        N
 com.linbit.linstor.spacetracking.h:260
    e                                        N
 com.linbit.linstor.spacetracking.j:737
    run                                      N
 com.linbit.linstor.spacetracking.j:283
    run                                      N      java.lang.Thread:833


END OF ERROR REPORT.


running a systemctl restart linstor-controller on the node that runs the
controller seems to bring things back in order.

any thoughts ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20231001/a18df8bf/attachment.htm>


More information about the drbd-user mailing list