[DRBD-user] linstor-server 1.9.0 release
Adam Goryachev
mailinglists at websitemanagers.com.au
Thu Sep 24 14:19:37 CEST 2020
On 24/9/20 15:30, Rene Peinthor wrote:
> Sounds weird, I don't think we changed anything with the database
> initialization.
>
> Can you post your /etc/linstor/linstor.toml?
>
> Cheers,
> Rene
>
Hi, please see the contents below (I assume these are default settings):
# Basic linstor configuration toml file
# For more options check documentation
[db]
user = "linstor"
password = "linstor"
connection_url = "jdbc:h2:/var/lib/linstor/linstordb"
Thanks,
Adam
> On Wed, Sep 23, 2020 at 6:54 PM Adam Goryachev
> <mailinglists at websitemanagers.com.au
> <mailto:mailinglists at websitemanagers.com.au>> wrote:
>
> Hi,
>
> After upgrading on one of my test nodes I got some errors, so I
> thought to upgrade the controller, and now I am unable to start
> the controller.
>
> Here is what I get in the system log:
>
> Sep 24 02:33:40 castle systemd[1]: Started LINSTOR Controller Service.
> Sep 24 02:33:41 castle Controller[1093]: LINSTOR, Module Controller
> Sep 24 02:33:41 castle Controller[1093]: Version: 1.9.0
> (678acd24a8b9b73a735407cd79ca33a5e95eb2e2)
> Sep 24 02:33:41 castle Controller[1093]: Build time:
> 2020-09-23T10:27:49+00:00
> Sep 24 02:33:41 castle Controller[1093]: Java Version: 11
> Sep 24 02:33:41 castle Controller[1093]: Java VM:
> Debian, Version 11.0.8+10-post-Debian-1deb10u1
> Sep 24 02:33:41 castle Controller[1093]: Operating system:
> Linux, Version 4.19.0-10-amd64
> Sep 24 02:33:41 castle Controller[1093]: Environment:
> amd64, 2 processors, 1988 MiB memory reserved for allocations
> Sep 24 02:33:41 castle Controller[1093]: System components
> initialization in progress
> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.355 [main] INFO
> LINSTOR/Controller - SYSTEM - ErrorReporter DB version 1 found.
> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.359 [main] INFO
> LINSTOR/Controller - SYSTEM - Log directory set to:
> '/var/log/linstor-controller'
> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.410 [main] WARN
> io.sentry.dsn.Dsn - *** Couldn't find a suitable DSN, Sentry
> operations will do nothing! See documentation:
> https://docs.sentry.io/clients/java/
> <https://docs.sentry.io/clients/java/> ***
> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.434 [Main] INFO
> LINSTOR/Controller - SYSTEM - Loading API classes started.
> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.046 [Main] INFO
> LINSTOR/Controller - SYSTEM - API classes loading finished: 611ms
> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.047 [Main] INFO
> LINSTOR/Controller - SYSTEM - Dependency injection started.
> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.065 [Main] INFO
> LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension
> module
> "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule"
> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.065 [Main] INFO
> LINSTOR/Controller - SYSTEM - Extension module
> "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule"
> is not installed
> Sep 24 02:33:43 castle Controller[1093]: WARNING: An illegal
> reflective access operation has occurred
> Sep 24 02:33:43 castle Controller[1093]: WARNING: Illegal
> reflective access by
> com.google.inject.internal.cglib.core.$ReflectUtils$1
> (file:/usr/share/linstor-server/lib/guice-4.2.2.jar) to method
> java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
> Sep 24 02:33:43 castle Controller[1093]: WARNING: Please consider
> reporting this to the maintainers of
> com.google.inject.internal.cglib.core.$ReflectUtils$1
> Sep 24 02:33:43 castle Controller[1093]: WARNING: Use
> --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> Sep 24 02:33:43 castle Controller[1093]: WARNING: All illegal
> access operations will be denied in a future release
> Sep 24 02:33:44 castle Controller[1093]: 02:33:44.962 [Main] INFO
> LINSTOR/Controller - SYSTEM - Dependency injection finished: 1915ms
> Sep 24 02:33:45 castle Controller[1093]: 02:33:45.533 [Main] INFO
> LINSTOR/Controller - SYSTEM - Initializing authentication subsystem
> Sep 24 02:33:45 castle Controller[1093]: 02:33:45.736 [Main] INFO
> LINSTOR/Controller - SYSTEM - Starting service instance
> 'TimerEventService' of type TimerEventService
> Sep 24 02:33:45 castle Controller[1093]: 02:33:45.740 [Main] INFO
> LINSTOR/Controller - SYSTEM - Initializing the database connection
> pool
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.303 [Main] ERROR
> LINSTOR/Controller - SYSTEM - Database initialization error
> [Report number 5F6B78E5-00000-000000]
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.343 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Shutdown in progress
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.344 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Shutting down service instance
> 'TaskScheduleService' of type TaskScheduleService
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.345 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Waiting for service instance
> 'TaskScheduleService' to complete shutdown
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.345 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Shutting down service instance
> 'DatabaseService' of type DatabaseService
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.349 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Waiting for service instance
> 'DatabaseService' to complete shutdown
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.350 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Shutting down service instance
> 'TimerEventService' of type TimerEventService
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.351 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Waiting for service instance
> 'TimerEventService' to complete shutdown
> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.351 [Thread-2]
> INFO LINSTOR/Controller - SYSTEM - Shutdown complete
> Sep 24 02:33:46 castle systemd[1]: linstor-controller.service:
> Main process exited, code=exited, status=20/n/a
> Sep 24 02:33:46 castle systemd[1]: linstor-controller.service:
> Failed with result 'exit-code'.
>
> Here is
> /var/log/linstor-controller/ErrorReport-5F6B78E5-00000-000000.log
>
> ERROR REPORT 5F6B78E5-00000-000000
>
> ============================================================
>
> Application: LINBIT® LINSTOR
> Module: Controller
> Version: 1.9.0
> Build ID: 678acd24a8b9b73a735407cd79ca33a5e95eb2e2
> Build time: 2020-09-23T10:27:49+00:00
> Error time: 2020-09-24 02:33:46
> Node: castle
>
> ============================================================
>
> Reported error:
> ===============
>
> Description:
> Database initialization error
>
> Category: LinStorException
> Class name: SystemServiceStartException
> Class canonical name: com.linbit.SystemServiceStartException
> Generated at: Method 'initialize', Source
> file 'DbConnectionPoolInitializer.java', Line #56
> Error message: Database initialization error
>
> Call backtrace:
>
> Method Native Class:Line number
> initialize N
> com.linbit.linstor.dbcp.DbConnectionPoolInitializer:56
> startSystemServices N
> com.linbit.linstor.core.ApplicationLifecycleManager:87
> start N
> com.linbit.linstor.core.Controller:337
> main N
> com.linbit.linstor.core.Controller:556
>
> Caused by:
> ==========
>
> Category: RuntimeException
> Class name: LinStorDBRuntimeException
> Class canonical name: com.linbit.linstor.LinStorDBRuntimeException
> Generated at: Method
> 'setTransactionIsolation', Source file 'DbConnectionPool.java',
> Line #485
>
> Error message: Failed to set transaction
> isolation
>
> Call backtrace:
>
> Method Native Class:Line number
> setTransactionIsolation N
> com.linbit.linstor.dbcp.DbConnectionPool:485
> migrate N
> com.linbit.linstor.dbcp.DbConnectionPool:202
> initialize N
> com.linbit.linstor.dbcp.DbConnectionPoolInitializer:50
> startSystemServices N
> com.linbit.linstor.core.ApplicationLifecycleManager:87
> start N
> com.linbit.linstor.core.Controller:337
> main N
> com.linbit.linstor.core.Controller:556
>
> Caused by:
> ==========
>
> Category: Exception
> Class name: JdbcSQLException
> Class canonical name: org.h2.jdbc.JdbcSQLException
> Generated at: Method 'getJdbcSQLException',
> Source file 'DbException.java', Line #357
>
> Error message: Wrong user name or password
> [28000-197]
>
> Call backtrace:
>
> Method Native Class:Line number
> getJdbcSQLException N
> org.h2.message.DbException:357
> get N
> org.h2.message.DbException:179
> get N
> org.h2.message.DbException:155
> get N
> org.h2.message.DbException:144
> validateUserAndPassword N
> org.h2.engine.Engine:341
> createSessionAndValidate N
> org.h2.engine.Engine:165
> createSession N
> org.h2.engine.Engine:140
> createSession N org.h2.engine.Engine:28
> connectEmbeddedOrServer N
> org.h2.engine.SessionRemote:351
> <init> N
> org.h2.jdbc.JdbcConnection:124
> <init> N
> org.h2.jdbc.JdbcConnection:103
> connect N org.h2.Driver:69
> getConnection N
> java.sql.DriverManager:677
> getConnection N
> java.sql.DriverManager:189
> createConnection N
> org.apache.commons.dbcp2.DriverManagerConnectionFactory:123
> makeObject N
> org.apache.commons.dbcp2.PoolableConnectionFactory:355
> create N
> org.apache.commons.pool2.impl.GenericObjectPool:889
> borrowObject N
> org.apache.commons.pool2.impl.GenericObjectPool:424
> borrowObject N
> org.apache.commons.pool2.impl.GenericObjectPool:349
> getConnection N
> org.apache.commons.dbcp2.PoolingDataSource:134
> setTransactionIsolation N
> com.linbit.linstor.dbcp.DbConnectionPool:477
> migrate N
> com.linbit.linstor.dbcp.DbConnectionPool:202
> initialize N
> com.linbit.linstor.dbcp.DbConnectionPoolInitializer:50
> startSystemServices N
> com.linbit.linstor.core.ApplicationLifecycleManager:87
> start N
> com.linbit.linstor.core.Controller:337
> main N
> com.linbit.linstor.core.Controller:556
>
> Finally I found another log in /var/lib/linstor/linstordb.trace.db
>
> 2020-09-24 02:33:45 database: wrong user or password; user: "LINSTOR"
> org.h2.message.DbException: Wrong user name or password [28000-197]
> at org.h2.message.DbException.get(DbException.java:179)
> at org.h2.message.DbException.get(DbException.java:155)
> at org.h2.message.DbException.get(DbException.java:144)
> at org.h2.engine.Engine.openSession(Engine.java:107)
> at org.h2.engine.Engine.openSession(Engine.java:179)
> at
> org.h2.engine.Engine.createSessionAndValidate(Engine.java:157)
> at org.h2.engine.Engine.createSession(Engine.java:140)
> at org.h2.engine.Engine.createSession(Engine.java:28)
> at
> org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:351)
> at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:124)
> at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:103)
> at org.h2.Driver.connect(Driver.java:69)
> at
> java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
> at
> java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
> at
> org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:123)
> at
> org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:889)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:424)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349)
> at
> org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134)
> at
> com.linbit.linstor.dbcp.DbConnectionPool.setTransactionIsolation(DbConnectionPool.java:477)
> at
> com.linbit.linstor.dbcp.DbConnectionPool.migrate(DbConnectionPool.java:202)
> at
> com.linbit.linstor.dbcp.DbConnectionPoolInitializer.initialize(DbConnectionPoolInitializer.java:50)
> at
> com.linbit.linstor.core.ApplicationLifecycleManager.startSystemServices(ApplicationLifecycleManager.java:87)
> at
> com.linbit.linstor.core.Controller.start(Controller.java:337)
> at
> com.linbit.linstor.core.Controller.main(Controller.java:556)
> Caused by: org.h2.jdbc.JdbcSQLException: Wrong user name or
> password [28000-197]
> at
> org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
> ... 26 more
>
>
> Luckily, this is just a test environment, but I'd like to know if
> I messed up in my upgrade process somehow or this is a bug in the
> new version. I haven't downgraded or anything yet.
>
> Regards,
> Adam
>
>
> On 23/9/20 22:31, Rene Peinthor wrote:
>> Hi!
>>
>> Here is a new minor Linstor release, this release contains more
>> bug fixes than new features.
>> Notable features are a new auto-unplace option, to reduce the
>> replica-count of resources
>> to the value set in the resource-group after a certain time. And
>> error-reports can now be deleted
>> from the REST-API (client).
>>
>> linstor-server 1.9.0
>> --------------------
>> * Added auto-"unplace", automatically removes resources if they
>> exceed the replica count if `DrbdOptions/auto-unplace` is set
>> * Added error-report deletion API
>> * Expose DRBD promotion_score and may_promote via REST-API and
>> server sent events
>> * Prometheus metrics are now mostly cached and much faster to query
>> * Send SD_NOTIFY READY to systemd if services are ready
>> * OpenFlex resource are now correctly limited to 32 characters
>> * Fix etcd modify net-interface (incorrect serialization)
>> * Fix autoplacer also toggling diskless resources
>> * Fix snapshot-shipping with more than 1 volume definitions
>> * Fix zfs create options handling
>> * Fixed recaching of lvm-filter
>> * Fix missing client answer if drbd device doesn't become ready
>> in reasonable time
>> * REST-API v1.4.0
>>
>> https://www.linbit.com/downloads/linstor/linstor-server-1.9.0.tar.gz
>> <https://www.linbit.com/downloads/linstor/linstor-server-1.9.0.tar.gz>
>>
>> Linstor PPA:
>> https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack
>> <https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack>
>>
>> Cheers,
>> Rene
>>
>> _______________________________________________
>> Star us on GITHUB:https://github.com/LINBIT <https://github.com/LINBIT>
>> drbd-user mailing list
>> drbd-user at lists.linbit.com <mailto:drbd-user at lists.linbit.com>
>> https://lists.linbit.com/mailman/listinfo/drbd-user <https://lists.linbit.com/mailman/listinfo/drbd-user>
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT
> <https://github.com/LINBIT>
> drbd-user mailing list
> drbd-user at lists.linbit.com <mailto:drbd-user at lists.linbit.com>
> https://lists.linbit.com/mailman/listinfo/drbd-user
> <https://lists.linbit.com/mailman/listinfo/drbd-user>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20200924/fcf1f402/attachment-0001.htm>
More information about the drbd-user
mailing list