<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Hi,</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Thank you all for your replies.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Below, the error reports:</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">I missed to say that zern1 uses
protocol A, I did set up it with drbdmanage sites, but I think
it's not relevant because errors are on all nodes<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<br>
<div class="moz-cite-prefix">root@pve211:~# linstor error-reports
list<br>
<br>
┊ 1 ┊ 5C3B5BC3-00000-000000 ┊ 2019-01-13 16:46:24 ┊ Controller ┊<br>
┊ 2 ┊ 5C3B5BC3-00000-000001 ┊ 2019-01-13 16:46:24 ┊ Controller ┊<br>
┊ 3 ┊ 5C3B5BC3-00000-000002 ┊ 2019-01-13 16:46:26 ┊ Controller ┊<br>
┊ 4 ┊ 5C3B5BC3-00000-000003 ┊ 2019-01-13 16:46:27 ┊ Controller ┊<br>
┊ 5 ┊ 5C3B5BC3-00000-000004 ┊ 2019-01-13 16:46:27 ┊ Controller ┊<br>
┊ 6 ┊ 5C3B5BC3-00000-000005 ┊ 2019-01-13 16:46:27 ┊ Controller ┊<br>
┊ 7 ┊ 5C3B5BC3-00000-000006 ┊ 2019-01-13 16:46:28 ┊ Controller ┊<br>
┊ 8 ┊ 5C3B5BC3-00000-000007 ┊ 2019-01-13 16:46:28 ┊ Controller ┊<br>
┊ 9 ┊ 5C3B5BC3-00000-000008 ┊ 2019-01-13 16:46:28 ┊ Controller ┊</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
root@pve211:~# linstor error-reports s 5C3B5BC3-00000-000000<br>
ERROR REPORT 5C3B5BC3-00000-000000<br>
<br>
============================================================<br>
<br>
Application: LINBIT® LINSTOR<br>
Module: Controller<br>
Version: 0.7.5<br>
Build ID:
d74305b420fdc878182afa162378a317e6a4a3b9<br>
Build time: 2018-12-21T08:28:19+00:00<br>
Error time: 2019-01-13 16:46:24<br>
Node: pve211<br>
Peer: 127.0.0.1:56966<br>
<br>
============================================================<br>
<br>
Reported error:<br>
===============<br>
<br>
Category: RuntimeException<br>
Class name: ApiRcException<br>
Class canonical name:
com.linbit.linstor.core.apicallhandler.response.ApiRcException<br>
Generated at: Method 'loadRscDfn', Source file
'CtrlApiDataLoader.java', Line #130<br>
<br>
Error message: Resource definition
'vm-108-disk-1' not found.<br>
<br>
Error context:<br>
Resource definition 'vm-108-disk-1' not found.<br>
<br>
Call backtrace:<br>
<br>
Method Native Class:Line
number<br>
loadRscDfn N
com.linbit.linstor.core.apicallhandler.controller.CtrlApiDataLoader:130<br>
loadRscDfn N
com.linbit.linstor.core.apicallhandler.controller.CtrlApiDataLoader:104<br>
createVolumeDefinitions N
com.linbit.linstor.core.apicallhandler.controller.CtrlVlmDfnApiCallHandler:128<br>
createVlmDfns N
com.linbit.linstor.core.apicallhandler.controller.CtrlApiCallHandler:324<br>
execute N
com.linbit.linstor.api.protobuf.controller.CreateVolumeDefinition:50<br>
executeNonReactive N
com.linbit.linstor.proto.CommonMessageProcessor:520<br>
lambda$execute$13 N
com.linbit.linstor.proto.CommonMessageProcessor:496<br>
doInScope N
com.linbit.linstor.core.apicallhandler.ScopeRunner:141<br>
lambda$null$0 N
com.linbit.linstor.core.apicallhandler.ScopeRunner:71<br>
call N
reactor.core.publisher.MonoCallable:92<br>
trySubscribeScalarMap N
reactor.core.publisher.FluxFlatMap:126<br>
subscribe N
reactor.core.publisher.MonoFlatMapMany:46<br>
subscribe N
reactor.core.publisher.Flux:6877<br>
onNext N
reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:184<br>
request N
reactor.core.publisher.Operators$ScalarSubscription:1640<br>
onSubscribe N
reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:131<br>
subscribe N
reactor.core.publisher.MonoCurrentContext:35<br>
subscribe N
reactor.core.publisher.MonoFlatMapMany:49<br>
subscribe N
reactor.core.publisher.FluxOnAssembly:252<br>
subscribe N
reactor.core.publisher.FluxOnAssembly:252<br>
subscribe N
reactor.core.publisher.FluxOnErrorResume:47<br>
subscribe N
reactor.core.publisher.FluxOnErrorResume:47<br>
subscribe N
reactor.core.publisher.FluxOnErrorResume:47<br>
subscribe N
reactor.core.publisher.FluxOnErrorResume:47<br>
subscribe N
reactor.core.publisher.FluxContextStart:49<br>
subscribe N
reactor.core.publisher.Flux:6877<br>
onNext N
reactor.core.publisher.FluxFlatMap$FlatMapMain:372<br>
slowPath N
reactor.core.publisher.FluxArray$ArraySubscription:126<br>
request N
reactor.core.publisher.FluxArray$ArraySubscription:99<br>
onSubscribe N
reactor.core.publisher.FluxFlatMap$FlatMapMain:332<br>
subscribe N
reactor.core.publisher.FluxMerge:70<br>
subscribe N
reactor.core.publisher.FluxOnErrorResume:47<br>
subscribe N
reactor.core.publisher.Flux:6877<br>
onComplete N
reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber:208<br>
subscribe N
reactor.core.publisher.FluxConcatArray:81<br>
subscribe N
reactor.core.publisher.FluxPeek:83<br>
subscribe N
reactor.core.publisher.FluxPeek:83<br>
subscribe N
reactor.core.publisher.FluxPeek:83<br>
subscribe N
reactor.core.publisher.FluxOnErrorResume:47<br>
subscribe N
reactor.core.publisher.FluxDefer:55<br>
subscribe N
reactor.core.publisher.Flux:6877<br>
onNext N
reactor.core.publisher.FluxFlatMap$FlatMapMain:372<br>
drainAsync N
reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:391<br>
drain N
reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:633<br>
onNext N
reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:238<br>
drainFused N
reactor.core.publisher.UnicastProcessor:234<br>
drain N
reactor.core.publisher.UnicastProcessor:267<br>
onNext N
reactor.core.publisher.UnicastProcessor:343<br>
next N
reactor.core.publisher.FluxCreate$IgnoreSink:573<br>
next N
reactor.core.publisher.FluxCreate$SerializedSink:151<br>
processInOrder N
com.linbit.linstor.netcom.TcpConnectorPeer:361<br>
doProcessMessage N
com.linbit.linstor.proto.CommonMessageProcessor:215<br>
lambda$processMessage$2 N
com.linbit.linstor.proto.CommonMessageProcessor:161<br>
onNext N
reactor.core.publisher.FluxPeek$PeekSubscriber:177<br>
runAsync N
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:396<br>
run N
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:480<br>
call N
reactor.core.scheduler.WorkerTask:84<br>
call N
reactor.core.scheduler.WorkerTask:37<br>
run N
java.util.concurrent.FutureTask:266<br>
access$201 N
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask:180<br>
run N
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask:293<br>
runWorker N
java.util.concurrent.ThreadPoolExecutor:1149<br>
run N
java.util.concurrent.ThreadPoolExecutor$Worker:624<br>
run N
java.lang.Thread:748<br>
<br>
<br>
END OF ERROR REPORT.<br>
<p><br>
</p>
5C3B5BC3-00000-000002
<div class="moz-cite-prefix">
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Category:
RuntimeException<br>
Class name: ApiRcException<br>
Class canonical name:
com.linbit.linstor.core.apicallhandler.response.ApiRcException<br>
Generated at: Method 'loadRscDfn', Source
file 'CtrlApiDataLoader.java', Line #130<br>
<br>
Error message: Resource definition
'vm-102-disk-1' not found.<br>
<br>
Error context:<br>
Resource definition 'vm-102-disk-1' not found.<br>
<br>
</div>
<p><br>
</p>
<p>Others are pretty the same...<br>
</p>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">root@pve211:~# journalctl -u
linstor-satellite<br>
-- Logs begin at Sun 2019-01-13 15:32:33 CET, end at Wed
2019-01-16 12:22:00 CET. --<br>
Jan 13 16:20:06 pve211 systemd[1]: Started LINSTOR Satellite
Service.<br>
Jan 13 16:20:07 pve211 Satellite[13805]: LINSTOR, Module Satellite<br>
Jan 13 16:20:07 pve211 Satellite[13805]: Version: 0.7.5
(d74305b420fdc878182afa162378a317e6a4a3b9)<br>
Jan 13 16:20:07 pve211 Satellite[13805]: Build time:
2018-12-21T08:28:19+00:00<br>
Jan 13 16:20:07 pve211 Satellite[13805]: Java Version: 1.8<br>
Jan 13 16:20:07 pve211 Satellite[13805]: Java VM:
Oracle Corporation, Version 25.181-b13<br>
Jan 13 16:20:07 pve211 Satellite[13805]: Operating system:
Linux, Version 4.13.13-6-pve<br>
Jan 13 16:20:07 pve211 Satellite[13805]: Environment:
amd64, 8 processors, 7138 MiB memory reserved for allocations<br>
Jan 13 16:20:07 pve211 Satellite[13805]: System components
initialization in progress<br>
Jan 13 16:20:07 pve211 Satellite[13805]: 16:20:07.527 [main] INFO
LINSTOR/Satellite - Log directory set to:
'/var/log/linstor-satellite'<br>
Jan 13 16:20:07 pve211 Satellite[13805]: 16:20:07.529 [Main] INFO
LINSTOR/Satellite - Loading API classes started.<br>
Jan 13 16:20:07 pve211 Satellite[13805]: 16:20:07.793 [Main] INFO
LINSTOR/Satellite - API classes loading finished: 263ms<br>
Jan 13 16:20:07 pve211 Satellite[13805]: 16:20:07.793 [Main] INFO
LINSTOR/Satellite - Dependency injection started.<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.626 [Main] INFO
LINSTOR/Satellite - Dependency injection finished: 832ms<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.801 [Main] INFO
LINSTOR/Satellite - Removing all res files from /var/lib/linstor.d<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.805 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DeviceManager' of
type DeviceManager<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.806 [Main] INFO
LINSTOR/Satellite - Starting service instance
'DrbdEventPublisher-1' of type DrbdEventPublish<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.806 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DrbdEventService-1'
of type DrbdEventService<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.808 [Main] INFO
LINSTOR/Satellite - Starting service instance 'FileEventService'
of type FileEventService<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.808 [Main] INFO
LINSTOR/Satellite - Starting service instance 'TimerEventService'
of type TimerEventService<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.808 [Main] INFO
LINSTOR/Satellite - Initializing main network communications
service<br>
Jan 13 16:20:08 pve211 Satellite[13805]: 16:20:08.820 [Main] INFO
LINSTOR/Satellite - NetComService started on port
/0:0:0:0:0:0:0:0:3366<br>
Jan 13 16:40:31 pve211 Satellite[13805]: 16:40:31.727
[MainWorkerPool-1] INFO LINSTOR/Satellite - Controller connected
and authenticated (10.12.150.1:55252)<br>
Jan 13 16:40:31 pve211 Satellite[13805]: 16:40:31.757
[MainWorkerPool-2] INFO LINSTOR/Satellite - Node 'pve211'
created.<br>
Jan 13 16:40:31 pve211 Satellite[13805]: 16:40:31.767
[MainWorkerPool-2] INFO LINSTOR/Satellite - Storage pool
'DfltDisklessStorPool' created.<br>
Jan 13 16:40:33 pve211 Satellite[13805]: 16:40:33.786
[MainWorkerPool-4] INFO LINSTOR/Satellite - Storage pool
'drbdpool' created.<br>
Jan 13 17:02:07 pve211 Satellite[13805]: 17:02:07.360
[MainWorkerPool-6] INFO LINSTOR/Satellite - Resource
'vm-108-disk-1' created for node 'pve211'.<br>
Jan 13 17:02:07 pve211 Satellite[13805]: 17:02:07.715
[MainWorkerPool-8] INFO LINSTOR/Satellite - Resource
'vm-108-disk-1' updated for node 'pve211'.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">root@pve212:~# journalctl -u
linstor-satellite<br>
-- Logs begin at Sun 2019-01-13 15:34:23 CET, end at Wed
2019-01-16 12:23:04 CET. --<br>
Jan 13 16:24:25 pve212 systemd[1]: Started LINSTOR Satellite
Service.<br>
Jan 13 16:24:26 pve212 Satellite[15540]: LINSTOR, Module Satellite<br>
Jan 13 16:24:26 pve212 Satellite[15540]: Version: 0.7.5
(d74305b420fdc878182afa162378a317e6a4a3b9)<br>
Jan 13 16:24:26 pve212 Satellite[15540]: Build time:
2018-12-21T08:28:19+00:00<br>
Jan 13 16:24:26 pve212 Satellite[15540]: Java Version: 1.8<br>
Jan 13 16:24:26 pve212 Satellite[15540]: Java VM:
Oracle Corporation, Version 25.181-b13<br>
Jan 13 16:24:26 pve212 Satellite[15540]: Operating system:
Linux, Version 4.13.13-6-pve<br>
Jan 13 16:24:26 pve212 Satellite[15540]: Environment:
amd64, 8 processors, 7138 MiB memory reserved for allocations<br>
Jan 13 16:24:26 pve212 Satellite[15540]: System components
initialization in progress<br>
Jan 13 16:24:26 pve212 Satellite[15540]: 16:24:26.353 [main] INFO
LINSTOR/Satellite - Log directory set to:
'/var/log/linstor-satellite'<br>
Jan 13 16:24:26 pve212 Satellite[15540]: 16:24:26.355 [Main] INFO
LINSTOR/Satellite - Loading API classes started.<br>
Jan 13 16:24:26 pve212 Satellite[15540]: 16:24:26.630 [Main] INFO
LINSTOR/Satellite - API classes loading finished: 275ms<br>
Jan 13 16:24:26 pve212 Satellite[15540]: 16:24:26.630 [Main] INFO
LINSTOR/Satellite - Dependency injection started.<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.458 [Main] INFO
LINSTOR/Satellite - Dependency injection finished: 828ms<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.639 [Main] INFO
LINSTOR/Satellite - Removing all res files from /var/lib/linstor.d<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.642 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DeviceManager' of
type DeviceManager<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.643 [Main] INFO
LINSTOR/Satellite - Starting service instance
'DrbdEventPublisher-1' of type DrbdEventPublisher<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.643 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DrbdEventService-1'
of type DrbdEventService<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.645 [Main] INFO
LINSTOR/Satellite - Starting service instance 'FileEventService'
of type FileEventService<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.645 [Main] INFO
LINSTOR/Satellite - Starting service instance 'TimerEventService'
of type TimerEventService<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.645 [Main] INFO
LINSTOR/Satellite - Initializing main network communications
service<br>
Jan 13 16:24:27 pve212 Satellite[15540]: 16:24:27.657 [Main] INFO
LINSTOR/Satellite - NetComService started on port
/0:0:0:0:0:0:0:0:3366<br>
Jan 13 16:42:17 pve212 Satellite[15540]: 16:42:17.206
[MainWorkerPool-1] INFO LINSTOR/Satellite - Controller connected
and authenticated (10.12.150.1:51862)<br>
Jan 13 16:42:17 pve212 Satellite[15540]: 16:42:17.274
[MainWorkerPool-2] INFO LINSTOR/Satellite - Node 'pve212'
created.<br>
Jan 13 16:42:17 pve212 Satellite[15540]: 16:42:17.285
[MainWorkerPool-2] INFO LINSTOR/Satellite - Storage pool
'DfltDisklessStorPool' created.<br>
Jan 13 16:42:19 pve212 Satellite[15540]: 16:42:19.085
[MainWorkerPool-4] INFO LINSTOR/Satellite - Storage pool
'drbdpool' created.<br>
Jan 13 16:42:23 pve212 Satellite[15540]: 16:42:23.522
[MainWorkerPool-6] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'pve212'.<br>
Jan 13 16:42:24 pve212 Satellite[15540]: 16:42:24.059
[MainWorkerPool-8] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' updated for node 'pve212'.<br>
Jan 13 16:47:25 pve212 Satellite[15540]: 16:47:25.530
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'pve213'.<br>
Jan 13 16:47:25 pve212 Satellite[15540]: 16:47:25.530
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' updated for node 'pve212'.<br>
Jan 13 16:48:09 pve212 Satellite[15540]: 16:48:09.210
[MainWorkerPool-4] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'zern1'.<br>
Jan 13 16:48:09 pve212 Satellite[15540]: 16:48:09.210
[MainWorkerPool-4] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' updated for node 'pve212'.<br>
Jan 13 16:48:09 pve212 Satellite[15540]: 16:48:09.210
[MainWorkerPool-4] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' updated for node 'pve213'.<br>
Jan 13 17:14:23 pve212 Satellite[15540]: 17:14:23.765
[MainWorkerPool-6] INFO LINSTOR/Satellite - Resource
'vm-202-disk-1' created for node 'pve212'.<br>
Jan 13 17:14:24 pve212 Satellite[15540]: 17:14:24.280
[MainWorkerPool-8] INFO LINSTOR/Satellite - Resource
'vm-202-disk-1' updated for node 'pve212'.<br>
Jan 13 17:20:09 pve212 Satellite[15540]: 17:20:09.412
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-102-disk-1' created for node 'pve212'.<br>
Jan 13 17:20:09 pve212 Satellite[15540]: 17:20:09.877
[MainWorkerPool-4] INFO LINSTOR/Satellite - Resource
'vm-102-disk-1' updated for node 'pve212'.<br>
Jan 13 17:25:17 pve212 Satellite[15540]: 17:25:17.380
[MainWorkerPool-6] INFO LINSTOR/Satellite - Resource
'vm-102-disk-1' created for node 'pve213'.<br>
Jan 13 17:25:17 pve212 Satellite[15540]: 17:25:17.380
[MainWorkerPool-6] INFO LINSTOR/Satellite - Resource
'vm-102-disk-1' updated for node 'pve212'.<br>
<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">root@pve213:~# journalctl -u
linstor-satellite<br>
-- Logs begin at Sun 2019-01-13 15:36:06 CET, end at Wed
2019-01-16 12:23:38 CET. --<br>
Jan 13 16:26:09 pve213 systemd[1]: Started LINSTOR Satellite
Service.<br>
Jan 13 16:26:09 pve213 Satellite[14120]: LINSTOR, Module Satellite<br>
Jan 13 16:26:09 pve213 Satellite[14120]: Version: 0.7.5
(d74305b420fdc878182afa162378a317e6a4a3b9)<br>
Jan 13 16:26:09 pve213 Satellite[14120]: Build time:
2018-12-21T08:28:19+00:00<br>
Jan 13 16:26:09 pve213 Satellite[14120]: Java Version: 1.8<br>
Jan 13 16:26:09 pve213 Satellite[14120]: Java VM:
Oracle Corporation, Version 25.181-b13<br>
Jan 13 16:26:09 pve213 Satellite[14120]: Operating system:
Linux, Version 4.13.13-6-pve<br>
Jan 13 16:26:09 pve213 Satellite[14120]: Environment:
amd64, 8 processors, 7138 MiB memory reserved for allocations<br>
Jan 13 16:26:09 pve213 Satellite[14120]: System components
initialization in progress<br>
Jan 13 16:26:10 pve213 Satellite[14120]: 16:26:10.020 [main] INFO
LINSTOR/Satellite - Log directory set to:
'/var/log/linstor-satellite'<br>
Jan 13 16:26:10 pve213 Satellite[14120]: 16:26:10.022 [Main] INFO
LINSTOR/Satellite - Loading API classes started.<br>
Jan 13 16:26:10 pve213 Satellite[14120]: 16:26:10.277 [Main] INFO
LINSTOR/Satellite - API classes loading finished: 254ms<br>
Jan 13 16:26:10 pve213 Satellite[14120]: 16:26:10.277 [Main] INFO
LINSTOR/Satellite - Dependency injection started.<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.032 [Main] INFO
LINSTOR/Satellite - Dependency injection finished: 755ms<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.208 [Main] INFO
LINSTOR/Satellite - Removing all res files from /var/lib/linstor.d<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.212 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DeviceManager' of
type DeviceManager<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.212 [Main] INFO
LINSTOR/Satellite - Starting service instance
'DrbdEventPublisher-1' of type DrbdEventPublisher<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.213 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DrbdEventService-1'
of type DrbdEventService<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.214 [Main] INFO
LINSTOR/Satellite - Starting service instance 'FileEventService'
of type FileEventService<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.215 [Main] INFO
LINSTOR/Satellite - Starting service instance 'TimerEventService'
of type TimerEventService<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.215 [Main] INFO
LINSTOR/Satellite - Initializing main network communications
service<br>
Jan 13 16:26:11 pve213 Satellite[14120]: 16:26:11.227 [Main] INFO
LINSTOR/Satellite - NetComService started on port
/0:0:0:0:0:0:0:0:3366<br>
Jan 13 16:43:58 pve213 Satellite[14120]: 16:43:58.127
[MainWorkerPool-1] INFO LINSTOR/Satellite - Controller connected
and authenticated (10.12.150.1:33896)<br>
Jan 13 16:43:58 pve213 Satellite[14120]: 16:43:58.159
[MainWorkerPool-2] INFO LINSTOR/Satellite - Node 'pve213'
created.<br>
Jan 13 16:43:58 pve213 Satellite[14120]: 16:43:58.168
[MainWorkerPool-2] INFO LINSTOR/Satellite - Storage pool
'DfltDisklessStorPool' created.<br>
Jan 13 16:44:00 pve213 Satellite[14120]: 16:44:00.096
[MainWorkerPool-4] INFO LINSTOR/Satellite - Storage pool
'drbdpool' created.<br>
Jan 13 16:49:06 pve213 Satellite[14120]: 16:49:06.059
[MainWorkerPool-6] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'pve212'.<br>
Jan 13 16:49:06 pve213 Satellite[14120]: 16:49:06.059
[MainWorkerPool-6] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'pve213'.<br>
Jan 13 16:49:49 pve213 Satellite[14120]: 16:49:49.729
[MainWorkerPool-8] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'zern1'.<br>
Jan 13 16:49:49 pve213 Satellite[14120]: 16:49:49.730
[MainWorkerPool-8] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' updated for node 'pve212'.<br>
Jan 13 16:49:49 pve213 Satellite[14120]: 16:49:49.730
[MainWorkerPool-8] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' updated for node 'pve213'.<br>
Jan 13 17:26:57 pve213 Satellite[14120]: 17:26:57.910
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-102-disk-1' created for node 'pve212'.<br>
Jan 13 17:26:57 pve213 Satellite[14120]: 17:26:57.910
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-102-disk-1' created for node 'pve213'.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">root@zern1:~# grep Satellite
/var/log/daemon.log<br>
Jan 13 16:22:01 zern1 systemd[1]: Started LINSTOR Satellite
Service.<br>
Jan 13 16:22:02 zern1 Satellite[20849]: LINSTOR, Module Satellite<br>
Jan 13 16:22:02 zern1 Satellite[20849]: Version: 0.7.5
(d74305b420fdc878182afa162378a317e6a4a3b9)<br>
Jan 13 16:22:02 zern1 Satellite[20849]: Build time:
2018-12-21T08:28:19+00:00<br>
Jan 13 16:22:02 zern1 Satellite[20849]: Java Version: 1.8<br>
Jan 13 16:22:02 zern1 Satellite[20849]: Java VM: Oracle
Corporation, Version 25.181-b13<br>
Jan 13 16:22:02 zern1 Satellite[20849]: Operating system: Linux,
Version 4.13.13-6-pve<br>
Jan 13 16:22:02 zern1 Satellite[20849]: Environment: amd64,
2 processors, 1653 MiB memory reserved for allocations<br>
Jan 13 16:22:02 zern1 Satellite[20849]: System components
initialization in progress<br>
Jan 13 16:22:02 zern1 Satellite[20849]: 16:22:02.710 [main] INFO
LINSTOR/Satellite - Log directory set to:
'/var/log/linstor-satellite'<br>
Jan 13 16:22:02 zern1 Satellite[20849]: 16:22:02.716 [Main] INFO
LINSTOR/Satellite - Loading API classes started.<br>
Jan 13 16:22:03 zern1 Satellite[20849]: 16:22:03.415 [Main] INFO
LINSTOR/Satellite - API classes loading finished: 698ms<br>
Jan 13 16:22:03 zern1 Satellite[20849]: 16:22:03.417 [Main] INFO
LINSTOR/Satellite - Dependency injection started.<br>
Jan 13 16:22:05 zern1 Satellite[20849]: 16:22:05.819 [Main] INFO
LINSTOR/Satellite - Dependency injection finished: 2398ms<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.380 [Main] INFO
LINSTOR/Satellite - Removing all res files from /var/lib/linstor.d<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.390 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DeviceManager' of
type DeviceManager<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.395 [Main] INFO
LINSTOR/Satellite - Starting service instance
'DrbdEventPublisher-1' of type DrbdEventPublisher<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.397 [Main] INFO
LINSTOR/Satellite - Starting service instance 'DrbdEventService-1'
of type DrbdEventService<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.407 [Main] INFO
LINSTOR/Satellite - Starting service instance 'FileEventService'
of type FileEventService<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.408 [Main] INFO
LINSTOR/Satellite - Starting service instance 'TimerEventService'
of type TimerEventService<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.417 [Main] INFO
LINSTOR/Satellite - Initializing main network communications
service<br>
Jan 13 16:22:06 zern1 Satellite[20849]: 16:22:06.452 [Main] INFO
LINSTOR/Satellite - NetComService started on port
/0:0:0:0:0:0:0:0:3366<br>
Jan 13 16:37:22 zern1 Satellite[20849]: 16:37:22.139
[MainWorkerPool-1] INFO LINSTOR/Satellite - Controller connected
and authenticated (10.12.150.1:34792)<br>
Jan 13 16:37:22 zern1 Satellite[20849]: 16:37:22.193
[MainWorkerPool-2] INFO LINSTOR/Satellite - Node 'zern1' created.<br>
Jan 13 16:37:22 zern1 Satellite[20849]: 16:37:22.212
[MainWorkerPool-2] INFO LINSTOR/Satellite - Storage pool
'DfltDisklessStorPool' created.<br>
Jan 13 16:37:24 zern1 Satellite[20849]: 16:37:24.210
[MainWorkerPool-2] INFO LINSTOR/Satellite - Storage pool
'drbdpool' created.<br>
Jan 13 16:43:12 zern1 Satellite[20849]: 16:43:12.833
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'pve212'.<br>
Jan 13 16:43:12 zern1 Satellite[20849]: 16:43:12.835
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'pve213'.<br>
Jan 13 16:43:12 zern1 Satellite[20849]: 16:43:12.836
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-101-disk-1' created for node 'zern1'.<br>
Jan 13 16:43:42 zern1 Satellite[20849]: 16:43:42.116
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-111-disk-1' created for node 'zern1'.<br>
Jan 13 16:44:23 zern1 Satellite[20849]: 16:44:23.286
[MainWorkerPool-2] INFO LINSTOR/Satellite - Resource
'vm-111-disk-1' updated for node 'zern1'.<br>
<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Finally, the generated resource files:</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">PVE211</div>
<div class="moz-cite-prefix"># This file was generated by
linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-108-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "TqrARbxMBdQr2xjJ0Fu4";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on pve211<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-108-disk-1_00;<br>
meta-disk internal;<br>
device minor 110;<br>
}<br>
node-id 1;<br>
}<br>
}<br>
<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">PVE212</div>
<div class="moz-cite-prefix"><a class="moz-txt-link-abbreviated" href="mailto:root@pve212:/var/lib/linstor.d#">root@pve212:/var/lib/linstor.d#</a> cat
vm-101-disk-1.res<br>
# This file was generated by linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-101-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "cAKoGGuAxS2dGoih7ahn";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on pve212<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-101-disk-1_00;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 1;<br>
}<br>
<br>
on pve213<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 2;<br>
}<br>
<br>
on zern1<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 0;<br>
}<br>
<br>
connection<br>
{<br>
host pve212 address ipv4 10.12.150.2:7000;<br>
host pve213 address ipv4 10.12.150.3:7000;<br>
}<br>
<br>
connection<br>
{<br>
host pve212 address ipv4 10.12.150.2:7000;<br>
host zern1 address ipv4 10.12.150.4:7000;<br>
}<br>
}<br>
<a class="moz-txt-link-abbreviated" href="mailto:root@pve212:/var/lib/linstor.d#">root@pve212:/var/lib/linstor.d#</a> cat vm-102-disk-1.res<br>
# This file was generated by linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-102-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "CpY6iL4Mnh9QAX1bP3Dd";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on pve212<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-102-disk-1_00;<br>
meta-disk internal;<br>
device minor 109;<br>
}<br>
node-id 2;<br>
}<br>
<br>
on pve213<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 109;<br>
}<br>
node-id 1;<br>
}<br>
<br>
connection<br>
{<br>
host pve212 address ipv4 10.12.150.2:7001;<br>
host pve213 address ipv4 10.12.150.3:7001;<br>
}<br>
}<br>
<a class="moz-txt-link-abbreviated" href="mailto:root@pve212:/var/lib/linstor.d#">root@pve212:/var/lib/linstor.d#</a> cat vm-202-disk-1.res<br>
# This file was generated by linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-202-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "psvtT2unU5ZgFUvSWUxG";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on pve212<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-202-disk-1_00;<br>
meta-disk internal;<br>
device minor 115;<br>
}<br>
node-id 1;<br>
}<br>
}<br>
<a class="moz-txt-link-abbreviated" href="mailto:root@pve212:/var/lib/linstor.d#">root@pve212:/var/lib/linstor.d#</a><br>
<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">PVE213</div>
<div class="moz-cite-prefix"><a class="moz-txt-link-abbreviated" href="mailto:root@pve213:/var/lib/linstor.d#">root@pve213:/var/lib/linstor.d#</a> cat
vm-101-disk-1.res vm-102-disk-1.res<br>
# This file was generated by linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-101-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "cAKoGGuAxS2dGoih7ahn";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on pve213<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-101-disk-1_00;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 2;<br>
}<br>
<br>
on pve212<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 1;<br>
}<br>
<br>
on zern1<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 0;<br>
}<br>
<br>
connection<br>
{<br>
host pve213 address ipv4 10.12.150.3:7000;<br>
host pve212 address ipv4 10.12.150.2:7000;<br>
}<br>
<br>
connection<br>
{<br>
host pve213 address ipv4 10.12.150.3:7000;<br>
host zern1 address ipv4 10.12.150.4:7000;<br>
}<br>
}<br>
# This file was generated by linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-102-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "CpY6iL4Mnh9QAX1bP3Dd";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on pve213<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-102-disk-1_00;<br>
meta-disk internal;<br>
device minor 109;<br>
}<br>
node-id 1;<br>
}<br>
<br>
on pve212<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 109;<br>
}<br>
node-id 2;<br>
}<br>
<br>
connection<br>
{<br>
host pve213 address ipv4 10.12.150.3:7001;<br>
host pve212 address ipv4 10.12.150.2:7001;<br>
}<br>
}<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">ZERN1</div>
<div class="moz-cite-prefix"><a class="moz-txt-link-abbreviated" href="mailto:root@zern1:/var/lib/linstor.d#">root@zern1:/var/lib/linstor.d#</a> cat
vm-101-disk-1.res_tmp vm-111-disk-1.res_tmp <br>
# This file was generated by linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-101-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "cAKoGGuAxS2dGoih7ahn";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on zern1<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-101-disk-1_00;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 0;<br>
}<br>
<br>
on pve212<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 1;<br>
}<br>
<br>
on pve213<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbd/this/is/not/used;<br>
meta-disk internal;<br>
device minor 108;<br>
}<br>
node-id 2;<br>
}<br>
<br>
connection<br>
{<br>
host zern1 address ipv4 10.12.150.4:7000;<br>
host pve212 address ipv4 10.12.150.2:7000;<br>
}<br>
<br>
connection<br>
{<br>
host zern1 address ipv4 10.12.150.4:7000;<br>
host pve213 address ipv4 10.12.150.3:7000;<br>
}<br>
}<br>
# This file was generated by linstor(0.7.5), do not edit manually.<br>
<br>
resource "vm-111-disk-1"<br>
{<br>
template-file "linstor_common.conf";<br>
<br>
net<br>
{<br>
cram-hmac-alg sha1;<br>
shared-secret "LRZVSP38T/1GryvfghVn";<br>
allow-two-primaries yes;<br>
}<br>
<br>
on zern1<br>
{<br>
volume 0<br>
{<br>
disk /dev/drbdpool/vm-111-disk-1_00;<br>
meta-disk internal;<br>
device minor 113;<br>
}<br>
node-id 0;<br>
}<br>
}<br>
<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Il 16/01/19 07:13, Rene Peinthor ha
scritto:<br>
</div>
<blockquote type="cite"
cite="mid:CAHmn2DOWEE+m991RtWtjStm4kNB9fFPDsyw090O3i7v-QceUsw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Hi!
<div><br>
</div>
<div>We would need more info here, like if error-reports were
generate:</div>
<div>`linstor error-reports list`</div>
<div><br>
</div>
<div>and the output of those error reports</div>
<div>`linstor error-reports [ids...]`</div>
<div><br>
</div>
<div>if the first command didn't output anything, also check the
service looks on the satellites</div>
<div>`journalctl -u linstor-satellite`</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Rene</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Jan 14, 2019 at 10:09
AM Denis <<a href="mailto:drbdsys@made.net"
moz-do-not-send="true">drbdsys@made.net</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello,<br>
<br>
Yesterday i did try the migration on a 4 node cluster with
proxmox 5.1, <br>
latest version of drbd and drbdmanage.<br>
<br>
I installed linstor controller and satellite on all nodes and
activate <br>
controller on the first.<br>
<br>
I logged to the controller, created and reviewed dmmigrate.sh
and <br>
executed it.<br>
<br>
<br>
### Nodes ###<br>
linstor node create --node-type Combined pve212 10.12.150.2<br>
linstor node create --node-type Combined pve213 10.12.150.3<br>
linstor node create --node-type Combined pve211 10.12.150.1<br>
linstor node create --node-type Combined zern1 10.12.150.4<br>
<br>
### Storage ###<br>
linstor storage-pool-definition create drbdpool<br>
linstor storage-pool create lvm pve212 drbdpool drbdpool<br>
linstor storage-pool create lvm pve213 drbdpool drbdpool<br>
linstor storage-pool create lvm pve211 drbdpool drbdpool<br>
linstor storage-pool create lvm zern1 drbdpool drbdpool<br>
<br>
### Resource: vm-101-disk-1 ###<br>
linstor resource-definition create --port 7000 vm-101-disk-1<br>
linstor resource-definition drbd-options --allow-two-primaries
yes <br>
vm-101-disk-1<br>
linstor volume-definition create --vlmnr 0 --minor 108
vm-101-disk-1 <br>
52428800K<br>
linstor volume-definition set-property vm-101-disk-1 0
OverrideVlmId <br>
vm-101-disk-1_00<br>
linstor volume-definition set-property vm-101-disk-1 0 <br>
AllowLargerVolumeSize true<br>
linstor resource create --node-id 1 --storage-pool drbdpool
pve212 <br>
vm-101-disk-1<br>
linstor resource create --node-id 2 --storage-pool drbdpool
pve213 <br>
vm-101-disk-1<br>
linstor resource create --node-id 0 --storage-pool drbdpool
zern1 <br>
vm-101-disk-1<br>
<br>
<br>
The last command timed out after 5 minutes with no errors.<br>
<br>
Satellite is running on all 4 nodes.<br>
<br>
Then I tried to continue:<br>
<br>
### Resource: vm-108-disk-1 ###<br>
linstor resource-definition create --port 7002 vm-108-disk-1<br>
linstor resource-definition drbd-options --allow-two-primaries
yes <br>
vm-108-disk-1<br>
linstor volume-definition create --vlmnr 0 --minor 110
vm-108-disk-1 <br>
52428800K<br>
linstor volume-definition set-property vm-108-disk-1 0
OverrideVlmId <br>
vm-108-disk-1_00<br>
linstor volume-definition set-property vm-108-disk-1 0 <br>
AllowLargerVolumeSize true<br>
linstor resource create --node-id 1 --storage-pool drbdpool
pve211 <br>
vm-108-disk-1<br>
<br>
...timed out after 5 minutes...<br>
<br>
I made other steps with same result and then rolled back to
drbdmanage.<br>
<br>
BTW: I did see resize and UUID modification during migration
process, <br>
can it be dangerous restoring old drbdmanage .res files?<br>
<br>
<br>
If you need some other information, please tell me.<br>
<br>
I have drbdmange and linstor running now, but linstor
resourced are not <br>
included by drbd and proxmox still uses drbdmanage plugin<br>
<br>
<br>
Thank you,<br>
<br>
Denis<br>
<br>
<br>
_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com" target="_blank"
moz-do-not-send="true">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user"
rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
</blockquote>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>